Formation conditions of mutual adaptation in human-agent collaborative interaction

When an adaptive agent works with a human user in a collaborative task, in order to enable flexible instructions to be issued by ordinary people, it is believed that a mutual adaptation phenomenon can enable the agent to handle flexible mapping relations between the human user’s instructions and the agent’s actions. To elucidate the conditions required to induce the mutual adaptation phenomenon, we designed an appropriate experimental environment called “WAITER” (Waiter Agent Interactive Training Experimental Restaurant) and conducted two experiments in this environment. The experimental results suggest that the proposed conditions can induce the mutual adaptation phenomenon.     

AR-based interaction for human-robot collaborative manufacturing

Industrial standards define safety requirements for Human-Robot Collaboration (HRC) in industrial manufacturing. The standards particularly require real-time monitoring and securing of the minimum protective distance between a robot and an operator. This paper proposes a depth-sensor based model for workspace monitoring and an interactive Augmented Reality (AR) User Interface (UI) for safe HRC. The AR UI is implemented on two different hardware: a projector-mirror setup and a wearable AR gear (HoloLens). The workspace model and UIs are evaluated in a realistic diesel engine assembly task. The AR-based interactive UIs provide 21–24% and 57–64% reduction in the task completion and robot idle time, respectively, as compared to a baseline without interaction and workspace sharing. However, user experience assessment reveal that HoloLens based AR is not yet suitable for industrial manufacturing while the projector-mirror setup shows clear improvements in safety and work ergonomics.     

Developing an emotional-based application for human-agent societies

The purpose of this paper is to present an emotional-based application for human-agent societies. This kind of applications are those where virtual agents and humans coexist and interact transparently into a fully integrated environment. Specifically, the paper presents an application where humans are immersed into a system that extracts and analyzes the emotional states of a human group trying to maximize the welfare of those humans by playing the most appropriate music in every moment. This system can be used not only online, calculating the emotional reaction of people in a bar to a new song, but also in simulation, to predict the people’s reaction to changes in music or in the bar layout.     

Probing changes in neural interaction during adaptation

A procedure is developed to probe the changes in the functional interactions among neurons in primary motor cortex of the monkey brain during adaptation. A monkey is trained to learn a new skill, moving its arm to reach a target under the influence of external perturbations. The spike trains of multiple neurons in the primary motor cortex are recorded simultaneously. We utilize the methodology of directed transfer function, derived from a class of linear stochastic models, to quantify the causal interactions between the neurons. We find that the coupling between the motor neurons tends to increase during the adaptation but return to the original level after the adaptation. Furthermore, there is evidence that adaptation tends to affect the topology of the neural network, despite the approximate conservation of the average coupling strength in the network before and after the adaptation.     

Using emotions for the development of human-agent societies

Human-agent societies refer to applications where virtual agents and humans coexist and interact transparently into a fully integrated environment. One of the most important aspects in this kind of applications is including emotional states of the agents (humans or not) in the decision-making process. In this sense, this paper presents the applicability of the JaCalIVE (Jason Cartago implemented intelligent virtual environment) framework for developing this kind of society. Specifically, the paper presents an ambient intelligence application where humans are immersed into a system that extracts and analyzes the emotional state of a human group. A social emotional model is employed to try to maximize the welfare of those humans by playing the most appropriate music in every moment.     

一种行为驱动的构件交互适配模型

软件构件化是提高软件开发效率和降低重复性劳动的重要途径之一,但一直面临着构件实体之间的行为交互失配问题。对此,构造了一个构件行为本体,其中包括构件体、构件行为及构件行为性质,并根据构件行为本体提出了构件交互适配模型。在该模型中,依据构件行为性质将构件集合划分成不同的构件组,利用构件组内复合行为的完整性探测构件交互死锁,并通过自定义行为规则适配构件交互死锁。最后,对构件组与适配器之间的一一对应关系进行证明。     

融合交互意图的图神经网络协同过滤算法

传统的协同过滤方法很大程度上以一种统一的方式进行建模,未从交互意图的细粒度上考虑建模关系。为此提出一种融合交互意图的图神经网络协同过滤算法（INTNGCF）。首先,通过将边分解成多个潜在空间来识别潜在意图;其次,利用交互意图融合层确定这些潜在意图的重要性;最后,生成用户对项目的预测评分。在三个真实数据集上与七种基线模型进行对比实验,结果表明,提出的算法与其他先进的推荐模型相比具有一定的性能优势。     

Analysis of interaction dynamics in collaborative and distributed design process

This paper proposes a computational interaction analysis approach for extraction of emergent patterns and organization structures during collaborative and distributed design process based on its emergent state-problems. A better source of information on both collaborative and distributed design process as a time-based activity and the agent-agent interactions during the design process will allow design researchers to develop richer models of designing which in turn will provide the basis for a better understanding of collaborative and distributed design process and developing intelligent tools to support this process. Therefore, the proposed approach consists of discerning from the real interactions the different state-problems characterizing the mentioned process. The modelling of interactions corresponding to state-problems, permitted to observe the emergent patterns of collaborative and distributed design process: micro-groups and pivot agent. The collaborative and distributed design process is characterized by the new micro-group formation on the one hand, and by their evolution in the time on the other. Based on the properties of interaction, the approach has discerned three types of cooperation: complete, bilateral and quasi null. Results from some real collaborative and distributed design process, allow us to observe certain properties related to micro-groups, such as self-organization, dynamics and self-similarity. Self-organization results from interaction among adaptive human agents. It is an emergent structuration in response to non-linear collaborative and distributed design process. Dynamics results from the variation of micro-groups formation and the collaborative and distributed design process seems to articulate around one or several pivot agents. Self-similarity results from the similar emergent pattern in every discussion. This research served as core for developments of multi-agents system ISIAD (Intelligent System for Interactions Analysis in Design).     

Group mirrors to support interaction regulation in collaborative problem solving

Two experimental studies test the effect of group mirrors upon quantitative and qualitative aspects of participation in collaborative problem solving. Mirroring tools consist of a graphical representation of the group’s actions which is dynamically updated and displayed to the collaborators. In addition, metacognitive tools display a standard for desirable behavior. Results show that a mirroring tool did not substantively affect the behavior of subjects while a metacognitive tool led to increased participation in dialogue, including more frequent and precise planning.     

区分交互意图的图卷积协同过滤算法

近几年提出了一些基于图卷积网络的协同过滤推荐模型,然而大部分模型将邻域权重视为常量且不区分用户和物品间的交互关系,无法获取令用户满意的推荐列表。因此,为了得到用户和物品更准确的嵌入表示,提出一种区分交互意图的图卷积协同过滤推荐算法MiGCCF(multi-intention graph convolutional collaborative filtering)。该算法将交互关系进行分解,细粒度分析用户与物品间的交互意图,并引入注意力机制,在消息传播过程中赋予邻域可学习的注意力权重,挖掘用户对于不同交互物品的喜爱度。在Gowalla与Amazon-book上的实验表明,该算法相比于基准算法,在两个数据集上的HR@50和NDCG@50指标分别提高了12.5%和8.5%,具有更好的性能表现。     

Active domain adaptation with noisy labels for multimedia analysis

Supervised learning methods require sufficient labeled examples to learn a good model for classification or regression. However, available labeled data are insufficient in many applications. Active learning (AL) and domain adaptation (DA) are two strategies to minimize the required amount of labeled data for model training. AL requires the domain expert to label a small number of highly informative examples to facilitate classification, while DA involves tuning the source domain knowledge for classification on the target domain. In this paper, we demonstrate how AL can efficiently minimize the required amount of labeled data for DA. Since the source and target domains usually have different distributions, it is possible that the domain expert may not have sufficient knowledge to answer each query correctly. We exploit our active DA framework to handle incorrect labels provided by domain experts. Experiments with multimedia data demonstrate the efficiency of our proposed framework for active DA with noisy labels.     

Digital twins for collaborative robots: A case study in human-robot interaction

Human-robot collaboration (HRC) can expand the level of automation in areas that have conventionally been difficult to automate such as assembly. However, the need of adaptability and the dynamics of human presence are keeping the full potential of human-robot collaborative systems difficult to achieve. This paper explores the opportunities of using a digital twin to address the complexity of collaborative production systems through an industrial case and a demonstrator. A digital twin, as a virtual counterpart of a physical human-robot assembly system, is built as a ‘front-runner’ for validation and control throughout its design, build and operation. The forms of digital twins along system's life cycle, its building blocks and the potential advantages are presented and discussed. Recommendations for future research and practice in the use of digital twins in the field of cobotics are given.     

Social and collaborative aspects of interaction with a service robot

To an increasing extent, robots are being designed to become a part of the lives of ordinary people. This calls for new models of the interaction between humans and robots, taking advantage of human social and communicative skills. Furthermore, human–robot relationships must be understood in the context of use of robots, and based on empirical studies of humans and robots in real settings. This paper discusses social aspects of interaction with a service robot, departing from our experiences of designing a fetch-and-carry robot for motion-impaired users in an office environment. We present the motivations behind the design of the Cero robot, especially its communication paradigm. Finally, we discuss experiences from a recent usage study, and research issues emerging from this work. A conclusion is that addressing only the primary user in service robotics is unsatisfactory, and that the focus should be on the setting, activities and social interactions of the group of people where the robot is to be used.     

Carpeno: interfacing remote collaborative virtual environments with table-top interaction

Creativity is enhanced by communication and collaboration. Thus, the increasing number of distributed creative tasks requires better support from computer-mediated communication and collaborative tools. In this paper we introduce “Carpeno”, a new system for facilitating intuitive face-to-face and remote collaboration on creative tasks. Normally the most popular and efficient way for people to collaborate is face-to-face, sitting around a table. Computer augmented surface environments, in particular interactive table-top environments, are increasingly used to support face-to-face meetings. They help co-located teams to develop new ideas by facilitating the presentation, manipulation, and exchange of shared digital documents displayed on the table-top surface. Users can see each other at the same time as the information they are talking about. In this way the task space and communication space can be brought together in a more natural and intuitive way. The discussion of digital content is redirected from a computer screen, back to a table where people can gather around. In contrast, collaborative virtual environments (CVE) are used to support remote collaboration. They frequently create familiar discussion scenarios for remote interlocutors by utilizing room metaphors. Here, virtual avatars and table metaphors are used, where the participants can get together and communicate with each other in a way that allows behaviour that is as close to face-to-face collaboration as possible. The Carpeno system described here combines table-top interaction with a CVE to support intuitive face-to-face and remote collaboration. This allows for simultaneous co-located and remote collaboration around a common, interactive table.

Virtual interpersonal touch: Haptic interaction and copresence in collaborative virtual environments

As digital communication becomes more commonplace and sensory rich, understanding the manner in which people interact with one another is crucial. In the current study, we examined the manners in which people touch digital representations of people, and compared those behaviors to the manner in which they touch digital representations of nonhuman objects. Results demonstrated that people used less force when touching people than other nonhuman objects, and that people touched the face with less force than the torso area. Finally, male digital representations were touched with more force than female representations by subjects of both genders. We discuss the implications of these data to the development of haptic communication systems as well as for a methodology of measuring the amount of copresence in virtual environments.

The effects of animations on verbal interaction in computer supported collaborative learning

Abstract This paper focuses on the interaction patterns of learners studying in pairs who were provided with multimedia learning material. In a previous article, we reported that learning scores were higher for dyads of an ‘animations’ condition than for dyads of a ‘static pictures’ condition. Results also showed that offering a persistent display of one snapshot of each animated sequence hindered collaborative learning. In the present paper, further analyses of verbal interactions within learning dyads were performed in order to have a better understanding of both the beneficial effect of animations and the detrimental effect of the presence of persistent snapshots of critical steps on collaborative learning. Results did not show any differences in terms of verbal categories between the two versions of the instructional material, that is, static versus animated pictures. Pairs who were provided with persistent snapshots of the multimedia sequences produced fewer utterances compared to participants without the snapshots. In addition, the persistent snapshots were detrimental both in terms of providing information about the learning content and in terms of producing utterances solely for the purpose of managing the interaction. In this study, evidence also showed that these two verbal categories were positively related to learning performances. Finally, mediation analyses revealed that the negative effect of persistent snapshots was mediated by the fact that peers of the snapshots condition produced less information providing and interaction management utterances. Results are interpreted using a psycholinguistic framework applied to computer‐supported collaborative learning (CSCL) literature and general guidelines are derived for the use of dynamic material and persistency tools in the design of CSCL environments.     

投入驱动的存储与计算一体化的事务处理效率优化方法

事务处理技术是报告信息一致性和可靠性的关键技术,决定了Web服务是否可以应用于电子商务。类型化的数据、信息和知识等资源繁杂冗余,导致资源存储和处理效率低下,长事务的处理过程往往会持续较长时间,使得锁定资源的策略不能一直适用,为了协调事务型资源存储和计算代价,提出了一种投入驱动的事务处理方法。从资源建模、资源处理、处理优化和资源管理等角度进行研究,基于对现有知识图谱概念的拓展提出了一种三层可自动抽象调整的解决架构。这个架构包括：数据图谱、信息图谱和知识图谱等三个层面,关键在于对搜索目标资源对象类型转移代价和在资源存储空间上的存储代价的计算,并根据用户投入协同调整搜索目标资源对象的搜索机制和存储方案,从而降低资源搜索的时间复杂度和资源存储的空间复杂度,优化事务处理的时空效率。     

User interaction in digital libraries: coping with diversity through adaptation

User interface adaptations can be used to address several user interaction challenges in the development of digital library systems. This paper: (a) examines some of the intrinsic characteristics of digital library systems; (b) identifies some of the key Human–Computer Interaction (HCI) challenges in relation to design and architectural abstraction for user interfaces to digital library systems; and (c) develops an argumentation for adaptations in digital library systems. By drawing parallels to recent HCI research into adaptable and adaptive user interaction, the paper illustrates potential areas in which user interface adaptation can provide a useful technique for advancing the quality of human interaction with a digital library system. Received: 18 December 1998 / Revised: 31 May 1999     

Ten challenges for making automation a "team player" in joint human-agent activity

We propose 10 challenges for making automation components into effective "team players" when they interact with people in significant ways. Our analysis is based on some of the principles of human-centered computing that we have developed individually and jointly over the years, and is adapted from a more comprehensive examination of common ground and coordination.