Let's talk! Socially intelligent agents for language conversationtraining

This paper promotes socially intelligent animated agents for the pedagogical task of English conversation training for native speakers of Japanese. As a novel feature, social role awareness is introduced to animated conversational agents, that are by non-strong affective reasoners, but otherwise often lack the social competence observed in humans. In particular, humans may easily adjust their behavior depending on their respective role in a social setting, whereas their synthetic pendants tend to be driven mostly by emotions and personality. Our main contribution is the incorporation of a “social filter program” to mental models of animated agents. This program may qualify an agent's expression of its emotional state by the social contest, thereby enhancing the agent's believability as a conversational partner. Our implemented system is web-based and demonstrates socially aware animated agents in a virtual coffee shop environment. An experiment with our conversation system shows that users consider socially aware agents as more natural than agents that violate conventional practices     

Scaled consensus for asynchronous high‐order discrete‐time multiagent systems

This paper investigates the distributed scaled consensus problem of multiple agents with high‐order dynamics under the asynchronous setting, where each agent measures the neighbors' information at certain discrete time instants according to its own clock rather than the whole discrete process and all agents' clocks are independent of each other. Assume that the communication topology can be arbitrarily switched and the information transfer between agents has a time‐varying delay. Under the designed asynchronous distributed control protocol, it is shown that the agents with the same scale will reach a common final state, while the agents with different scales will reach different final states. Moreover, an effective parameters selection strategy is presented for a large number of gain parameters in high‐order multiagent systems based on novel model transformation techniques. Simulation examples are provided to demonstrate the high‐order scaled consensus performances for the agents in the presence of asynchronous setting.     

Conversation with and through computers

People design what they say specifically for their conversational partners, and they adapt to their partners over the course of a conversation. A comparison of keyboard conversations involving a simulated computer partner (as in a natural language interface) with those involving a human partner (as in teleconferencing) yielded striking differences and some equally striking similarities. For instance, there were significantly fewer acknowledgments in human/computer dialogue than in human/human. However, regardless of the conversational partner, people expected connectedness across conversational turns. In addition, the style of a partner's response shaped what people subsequently typed. These results suggest some issues that need to be addressed before a natural language computer interface will be able to hold up its end of a conversation.     

Remote Conversations: The Effects of Mediating Talk With Technology

Three different videoconferencing systems for supporting multiparty, remote conversations are described and evaluated experimentally. The three systems differed by how many participants were visible at once, their spatial arrangement, and control over who was seen. Conversations using these systems were compared to same-room (Experiment 1) and audio-only (Experiment 2) conversations. Specialized speech-tracking equipment recorded the on-off patterns of speech that allowed objective measurement of structural aspects of the conversations, such as turn length, pauses, and interruptions. Questionnaires and interviews also documented participants' opinions and perceptions in the various settings. Contrary to expectation, systems in which visual cues such as selective gaze were absent produced no differences in turn-taking or in any other aspect of the structure of conversation. In fact, turn-taking was unaffected even when visual information was completely absent. Overall, only the same-room condition showed any significant differences from any other condition; people in the same room produced more interruptions and fewer formal handovers of the floor than in any of the technology-mediated conditions. In this respect, the audio-only and video systems examined in these studies were equivalent. However, analyses of participants' perceptions showed that participants felt that visual access in mediated conversations was both important and beneficial in conversation. Further, there were indications that the particular design of the different video systems did affect some aspects of conversational behavior, such as the ability to hold side and parallel conversations.     

Collaborative conversational interfaces

This paper proposes a method of designing human-computer speech interfaces based on principles of human conversation. It argues that conversation is the primary mode of language use and that it is fundamentally collaborative. Speech interfaces should therefore be designed to recreate the collaborative nature of natural conversations. The paper presents five strategies for designingcollaborative conversational interfaces, and it describes the principles of human-language use that underly these strategies. The paper also argues that collaborative conversational interfaces have a crucial advantage over other kinds of interfaces in that they are readily adaptive to different levels of experience and styles of use. The paper gives examples of collaborative conversational interfaces that we have developed, and discusses the ways in which these interfaces have been made adaptive.     

Leader‐following consensus for nonlinear agents with measurement feedback

The leader‐following consensus problem is investigated for large classes of nonlinear identical agents. Sufficient conditions are provided for achieving consensus via state and measurement feedback laws based on a local (ie, among neighbors) information exchange. The leader's trajectories are assumed bounded without knowledge of the containing compact set and the agents' trajectories possibly unbounded under the action of a bounded input. Generalizations to heterogeneous agents and robustness are also discussed.     

ConToVi: Multi‐Party Conversation Exploration using Topic‐Space Views

We introduce a novel visual analytics approach to analyze speaker behavior patterns in multi‐party conversations. We propose Topic‐Space Views to track the movement of speakers across the thematic landscape of a conversation. Our tool is designed to assist political science scholars in exploring the dynamics of a conversation over time to generate and prove hypotheses about speaker interactions and behavior patterns. Moreover, we introduce a glyph‐based representation for each speaker turn based on linguistic and statistical cues to abstract relevant text features. We present animated views for exploring the general behavior and interactions of speakers over time and interactive steady visualizations for the detailed analysis of a selection of speakers. Using a visual sedimentation metaphor we enable the analysts to track subtle changes in the flow of a conversation over time while keeping an overview of all past speaker turns. We evaluate our approach on real‐world datasets and the results have been insightful to our domain experts.     

Analyzing conversations of Web services

A composite Web service consists of a set of individual services (or peers), which interact with each other via messages. A conversation is a global sequence of messages exchanged among peers participating in a composite Web service. Interestingly, conversation behavior differs significantly for synchronous and asynchronous communication, even if peers' local behaviors remain the same. The authors use synchronizability and realizability analysis techniques to analyze conversations in the presence of asynchronous communication. With these analyses, they can use model checking to automatically verify conversation behavior for bottom-up and top-down Web service specifications.     

Pauses in doctor–patient conversation during computer use: The design significance of their durations and accompanying topic changes

Talk is often suspended during medical consultations while the clinician interacts with the patient's records and other information. This study of four general practitioners (GPs) focused on these suspensions and the adjacent conversational turns. Conversation analysis revealed how GPs took action to close conversations down prior to attending to the records, resulting in a ‘free turn’ that could be taken up by either GP or patient. The durations of the intervening pauses were also analysed, exposing a hitherto unobserved 10-second timeframe within which both GP and patient showed a preference for the conversation to be resumed. Resumption was more likely to be achieved within 10 s when the GP's records were paper-based rather than computer-based. Subsequent analysis of topic changes on resumption of talk has revealed a 5-second timeframe, also undocumented; when pauses exceed this timeframe, it is rare for the previous topic to be resumed without a restatement. Data recorded in the home suggest that these timeframes are also present in family conversations. We argue for considering the two timeframes when designing systems for use in medical consultations and other conversational settings, and discuss possible outcomes.     

Output feedback distributed optimization algorithms of second-order Lipschitz nonlinear multi-agent systems

This paper introduces output feedback distributed optimization algorithms designed specifically for second-order nonlinear multi-agent systems. The agents are allowed to have heterogeneous dynamics, characterized by distinct nonlinearities, as long as they satisfy the Lipschitz continuity condition. For the case with unknown states, nonlinear state observers are designed first for each agent to reconstruct agents' unknown states. It is proven that the agents' unknown states are estimated accurately by the developed state observers. Then, based on the agents' state estimates and the gradient of each agent local cost function, a kind of output feedback distributed optimization algorithms are proposed for the considered multi-agent systems. Under the proposed distributed optimization algorithms, all the agents' outputs asymptotically approach the minimizer of the global cost function which is the sum of all the local cost functions. By using Lyapunov stability theory, convex analysis, and input-to-state stability theory, the asymptotical convergence of the output feedback distributed optimization closed-loop system is proven. Simulations are conducted to validate the efficacy of the proposed algorithms.     

Conversational transfer learning for emotion recognition

Recognizing emotions in conversations is a challenging task due to the presence of contextual dependencies governed by self- and inter-personal influences. Recent approaches have focused on modeling these dependencies primarily via supervised learning. However, purely supervised strategies demand large amounts of annotated data, which is lacking in most of the available corpora in this task. To tackle this challenge, we look at transfer learning approaches as a viable alternative. Given the large amount of available conversational data, we investigate whether generative conversational models can be leveraged to transfer affective knowledge for detecting emotions in context. We propose an approach, TL-ERC, where we pre-train a hierarchical dialogue model on multi-turn conversations (source) and then transfer its parameters to a conversational emotion classifier (target). In addition to the popular practice of using pre-trained sentence encoders, our approach also incorporates recurrent parameters that model inter-sentential context across the whole conversation. Based on this idea, we perform several experiments across multiple datasets and find improvement in performance and robustness against limited training data. TL-ERC also achieves better validation performances in significantly fewer epochs. Overall, we infer that knowledge acquired from dialogue generators can indeed help recognize emotions in conversations.     

多主体会话策略研究

用交互协议对主体之间的交互进行抽象的研究已经好多年了。然而,主体间会话的效率很低,而且会话的语义冲突问题一直没有解决。本文分析了主体之间通信协议和会话策略的关系,利用引入了本体的Dooley图描述出了主体间会话策略,并通过实验对主体选择不同的会话策略做了比较,其意义在于提高主体之间消除语义冲突的会话的效率,降低会话成本。     

Robust containment control in a leader–follower network of uncertain Euler–Lagrange systems

A distributed controller is developed that yields cooperative containment control of a network of autonomous dynamical systems. The networked agents are modeled with uncertain nonlinear Euler–Lagrange dynamics affected by an unknown time‐varying exogenous disturbance. The developed continuous controller is robust to input disturbances and uncertain dynamics such that asymptotic convergence of the follower agents' states to the dynamic convex hull formed by the leaders' time‐varying states is achieved. Simulation results are provided to demonstrate the effectiveness of the developed controller. Copyright © 2016 John Wiley & Sons, Ltd.     

Specification of realizable service conversations using collaboration diagrams

Specification, modeling and analysis of interactions among peers that communicate via messages are becoming increasingly important due to the emergence of service oriented computing. Collaboration diagrams provide a convenient visual model for specifying such interactions. An interaction among a set of peers can be characterized as a conversation. A conversation is the global sequence of messages exchanged among the peers, listed in the order they are sent. A collaboration diagram can be used to specify the set of allowable conversations among the peers participating to a composite web service. Specification of interactions from a global perspective leads to the realizability problem: Is it possible to construct a set of peers that generate exactly the specified set of conversations? In this paper, we investigate the realizability of conversations specified by collaboration diagrams. We formalize the realizability problem by modeling peers as concurrently executing finite state machines, and we give sufficient realizability conditions for a class of collaboration diagrams.     

Reading dynamically displayed text

Sixty four members of the public were exposed to the same staged conversation either while waiting in a bus station or travelling on a train. Half of the conversations were by mobile phone, so that only one end of the conversation was heard, and half were co present face-to-face conversations. The volume of the conversations was controlled at one of two levels: the actors' usual speech level and exaggeratedly loud. Following exposure to the conversation participants were approached and asked to give verbal ratings on six scales. Analysis of variance showed that mobile phone conversations were significantly more noticeable and annoying than face-to-face conversations at the same volume when the content of the conversation is controlled. Indeed this effect of medium was as large as the effect of loudness. Various explanations of this effect are explored, with their practical implications.     

To Commit or Not to Commit: Modeling Agent Conversations for Action

Conversations are sequences of messages exchanged among interacting agents. For conversations to be meaningful, agents ought to follow commonly known specifications limiting the types of messages that can be exchanged at any point in the conversation. These specifications are usually implemented using conversation policies (which are rules of inference) or conversation protocols (which are predefined conversation templates). In this article we present a semantic model for specifying conversations using conversation policies. This model is based on the principles that the negotiation and uptake of shared social commitments entail the adoption of obligations to action, which indicate the actions that agents have agreed to perform. In the same way, obligations are retracted based on the negotiation to discharge their corresponding shared social commitments. Based on these principles, conversations are specified as interaction specifications that model the ideal sequencing of agent participations negotiating the execution of actions in a joint activity. These specifications not only specify the adoption and discharge of shared commitments and obligations during an activity, but also indicate the commitments and obligations that are required (as preconditions) or that outlive a joint activity (as postconditions). We model the Contract Net Protocol as an example of the specification of conversations in a joint activity.     

Specifying and animating facial signals for discourse in embodied conversational agents

People highlight the intended interpretation of their utterances within a larger discourse by a diverse set of non‐verbal signals. These signals represent a key challenge for animated conversational agents because they are pervasive, variable, and need to be coordinated judiciously in an effective contribution to conversation. In this paper, we describe a freely available cross‐platform real‐time facial animation system, RUTH , that animates such high‐level signals in synchrony with speech and lip movements. RUTH adopts an open, layered architecture in which fine‐grained features of the animation can be derived by rule from inferred linguistic structure, allowing us to use RUTH , in conjunction with annotation of observed discourse, to investigate the meaningful high‐level elements of conversational facial movement for American English speakers. Copyright © 2004 John Wiley & Sons, Ltd.     

Toward Spoken Human–Computer Tutorial Dialogues

Oral discourse is the primary form of human–human communication, hence, computer interfaces that communicate via unstructured spoken dialogues will presumably provide a more efficient, meaningful, and naturalistic interaction experience. Within the context of learning environments, there are theoretical positions supporting a speech facilitation hypothesis that predicts that spoken tutorial dialogues will increase learning more than typed dialogues. We evaluated this hypothesis in an experiment where 24 participants learned computer literacy via a spoken and a typed conversation with AutoTutor, an intelligent tutoring system with conversational dialogues. The results indicated that (a) enhanced content coverage was achieved in the spoken condition; (b) learning gains for both modalities were on par and greater than a no-instruction control; (c) although speech recognition errors were unrelated to learning gains, they were linked to participants' evaluations of the tutor; (d) participants adjusted their conversational styles when speaking compared to typing; (e) semantic and statistical natural language understanding approaches to comprehending learners' responses were more resilient to speech recognition errors than syntactic and symbolic-based approaches; and (f) simulated speech recognition errors had differential impacts on the fidelity of different semantic algorithms. We discuss the impact of our findings on the speech facilitation hypothesis and on human–computer interfaces that support spoken dialogues.     

基于非光滑采样控制算法的二阶有向多智能体系统的一致性

针对二阶有向多智能体系统的一致性问题,在连续时间域和离散时间域分别提出一种非光滑控制协议.首先,提出一种连续时间非光滑一致性协议,通过李雅普诺夫理论和齐次系统理论,证明在无扰动情况下可以实现智能体状态的有限时间一致,而在有扰动情况下智能体状态之间的误差将收敛到一个与控制参数和外部扰动相关的范围内;然后,基于采样控制,提出离散非光滑一致性协议,进一步分析采样周期对智能体状态之间误差的影响,并给出误差收敛区间与控制参数、外部扰动和采样周期关系的显性表达式;最后,通过仿真实例验证理论的正确性和有效性.