Interaction With Robots: Physical Constraints on the Interpretation of Demonstrative Pronouns

This study investigated what effect physical constraints have on the interpretation of demonstrative pronouns when a user navigates a robot. For this investigation, a robot navigation environment called Spondia-II was develope, and an experiment conducted. It is known that the interpretation of demonstrative pronouns requires information about not only the situation (or context) but also the speaker's viewpoint during a dialogue. The results of the experiment suggest that physical constraints do affect the user's viewpoint, especially when a user utters a demonstrative pronoun while navigating the robot. In actual fact, the user alters the use of demonstrative pronouns according to the change in the user's viewpoint. It is also suggested that the user and the robot share the same viewpoint during the physical interaction.     

Level of interest sensing in spoken dialog using decision-level fusion of acoustic and lexical evidence

Automatic detection of a user's interest in spoken dialog plays an important role in many applications, such as tutoring systems and customer service systems. In this study, we propose a decision-level fusion approach using acoustic and lexical information to accurately sense a user's interest at the utterance level. Our system consists of three parts: acoustic/prosodic model, lexical model, and a model that combines their decisions for the final output. We use two different regression algorithms to complement each other for the acoustic model. For lexical information, in addition to the bag-of-words model, we propose new features including a level-of-interest value for each word, length information using the number of words, estimated speaking rate, silence in the utterance, and similarity with other utterances. We also investigate the effectiveness of using more automatic speech recognition (ASR) hypotheses (n-best lists) to extract lexical features. The outputs from the acoustic and lexical models are combined at the decision level. Our experiments show that combining acoustic evidence with lexical information improves level-of-interest detection performance, even when lexical features are extracted from ASR output with high word error rate.     

基于用户兴趣模型的推荐算法

传统的协同过滤算法虽然可以很容易地挖掘出用户的兴趣爱好,但存在数据冷启动和稀疏性问题.针对这些问题,提出一种基于用户兴趣模型的推荐算法.首先通过LDA主题模型训练数据集得到物品-主题概率分布矩阵,利用物品-主题概率分布矩阵得到用户历史兴趣模型,然后结合用户历史行为信息和物品内容信息得到用户兴趣模型,最后计算用户与候选集之间的相似度,进行TOP-N推荐.在豆瓣电影数据集上的实验结果表明,改进后的推荐算法能够更好地处理稀疏数据和冷启动问题,并且明显提高了推荐质量.     

Virtual Force/Tactile Sensors for Interactive Machines Using the User's Biological Signals

This study proposes a new approach to virtual realization of force/tactile sensors in machines equipped with no real sensors. The key of our approach is that a machine exploits the user's biological signals. Therefore, this approach is not dependent on controlled objects and is expected to be widely applicable for a variety of machines including robots. This article describes an example robotic system comprised of an industrial robot manipulator, a motion capture system and a surface electromyogram (EMG) measurement apparatus. By monitoring/recording the user's surface EMG and postural information in real-time, we show that a robot equipped with no force/tactile sensors behaved similarly to one possessing sensors over its body. Another advantage of our approach is demonstrated by a task in which a robot and a user cooperatively hold and move a heavy load.     

Expressing reactive emotion based on multimodal emotion recognition for natural conversation in human–robot interaction*

Human–human interaction consists of various nonverbal behaviors that are often emotion-related. To establish rapport, it is essential that the listener respond to reactive emotion in a way that makes sense given the speaker's emotional state. However, human–robot interactions generally fail in this regard because most spoken dialogue systems play only a question-answer role. Aiming for natural conversation, we examine an emotion processing module that consists of a user emotion recognition function and a reactive emotion expression function for a spoken dialogue system to improve human–robot interaction. For the emotion recognition function, we propose a method that combines valence from prosody and sentiment from text by decision-level fusion, which considerably improves the performance. Moreover, this method reduces fatal recognition errors, thereby improving the user experience. For the reactive emotion expression function, the system's emotion is divided into emotion category and emotion level, which are predicted using the parameters estimated by the recognition function on the basis of distributions inferred from human–human dialogue data. As a result, the emotion processing module can recognize the user's emotion from his/her speech, and expresses a reactive emotion that matches. Evaluation with ten participants demonstrated that the system enhanced by this module is effective to conduct natural conversation.     

基于优化“未定义”类话语检测的话语领域分类

“未定义”类话语在面向任务的对话语料中广泛存在,具有成分复杂,与其余“已定义”类话语边界模糊的特点,影响着话语领域的分类总体正确率。“未定义”类话语一旦错分,将会使用户对口语对话系统的功能有效性产生怀疑,导致大大降低用户体验。该文提出一种基于优化“未定义”类话语检测的领域分类方案,采用两阶段法完成口语话语的领域分类任务。首先,采用聚类方法将“已定义”类话语聚为几个大类,简化众多的“已定义”类话语独立存在时与“未定义”类话语之间的边界。进而利用分类模型对聚类后的“已定义”类话语大类以及“未定义”类话语进行领域分类,优化目标是“未定义”类话语的检测效率。最后,将第一阶段分类为“已定义”类的话语,在去除了绝大部分“未定义”类话语干扰的基础上进行再次分类。该文的分类模型采用了深度学习模型LSTM,并利用无标签微博数据训练词向量用于话语特征表达。在SMP 2017 意图领域分类比赛的多任务语料上的评测结果表明,该方案在 “未定义”类话语检测的F1值以及所有话语的领域分类总正确率上均有明显提升。     

基于文本聚类与兴趣衰减的微博用户兴趣挖掘方法

微博平台隐含潜在的用户信息,通过微博数据挖掘用户兴趣具有重要的社会意义。结合用户兴趣与微博信息的特点,提出了一种文本聚类与兴趣衰减的微博用户兴趣挖掘（TCID-MUIM）方法。首先,通过基于词林的同义词合并策略弥补建模时词频信息不足的弊端;其次,利用二次Single-Pass不完全聚类算法将用户微博划分为多个簇,将簇合并为同一文档以弥补微博文本短小难以挖掘主题信息的问题;最后,通过LDA模型建模,并考虑用户兴趣随时间变化的问题,引入时间因子,将微博—主题矩阵压缩为用户—主题矩阵,获取用户兴趣。实验表明,较之传统建模方法与合并用户历史微博为同一文档的建模方法,TCID-MUIM方法挖掘的用户兴趣主题具有更好的主题区分度,且更贴合用户的真实兴趣偏好。     

基于情景和浏览内容的层次性用户兴趣建模

用户兴趣建模是个性化服务的核心,考虑到情景信息对用户偏好的影响,对融和情景信息的用户行为日志数据进行深入研究,提出了一种基于情景信息的用户兴趣建模方法.该方法首先通过计算情景相似度来获得用户当前情景的近似情景集;对“用户-兴趣项-情景”三维模型采用情景预过滤的方法降维处理.然后根据用户浏览内容得到用户兴趣主题,分析页面内容得到每种主题的兴趣关键词,建立基于层次向量空间模型的用户兴趣模型.实验结果表明,本文提出的基于情景信息的用户兴趣模型对用户兴趣的预测误差控制在9%以内,是有效的.     

一种改进的基于身份广播代理重加密云存储方案

存储安全是公共云应用诸多安全问题中最关键的问题之一,对云服务的快速发展有着重大影响。结合身份加密、代理重加密以及广播加密的特点,提出了一种新的云存储方案。该方案通过条件控制,实现了用户对远程密文数据代理重加密过程中的细粒度访问控制;基于身份加密,利用用户的身份属性作为公钥,减少了证书验证过程;结合广播的思想,以用户集合为单位进行加密,减少系统的计算消耗。实验表明,文章提出的方案可以实现密文数据云存储和共享,主要函数的时间开销合理,能够保证大规模用户接入时系统高效。     

Enabling effective design of multimodal interfaces for speech-to-speech translation system: An empirical study of longitudinal user behaviors over time and user strategies for coping with errors

The study provides an empirical analysis of long-term user behavioral changes and varying user strategies during cross-lingual interaction using the multimodal speech-to-speech (S2S) translation system of USC/SAIL. The goal is to inform user adaptive designs of such systems. A 4-week medical-scenario-based study provides the basis for our analysis. The data analyzed includes user interviews, post-session surveys, and the extensive system logs that were post-processed and annotated. The annotations measured the meaning transfer rates using human evaluations and a scale defined here called the concept matching score.First, qualitative data analysis investigates user strategies in dealing with errors, such as repeat, rephrase, change topic, start over, and the participants’ self-reported longitudinal adaptation to errors. Post-session surveys explore participant experience with the system and point to a trend of user-perceived increased performance over time.The log data analysis provides further insightful results. Users chose to allow some degradation (84% of original concepts) of their intended meaning to proceed through the system, even after they observed potential errors in the visual output from the speech recognizer. The rejected utterances, on average, had only 25% of the original concepts. This user-filtered outcome, after the complete channel transfer through the S2S system, is that 91% of the successful turns result in transfer of at least half the intended concepts while 90% of the user rejected turns would have conveyed less than half the intended meaning.The multimodal interface results in 24% relative improvement in the confirmation mode and in 31% relative improvement in the choice mode compared to the speech-only modality. Analysis also showed that users of the multimodal interface temporally change their strategies by accepting more system-produced choices. This user behavior can expedite communication seeking an operating balance between user strategies and system performance factors. Lastly, user utterance length is analyzed. Longer utterances in general imply more information delivered per utterance but potentially at the cost of increased processing degradation. The analysis demonstrates that users reduce their utterance length after unsuccessful turns and increase it after successful turns and that there is a learning effect that increases this behavior over the duration of the study.     

Control of data processing systems by voice commands

Abstract

A spoken dialogue between a user and a computer system has to be governed by the system because of: (i) the limited capabilities of present word-recognition apparatus; and (ii) the limited possibilities of the system for ‘understanding’ its user. The user's part of the dialogue therefore needs to be unobtrusively controlled by the system by carefully phrased and timed prompts. Short pauses in these prompts enable the experienced user to make shortcuts through the dialogue, without forsaking complete explanations for the inexperienced user. The user is also able to control the system by utilizing other pauses in the system utterances for corrective words or protests in case of incorrect recognition.     

Efficient sketch‐based creation of detailed character models through data‐driven mesh deformations

Creation of detailed character models is a very challenging task in animation production. Sketch‐based character model creation from a 3D template provides a promising solution. However, how to quickly find correct correspondences between user's drawn sketches and the 3D template model, how to efficiently deform the 3D template model to exactly match user's drawn sketches, and realize real‐time interactive modeling is still an open topic. In this paper, we propose a new approach and develop a user interface to effectively tackle this problem. Our proposed approach includes using user's drawn sketches to retrieve a most similar 3D template model from our dataset and marrying human's perception and interactions with computer's highly efficient computing to extract occluding and silhouette contours of the 3D template model and find correct correspondences quickly. We then combine skeleton‐based deformation and mesh editing to deform the 3D template model to fit user's drawn sketches and create new and detailed 3D character models. The results presented in this paper demonstrate the effectiveness and advantages of our proposed approach and usefulness of our developed user interface. Copyright © 2015 John Wiley & Sons, Ltd.     

Reading a robot's mind: a model of utterance understanding based on the theory of mind mechanism

The purpose of this paper is to construct a methodology for smooth communication between humans and robots. Here, focus is on a mindreading mechanism, which is indispensable in human-human communication. We propose a model of utterance understanding based on this mechanism. Concretely speaking, we apply the model of a mindreading system to a model of human-robot communication. Moreover, we implement a robot interface system that applies our proposed model. The characteristic of our interface system is its ability to construct a relationship between a human and a robot by a method of having an agent, which interacts with the person, migrate from the mobile PC of the person to the robot. Psychological experiments were carried out to explore the validity of the following hypothesis: By reading a robot's mind based on such a relationship, a person can estimate the robot's intention with ease and, moreover, the person can even understand the robot's unclear utterances made by synthesized speech sounds. The results of the experiments statistically supported our hypothesis.     

基于频繁主题集偏好的学术论文推荐算法

针对学术论文推荐中项目冷启动问题,提出了一种基于频繁主题集偏好的协同主题回归模型。该算法考虑到用户在选择学术论文时对研究热点的偏好,使用频繁主题集代表研究热点,将用户对研究热点的偏好表示成用户对频繁主题集的偏好。通过潜在狄利克雷分布主题模型挖掘得到论文—主题概率分布矩阵,并筛选出论文中概率较高的主题;然后挖掘出频繁出现的主题集合,并得到论文—频繁主题集矩阵;最后在预测未知评分时融入用户对频繁主题集的偏好。在CiteULike数据集上的实验表明,相比于矩阵分解模型和协同主题回归模型,该算法在召回率、准确率和RMSE三个指标上都有所提升。     

Flexible service selection with user-specific QoS support in service-oriented architecture

Along with the development of the service-oriented architecture (SOA) and cloud computing, a large number of service providers have created an intense competitive world of business. Consequently, it is becoming increasingly complex to select a service provider for a user as a result of their various economic and social attributes. In this paper, we state the problem of how to find the appropriate services with satisfying the users' multiple QoS requirements. We consider the service's response time, trust degree and monetary cost. And inspired from the mode of Web search engine, such as Yahoo, Google, we propose an innovative service selection algorithm for SOA systems. The algorithm can recommend a number of suitable services based on the user's QoS requirements.Compared with the existing scheduling algorithms, our solution is much more flexible in supporting the multiple objectives and user personalization. We study the scalability of the algorithm with different numbers of jobs, service providers and QoS criteria. And we find that it can capture user's preferences value in less than six times of job submissions.     

针对复杂用户评论的代码质量属性判断

随着开发者社区和代码托管平台成为程序员获取代码的主要途径,针对代码的用户评论数量急剧增加.用户在使用代码后给出的评论中包含多种静态和动态的代码质量属性信息,但是由于用户评论多为复杂句,使得评论中包含的代码质量属性难以判断.针对复杂用户评论的代码质量属性判断将有助于分析用户评论中的代码质量信息,有助于开发者在了解用户的代...     

Search personalization through query and page topical analysis

Thousands of users issue keyword queries to the Web search engines to find information on a number of topics. Since the users may have diverse backgrounds and may have different expectations for a given query, some search engines try to personalize their results to better match the overall interests of an individual user. This task involves two great challenges. First the search engines need to be able to effectively identify the user interests and build a profile for every individual user. Second, once such a profile is available, the search engines need to rank the results in a way that matches the interests of a given user. In this article, we present our work towards a personalized Web search engine and we discuss how we addressed each of these challenges. Since users are typically not willing to provide information on their personal preferences, for the first challenge, we attempt to determine such preferences by examining the click history of each user. In particular, we leverage a topical ontology for estimating a user’s topic preferences based on her past searches, i.e. previously issued queries and pages visited for those queries. We then explore the semantic similarity between the user’s current query and the query-matching pages, in order to identify the user’s current topic preference. For the second challenge, we have developed a ranking function that uses the learned past and current topic preferences in order to rank the search results to better match the preferences of a given user. Our experimental evaluation on the Google query-stream of human subjects over a period of 1 month shows that user preferences can be learned accurately through the use of our topical ontology and that our ranking function which takes into account the learned user preferences yields significant improvements in the quality of the search results.     

Interactive Curation of Datasets for Training and Refining Generative Models

We present a novel interactive learning‐based method for curating datasets using user‐defined criteria for training and refining Generative Adversarial Networks. We employ a novel batch‐mode active learning strategy to progressively select small batches of candidate exemplars for which the user is asked to indicate whether they match the, possibly subjective, selection criteria. After each batch, a classifier that models the user's intent is refined and subsequently used to select the next batch of candidates. After the selection process ends, the final classifier, trained with limited but adaptively selected training data, is used to sift through the large collection of input exemplars to extract a sufficiently large subset for training or refining the generative model that matches the user's selection criteria. A key distinguishing feature of our system is that we do not assume that the user can always make a firm binary decision (i.e., “meets” or “does not meet” the selection criteria) for each candidate exemplar, and we allow the user to label an exemplar as “undecided”. We rely on a non‐binary query‐by‐committee strategy to distinguish between the user's uncertainty and the trained classifier's uncertainty, and develop a novel disagreement distance metric to encourage a diverse candidate set. In addition, a number of optimization strategies are employed to achieve an interactive experience. We demonstrate our interactive curation system on several applications related to training or refining generative models: training a Generative Adversarial Network that meets a user‐defined criteria, adjusting the output distribution of an existing generative model, and removing unwanted samples from a generative model.     

Building respectful interface agents

To provide personalized assistance to users, interface agents have to learn not only a user's preferences and interests with respect to a software application, but also when and how the user prefers to be assisted. Interface agents have to detect the user's intention to determine when to assist the user, and the user's interaction and interruption preferences to provide the right type of assistance without hindering the user's work. In this work we describe a user profiling approach that considers these issues within a user profile and a decision making approach that enables the agent to choose the best type of assistance for a given user in a given situation. We also describe the results obtained when evaluating our proposal in the tourism domain, and we compare these results with some previous ones in the calendar management domain.