Adaptation of robot’s perception of fuzzy linguistic information by evaluating vocal cues for controlling a robot manipulator

This article proposes a method for adapting a robot’s perception of fuzzy linguistic information by evaluating vocal cues. The robot’s perception of fuzzy linguistic information such as “very little” depends on the environmental arrangements and the user’s expectations. Therefore, the robot’s perception of the corresponding environment is modified by acquiring the user’s perception through vocal cues. Fuzzy linguistic information related to primitive movements is evaluated by a behavior evaluation network (BEN). A vocal cue evaluation system (VCES) is used to evaluate the vocal cues for modifying the BEN. The user’s satisfactory level for the robot’s movements and the user’s willingness to change the robot’s perception are identified based on a series of vocal cues to improve the adaptation process. A situation of cooperative rearrangement of the user’s working space is used to illustrate the proposed system by a PA-10 robot manipulator.     

User—robot personality matching and assistive robot behavior adaptation for post-stroke rehabilitation therapy

This paper describes a hands-off socially assistive therapist robot designed to monitor, assist, encourage, and socially interact with post-stroke users engaged in rehabilitation exercises. We investigate the role of the robot’s personality in the hands-off therapy process, focusing on the relationship between the level of extroversion–introversion of the robot and the user. We also demonstrate a behavior adaptation system capable of adjusting its social interaction parameters (e.g., interaction distances/proxemics, speed, and vocal content) toward customized post-stroke rehabilitation therapy based on the user’s personality traits and task performance. Three validation experiment sets are described. The first maps the user’s extroversion–introversion personality dimension to a spectrum of robot therapy styles that range from challenging to nurturing. The second and the third experiments adjust the personality matching dynamically to adapt the robot’s therapy styles based on user personality and performance. The reported results provide first evidence for user preference for personality matching in the assistive domain and demonstrate how the socially assistive robot’s autonomous behavior adaptation to the user’s personality can result in improved human task performance. This work was supported by USC Women in Science and Engineering (WiSE) Program and the Okawa Foundation.     

Prediction of user navigation patterns by mining the temporal web usage evolution

Advances in the data mining technologies have enabled the intelligent Web abilities in various applications by utilizing the hidden user behavior patterns discovered from the Web logs. Intelligent methods for discovering and predicting user’s patterns is important in supporting intelligent Web applications like personalized services. Although numerous studies have been done on Web usage mining, few of them consider the temporal evolution characteristic in discovering web user’s patterns. In this paper, we propose a novel data mining algorithm named Temporal N-Gram (TN-Gram) for constructing prediction models of Web user navigation by considering the temporality property in Web usage evolution. Moreover, three kinds of new measures are proposed for evaluating the temporal evolution of navigation patterns under different time periods. Through experimental evaluation on both of real-life and simulated datasets, the proposed TN-Gram model is shown to outperform other approaches like N-gram modeling in terms of prediction precision, in particular when the web user’s navigating behavior changes significantly with temporal evolution.     

A multi-agent based user context Bayesian neural network analysis system

The increasing user mobility demands placed upon IT services necessitates an environment that enables users to access optimal services at any time and in any place. This study presents research conducted to develop a system that is capable of analyzing user IT service patterns and tendencies and provides the necessary service resources by sharing each user’s context information. First, each user’s context information is gathered to provide the multi-agent software training data necessary to describe user operations in a hybrid peer-to-peer (P2P) structured communication environment. Next, the data collected about each user’s mobile device is analyzed through a Bayesian based neural network system to identify the user’s tendency and extract essential service information. This information provides a communication configuration allowing the user access to the best communication service between the user’s mobile device and the local server at any time and in any place, thereby enhancing the ubiquitous computing environment.     

Perception-based lossy haptic compression considerations for velocity-based interactions

The ability of technology to transmit multi-media is very dependent on compression techniques. In particular lossy compression has been used in image compression (jpeg) audio compression (mp3) and video compression (mpg) to allow the transmission of audio and video over broadband network connections. Recently the sense of touch or haptics is becoming more important with its addition in computer games or in cruder applications such as vibrations in a cell phone. As haptic technology improves the ability to transmit compressed force sensations becomes more critical. Most lossy audio and visual compression techniques rely on the lack of sensitivity in humans to pick up detailed information in certain scenarios. Similarly limitations in the sensitivity of human touch could be exploited to create haptic models with much less detail and thus requiring smaller bandwidth. The focus of this paper is on the force thresholds of the human haptic system that can be used in a psychophysically motivated lossy haptic (force) compression technique. Most of the research in this field has measured the just noticeable difference (JND) of the human haptic system with a human user in static interaction with a stationary rigid object. In this paper our focus involves cases where the human user or the object are in relative motion. An example of such an application would be the haptic rendering of the user’s hand in contact with of a high-viscous material or interacting with a highly deformable object. Thus an approach is presented to measure the force threshold based on the velocity of the user’s hand motion. Two experiments are conducted to detect the absolute force threshold (AFT) of the human haptic system using methodologies from the field of psychophysics. The AFTs are detected for three different ranges of velocity of the user’s hand motion. This study implies that when a user’s hand is in motion fewer haptic details are required to be stored calculated or transmitted. Finally the implications of this study on a more complete future study will be discussed.

Toward accessible self-service kiosks through intelligent user interfaces

Public self-service kiosks provide key services such as ticket sales, airport check-in and general information. Such kiosks must be universally designed to be used by society at large, irrespective of the individual users’ physical and cognitive abilities, level of education and familiarity with the system. The noble goal of universal accessibility is hard to achieve. This study reports experiences with a universally designed kiosk prototype based on a multimodal intelligent user interface that adapts to the user’s physical characteristics. The user interacts with the system via a tall rectangular touch-sensitive display where the interaction area is adjusted to fit the user’s height. A digital camera is used to measure the user’s approximate reading distance from the display such that the text size can be adjusted accordingly. The user’s touch target accuracy is measured, and the target sizes are increased for users with motor difficulties. A Byzantine visualization technique is employed to exploit unused and unreachable screen real estate to provide the user with additional visual cues. The techniques explored in this study have potential for most public self-service kiosks.     

Rapid behavior adaptation for human-centered robots in a dynamic environment based on the integration of primitive confidences on multi-sensor elements

This article presents a method for tele-operated mobile robots to rapidly adapt to behavior policies. Since real-time adaptation requires frequent observations of sensors and the behavior of users, rapid policy adaptation cannot be achieved when significant data are not differentiated from insignificant data in every process cycle. Our method solves this problem by evaluating the significance of data for learning based on changes in the degree of confidence. A small change in the degree of confidence can be regarded as reflecting insignificant data for learning (that data can be discarded). Accordingly, the system can avoid having to store experience data too frequently, and the robot can adapt more rapidly to changes in the user’s policy. In this article, we confirm that by taking advantage of a significance evaluation not only of proposition of behavior, but also of each proposition of each piece of sensor-level data, a robot can rapidly adapt to a user’s policy. We discuss the results of two experiments in static and dynamic environments, in both of which the user switched policies between “avoid” and “approach.”     

Event detection in real time on a personal camera phone image collection

Grouping photos of the same event together is extremely useful for the management of personal photo collections. However, most methods cannot be applied to the problem of online event detection in embedded devices because they do not consider hardware constraints or a user’s photo-taking behavior. In this paper, we propose an efficient and effective event detection algorithm for managing personal photo collections in camera phones or digital cameras. The proposed algorithm fuses time and location information, which is deemed the most important information for personal photo management, and works in real time in embedded devices. We model event occurrences in a user’s photo-taking behavior as a Poisson process by imposing certain constraints on calculating the elapsed time. Location information is incorporated into event detection when confidence in a decision based on the Poisson process is not high enough. The algorithm is user-centric because it provides the unique capabilities of accepting and adjusting to user feedback. Our experiment results show that the proposed event detection method has the potential to support emerging multimedia applications in embedded devices.     

The graph-based behavior-aware recommendation for interactive news

Applied Intelligence - Interactive news recommendation has been launched and attracted much attention recently. In this scenario, user’s behavior evolves from single click behavior to...     

A user-adaptive city guide system with an unobtrusive navigation interface

In this paper, we describe an intelligent location-aware city guide system, which adapts to each user’s preferences, and uses an intuitive “metal detector” interface for navigation. Our system analyzes each user’s past location data history to estimate individual preferences, and allows users to find shops that match their tastes in the same way a metal detector would be used to detect metal objects. The procedure with which the system picks out shops that match each user’s preferences includes a newly developed place learning algorithm, which can efficiently find frequented places, complete with their proper names (e.g. “The Ueno Royal Museum”). We have conducted a series of evaluation tests at a popular shopping district inside Tokyo, and the results validate the effectiveness of our overall approach.     

Human-agent service matching using natural language queries: system test and training

Smart environments, ambient intelligence and intelligent agents leave the user lost between large amounts of services. Ad-hoc networks, mobile agents and mobile devices make the set of available services dynamic over time and space, increasing the user’s problems to find the service he needs. Earlier, we presented a ServiceMatcher that can find the agent best fitting to the user’s natural language request. This paper presents performance results of the ServiceMatcher. The test queries come from human users in a realistic scenario (see our other paper in this issue). With a short training of the agent vocabularies, over 80% correct service matches are found.     

Formation conditions of mutual adaptation in human-agent collaborative interaction

When an adaptive agent works with a human user in a collaborative task, in order to enable flexible instructions to be issued by ordinary people, it is believed that a mutual adaptation phenomenon can enable the agent to handle flexible mapping relations between the human user’s instructions and the agent’s actions. To elucidate the conditions required to induce the mutual adaptation phenomenon, we designed an appropriate experimental environment called “WAITER” (Waiter Agent Interactive Training Experimental Restaurant) and conducted two experiments in this environment. The experimental results suggest that the proposed conditions can induce the mutual adaptation phenomenon.     

Finding User’s Interest Blocks using Significant Implicit Evidence for Web Browsing on Small Screen Devices

Recent researches on improving the efficiency and user experience of Web browsing on handhelds are seeking to solve the problem by re-authoring Web pages or making adaptations and recommendations according to user preference. Their basis is a good understanding of the relationship between user behaviors and user preference. We propose a practical method to find user’s interest blocks by machine learning using the combination of significant implicit evidences, which is extracted from four aspects of user behaviors: display time, viewing information items, scrolling and link selection. We also develop a customized Web browser for small screen devices to collect user behaviors accurately. For evaluation, we conduct an on-line user study and make statistical analysis based on the dataset, which shows that most types of the suggested implicit evidences are significant, and viewing information items is the least indicative aspect of user behaviors. The dataset is then processed off-line to find user’s interest blocks using the proposed method. Experimental results demonstrate the effectiveness of finding user’s interest blocks by machine learning using the combination of significant implicit evidences. Further analysis reveals the great effect of users and moderate effect of Websites on the usefulness of significant implicit evidences.     

Evaluation of contextual information retrieval effectiveness: overview of issues and research

The increasing prominence of information arising from a wide range of sources delivered over electronic media has made traditional information retrieval systems less effective. Indeed, users are overwhelmed by the information delivered by such systems in response to their queries, particularly when the latter are ambiguous. In order to tackle this problem, the state-of-the-art reveals that there is a growing interest towards contextual information retrieval which relies on various sources of evidence issued from the user’s search background and environment like interests, preferences, time and location, in order to improve the retrieval accuracy. Contextual information retrieval systems are based on different definitions of the core concept of user’s context, various user’s context modeling approaches and several techniques of document relevance measurement, but all share the goal of providing the most useful information to the users in accordance with their context. However, the evaluation methodologies conceived in the past several years for traditional information retrieval and widely used in the evaluation campaigns have been challenged by the consideration of user’s context in the information retrieval process. Thus, we recognize that a critical review of existing evaluation methodologies in contextual information retrieval area is needed in order to design and develop standard evaluation frameworks. We present in this paper a comprehensive survey of contextual information retrieval evaluation methodologies and provide insights into how and why they are appropriate to measure the retrieval effectiveness. We also highlight some of the research challenges ahead that would constitute substantive research area for future research.     

Content-based image retrieval using fuzzy perceptual feedback

In this paper, a new framework called fuzzy relevance feedback in interactive content-based image retrieval (CBIR) systems is introduced. Conventional binary labeling scheme in relevance feedback requires a crisp decision to be made on the relevance of the retrieved images. However, it is inflexible as user interpretation of visual content varies with respect to different information needs and perceptual subjectivity. In addition, users tend to learn from the retrieval results to further refine their information requests. It is, therefore, inadequate to describe the user’s fuzzy perception of image similarity with crisp logic. In view of this, we propose a fuzzy relevance feedback approach which enables the user to make a fuzzy judgement. It integrates the user’s fuzzy interpretation of visual content into the notion of relevance feedback. An efficient learning approach is proposed using a fuzzy radial basis function (FRBF) network. The network is constructed based on the user’s feedbacks. The underlying network parameters are optimized by adopting a gradient-descent training strategy due to its computational efficiency. Experimental results using a database of 10,000 images demonstrate the effectiveness of the proposed method.

A similarity measure for the negotiation in web services

This paper presents a new negotiation approach integrating a similarity-based metric to measure the client’s satisfaction level by quantifying the correspondence between the features required by the client and the ones proposed by the service. This negotiation process is integrated within an adaptation platform for multimedia presentations supporting different types of terminals. The negotiation and adaptation processes are based on the management of user’s and service’s profiles. We also propose an extension of the SIP protocol to ensure the communication between the client and the server by defining new functionalities.     

Exploring the feasibility of web form adaptation to users’ cultural dimension scores

With many daily tasks now performed on the Internet, productivity and efficiency in working with web pages have become transversal necessities for all users. Many of these tasks involve the inputting of user information, obligating the user to interact with a webform. Research has demonstrated that productivity depends largely on users’ personal characteristics, implying that it will vary from user to user. The webform development process must therefore, include modeling of its intended users to ensure the interface design is appropriate. Taking all potential users into account is difficult, however, primarily because their identity is unknown, and some may be effectively excluded by the final design. Such discrimination can be avoided by incorporating rules that allow webforms to adapt automatically to the individual user’s characteristics, the principal one being the person’s culture. In this paper we report two studies that validate this option. We begin by determining the relationships between a user’s cultural dimension scores and their behavior when faced with a webform. We then validate the notion that rules based on these relationships can be established for the automatic adaptation of a webform in order to reduce the time taken to complete it. We conclude that the automatic webform adaptation to the cultural dimensions of users improves their performance.     

Multimodal focus attention and stress detection and feedback in an augmented driver simulator

This paper presents a driver simulator, which takes into account the information about the user’s state of mind (level of attention, fatigue state, stress state). The user’s state of mind analysis is based on video data and biological signals. Facial movements such as eyes blinking, yawning, head rotations, etc., are detected on video data: they are used in order to evaluate the fatigue and the attention level of the driver. The user’s electrocardiogram and galvanic skin response are recorded and analyzed in order to evaluate the stress level of the driver. A driver simulator software is modified so that the system is able to appropriately react to these critical situations of fatigue and stress: some audio and visual messages are sent to the driver, wheel vibrations are generated and the driver is supposed to react to the alert messages. A multi-threaded system is proposed to support multi-messages sent by the different modalities. Strategies for data fusion and fission are also provided. Some of these components are integrated within the first prototype of OpenInterface: the multimodal similar platform.