KAT II: Tactile Display Mouse for Providing Tactile and Thermal Feedback

This paper proposes a tactile display mouse providing both pin-array-type tactile feedback and thermal feedback. The pin-array-type tactile display is composed of a 6 × 5 pin-array that is actuated by 30 piezo-electric bimorphs. Micro shape and vibrotactile feedback can be generated by the device, and various planar distributed patterns can be displayed as can Braille cell patterns. The thermal feedback device is composed of a thin-film resistance temperature detector, a Peltier thermoelectric heat pump and a water cooling jacket. Users can discriminate among different materials by considering the temperature variation that can be sensed as they touch an object's surface. This paper also includes an experimental evaluation of the tactile display mouse to prove the effectiveness of displaying textures. Evaluation of the ability to identify material properties was conducted using the thermal feedback part that displays a simulated temperature profile. To investigate thermo–tactile interaction, an experiment determining perceived magnitude of vibrotactile stimulus according to different temperature conditions was conducted.     

Gaze Interaction With Vibrotactile Feedback: Review and Design Guidelines

Vibrotactile feedback is widely used in mobile devices because it provides a discreet and private feedback channel. Gaze-based interaction, on the other hand, is useful in various applications due to its unique capability to convey the focus of interest. Gaze input is naturally available as people typically look at things they operate, but feedback from eye movements is primarily visual. Gaze interaction and the use of vibrotactile feedback have been two parallel fields of human–computer interaction research with a limited connection. Our aim was to build this connection by studying the temporal and spatial mechanisms of supporting gaze input with vibrotactile feedback. The results of a series of experiments showed that the temporal distance between a gaze event and vibrotactile feedback should be less than 250 ms to ensure that the input and output are perceived as connected. The effectiveness of vibrotactile feedback was largely independent of the spatial body location of vibrotactile actuators. In comparison to other modalities, vibrotactile feedback performed equally to auditory and visual feedback. Vibrotactile feedback can be especially beneficial when other modalities are unavailable or difficult to perceive. Based on the findings, we present design guidelines for supporting gaze interaction with vibrotactile feedback.     

Effect of kinesthetic and tactile haptic feedback on the quality of experience of edutainment applications

Haptic technologies and applications have received enormous attention in the last decade. The incorporation of haptic modality into multimedia applications adds excitement and enjoyment to an application. It also adds a more natural feel to multimedia applications, that otherwise would be limited to vision and audition, by engaging as well the user’s sense of touch, giving a more intrinsic feel essential for ambient intelligent applications. However, the improvement of an application’s Quality of Experience (QoE) by the addition of haptic feedback is still not completely understood. The research presented in this paper focuses on the effect of haptic feedback and what it potentially adds to the experience of the user as opposed to the traditional visual and auditory feedback. In essence, it investigates certain issues regarding stylus-based haptic education applications and haptic-enhanced entertainment videos. To this end, we used two haptic applications: the haptic handwriting learning tool to experiment with force feedback haptic interaction and the tactile YouTube application for tactile haptic feedback. In both applications, our analysis shows that the addition of haptic feedback will increase the QoE in the absence of fatigue or discomfort for this category of applications. This implies that the incorporation of haptic modality (both force feedback as well as tactile feedback) has positively contributed to the overall QoE for the users.     

Multimodal feedback for the acquisition of small targets

This paper examines how multimodal feedback assists small-target acquisition in graphical user interfaces. All combinations of three feedback modes are analysed: non-speech audio; tactile; and pseudo-haptic ‘sticky’ feedback. The tactile conditions used stimulation through vibration (rather than force-feedback), and the sticky conditions were implemented by dynamically reconfiguring mouse control-display gain as the cursor entered the target. Results show that for small, discretely located targets all feedback modes reduce targeting times, with stickiness providing substantial improvements. Furthermore, stickiness and tactile appear to combine well. However, the results of a more ecologically oriented menu-selection task show the need for caution, revealing that excessive feedback can damage interaction though ‘noise’ that interferes with the acquisition of neighbouring targets.     

Helping older pedestrians navigate in the city: comparisons of visual,auditory and haptic guidance instructions in a virtual environment

Preserving older pedestrians’ navigation skills in urban environments is a challenge for maintaining their quality of life. However, existing aids do not take into account older people’s perceptual and cognitive declines nor their user experience, and they call upon sensory modalities that are already used during walking. The present study was aimed at comparing different guidance instructions using visual, auditory, and haptic feedback in order to identify the most efficient and best accepted one(s). Sixteen middle-age (non-retired) adults, 21 younger-old (young-retired) adults, and 21 older-old (old-retired) adults performed a navigation task in a virtual environment. The task was performed with visual feedback (directional arrows superimposed on the visual scenes), auditory feedback (sounds in the left/right ear), haptic feedback (vibrotactile information delivered by a wristband), combinations of different types of sensory feedback, or a paper map. The results showed that older people benefited from the sensory guidance instructions, as compared to the paper map. Visual and auditory feedbacks were associated with better performance and user experience than haptic feedback or the paper map, and the benefits were the greatest among the older-old participants, even though the paper-map familiarity was appreciated. Several recommendations for designing pedestrian navigation aids are proposed.     

Spatialized Vibrotactile Feedback Improves Goal-Directed Movements in Cluttered Virtual Environments

Spatial awareness in virtual reality (VR) is a dominant research topic. It plays an essential role in the assessment of human operators’ behavior in simulated tasks, notably for the evaluation of the feasibility of manual maintenance tasks in cluttered industrial settings. In such contexts, it is decisive to evaluate the spatial and temporal correspondence between the operator’s movement kinematics and that of his/her virtual avatar in the virtual environment (VE). Often, in a cluttered VE, direct kinesthetic (force) feedback is limited or absent. We tested whether vibrotactile (cutaneous) feedback would increase visuo-proprioceptive consistency, spatial awareness, and thus the validity of VR studies, by augmenting the perception of the operator’s contact(s) with virtual objects. We present experimental results obtained using a head-mounted display (HMD) during a goal-directed task in a cluttered VE. Results suggest the contribution of spatialized vibrotactile feedback to visuo-proprioceptive consistency.     

基于视-触跨模态感知的智能导盲系统

盲人活动援助是盲人日常生活的重要组成部分。这些技术大多用于帮助盲人导航和躲避障碍物,很少有研究将地面信息转换成一种给用户直观感受的触觉信息。为了满足上述需求,本文提出了一种可以提供触觉反馈的盲人辅助地面识别智能导盲杖系统。试图利用深度生成对抗训练的方法来产生振动触觉刺激,使用改进的DiscoGAN训练了我们的端到端生成网络。为了训练我们的网络,构建了视触跨模态数据集GroVib。通过上机实验和实物实验来评估方案的可行性,通过上机实验结果表明参与者通过触觉识别地面的准确率为84.7%,触觉的平均真实感受得分为71.3,在真实场景实验中,参与者只需平均3.35次尝试就可以根据触觉反馈来识别地面。     

Haptic discrimination of different types of pencils during writing

Professional artists seem to develop a preference for certain types of pencils and paper (Akyuz 1995 ). While theaesthetic outcome may be the obvious reason for this preference, haptic feedback may also play a role in selecting media (Prytherch and Jerrard 2001 , 2003 ). This suggestion implies that artists may be sensitive to vibrotactile signals resulting from the interaction between pencil and paper during drawing. Tribological tests showed that H-range leads generate more resistance than B-range pencils when writing on a standard 80 g/m 2 paper. Moreover, a constant stimulus 2IFC procedure showed that Fine Arts students found it easier to discriminate between H-range and HB than B-range and an HB. This suggests that the human haptic system may be sensitive to the vibrotactile signals generated by the differences in resistance resulting from the interaction of leads with the paper's textured surface. Implications for pencil manufacturing and simulations of virtual writing are discussed. Statement of Relevance: The study assesses the tactile sensitivity of Fine Artists to the physical properties of the interaction between different types of lead and a standard printing paper. The results may help in designing more responsive drawing instruments as well as more realistic virtual drawing interfaces.     

Mobile technology in mobile work: contradictions and congruencies in activity systems

Mobile technology and the information and communication services supported by it have become increasingly embedded in, and in some cases transformed, work and social activity and created new challenges for studying information systems. This paper focuses on the experience with mobile technology in an inherently mobile and information-intensive work activity – policing. Drawing upon data from this context this paper makes two key sets of contributions. Empirically, we illuminate the congruencies and contradictions between mobile technology and mobile working, and the relationship between the two, revealing a state of change based upon dialectic interaction. We highlight several ways in which mobile technology has changed the nature of mobile work activity. Theoretically we advance the use of activity theory to better understand the changes of mobile technology-mediated work. We extend traditional use of activity theory by adopting congruencies as an analytical lens, in addition to the approach of examining contradictions. The findings are applicable to other areas of mobile work and contribute to the body of knowledge concerning mobile technology-mediated work.     

基于微电流刺激的多强度分级虚拟触觉反馈

触觉反馈可以有效提高用户的虚拟现实交互的沉浸感,但是采用振动方式的触觉反馈存在反馈模态单一的缺陷;采用机械传动式、微流驱动方式的触觉反馈存在结构复杂、难以集成等缺陷. 微电流触觉反馈具有集成度高、反馈模态丰富等优点,但是存在反馈强度识别准确率不够和长时间作用下易造成不适感等问题. 为了解决这些问题,设计了基于微电流刺激的新型多强度电触觉反馈系统,通过研究电流参数、电极阵列和接地电极等影响因素,并引入双相电流脉冲,优化电流正负电荷量比值等方式,确定了该系统的刺激模式. 35名受试者的心理物理学实验结果表明该反馈系统能够在有效减少微电流刺激带来不适的同时实现93.3%和81.7%的四级和五级强度识别准确率,优于传统方法,这可能是具有广泛应用场景的触觉反馈设备.

     

Cubic-Mouse-based interaction in virtual environments

Recently prop-based interfaces in combination with two-handed interaction techniques have become increasingly popular. Passive real-world props augment interaction through tactile feedback and often lead to more intuitive interaction. The Cubic Mouse is a new, 3D input device based on the prop idea. It consists of a cube-shaped box with three perpendicular rods passing through its center. We use a six-degrees-of-freedom (6-DOF) tracker embedded in the Cubic Mouse to track the device's position and orientation. The rods can be pushed and pulled, allowing the constrained input of three degrees of freedom. The 6-DOF Cubic Mouse, also allows rotation of the rods, which adds another three DOF. Altogether, we have a total of 12 analog degrees of freedom available with the Cubic Mouse, six DOF through the tracking and another six through the rods. In addition, the Cubic Mouse has application-programmable control buttons. We present a variety of interaction techniques developed around the Cubic Mouse, which we have implemented in various application prototypes. Our geo-scientific visualization system allows the exploration of data from the oil and gas industry in local and distributed virtual environments. With our automotive partners, we're developing a system for steering and visualizing crash simulations. Other application areas include medical visualization and terrain visualization     

TIKL: Development of a Wearable Vibrotactile Feedback Suit for Improved Human Motor Learning

When humans learn a new motor skill from a teacher, they learn using multiple channels. They receive high level information aurally about the skill, visual information about how another performs the skill, and at times, tactile information from the teacher's physical guidance. This research proposes a novel approach where the student receives real-time tactile feedback, simultaneously over all joints, delivered through a wearable robotic system. This tactile feedback can supplement the visual or auditory feedback from the teacher. Our results using a 5-DOF robotic suit show a 27% improvement in accuracy while performing the target motion, and an accelerated learning rate of up to 23%. We report both of these results with high statistical significance (p les 0.01). This research is intended for use in a diverse set of applications including sports training, motor rehabilitation after neurological damage, dance, postural retraining for health, and many others. We call this system tactile interaction for kinesthetic learning (TIKL).     

图像局部纹理特性的静电力触觉渲染

目的 触摸触觉设备感知物体时,需要实现视觉-力触觉同步反馈,其中图像-力触觉反馈难点在于再现更真实的纹理触感的触觉渲染过程。本文提出了一种基于图像局部纹理特征的静电力触觉渲染模型,实现了更加清晰、触感真实的图像纹理的静电力触觉反馈。方法 首先,采用局部傅里叶变换方法强化局部纹理特征,提取傅里叶变换系数分离出表征形状和局部纹理、边缘的频域分量。其次,对局部纹理特征进行力触觉渲染,建立局部纹理特征与驱动信号的映射模型,采用比例模型将局部纹理特征值转化为同等级的静电力表达。最后,根据静电力与驱动信号的心理学模型,由局部纹理特征控制不同驱动信号的输出产生静电力触觉。结果 进行纹理触觉对比感知实验验证算法有效性,62.5%的实验参与者偏爱基于图像局部纹理的触觉渲染算法反馈的纹理触感,本文算法可以模拟多种图像的纹理、边缘的触感。结论 算法在频域分离图像局部纹理、边缘和形状特征,建立纹理-力触觉渲染模型,针对大多数图片可以有效地增强纹理触感,提升触觉再现交互技术的沉浸感。     

Designing motivation using persuasive ambient mirrors

In this article, we describe four case studies of ubiquitous persuasive technologies that support behavior change through personalized feedback reflecting a user’s current behavior or attitude. The first case study is Persuasive Art, reflecting the current status of a user’s physical exercise in artistic images. The second system, Virtual Aquarium, reflects a user’s toothbrushing behavior in a Virtual Aquarium. The third system, Mona Lisa Bookshelf, reflects the situation of a shared bookshelf on a Mona Lisa painting. The last case study is EcoIsland, reflecting cooperative efforts toward reducing CO₂ emissions as a set of virtual islands shared by a neighborhood. Drawing from the experience of designing and evaluating these systems, we present guidelines for the design of persuasive ambient mirrors: systems that use visual feedback to effect changes in users’ everyday living patterns. In particular, we feature findings in choosing incentive systems, designing emotionally engaging feedback, timing feedback, and persuasive interaction design. Implications for current design efforts as well as for future research directions are discussed.     

Vibrotactile and vibroacoustic communications: pairs in interaction and play—an interactive structure and bodies in an urban environment

We designed a vibrotactile vest and The Humming Wall, a vibroacoustic interactive furniture set in an urban environment to interact with each other. We developed the vibrotactile patterns in the vest as a form of vibrotactile language to convey information to the wearer. In addition, we designed a set of interactive movements on The Humming Wall that would trigger patterns on the vest and elicit sensations and encourage body movements onto the wearer’s body. We invited people to interact in pairs at The Humming Wall, with one at the wall and one wearing the vest (they later swapped roles). Actions by the one at the wall, such as swiping or knocking on the wall were repeated on the vest wearer’s body. In addition, participants could ‘feel’ (vibroacoustically) and hear their own heartbeats and breath rates at the wall. We conducted a field trial with 39 participants over a 5-week period. Participants wearing the vest (and their pair) completed a set of tasks. We logged use and responses, recorded all activities on video and conducted post-experiment interviews and questionnaires. The results depicted the participants’ experience, communication and connection while wearing the vibrotactile vest and interacting with the wall. The findings show convincing, strong and positive responses to novel interactions between the responsive vibroacoustic environment and the vibrotactile vest. This work constitutes the first field trial with people ‘working’ in pairs with a vibrotactile wearable responding to and driving vibroacoustic displays with an interactive vibroacoustic environment.     

AR-based serious game framework for post-stroke rehabilitation

Stroke is considered one of the main causes of death around the world. Survivors often suffer different kinds of disabilities in terms of their cognitive and motor capabilities, and are therefore unable to perform their day-to-day activities. To regain some of their cognitive as well as motor abilities, they require rehabilitation. To this end, we present a serious game framework based on augmented reality technology that may motivate the patients’ involvement in the rehabilitation exercise. Additionally, we analyze the requirements for such a framework and describe the concept and implementation of the proposed approach. Furthermore, we designed a wireless vibrotactile output device that is attached to a tangible object. The tangible object that is connected to the framework can give haptic as well as audio-visual feedback to the patient in a more motivating and entertaining environment for rehabilitation exercises. The suitability and utility of the proposed framework was evaluated with real stroke patients and compared against the performance of a healthy control group, thus facilitating occupational therapists in assessing a patient’s progress. Our evaluations show that the serious games with vibrotactile feedback are well accepted by patients.     

Tactile feedback enhanced hand gesture interaction at large, high-resolution displays

Human beings perceive their surroundings based on sensory information from diverse channels. However, for human–computer interaction we mostly restrict the user on visual perception. In this paper, we contribute to the investigation of tactile feedback as an additional perception modality. Therefore, we will first discuss existing user studies and provide a classification scheme for tactile feedback techniques. We will then present and discuss a comparative evaluation study based on the ISO 9241-9 [Ergonomic requirements for office work with visual display terminals (VDTs) – Part 9: requirements for non-keyboard input devices, 2000]. The 20 participants performed horizontal and vertical one-directional tapping tasks with hand gesture input with and without tactile feedback in front of a large, high-resolution display. In contrast to previous research, we cannot confirm a benefit of tactile feedback on user performance. Our results show no significant effect in terms of throughput (effective index of performance (IPe)) and even a significant higher error rate for horizontal target alignment when using tactile feedback. Based on these results, we suggest that tactile feedback can interfere with other senses in a negative way, resulting in the observed higher error rate for horizontal targets. Therefore, more systematic research is needed to clarify the influencing factors on the usefulness of tactile feedback. Besides these results, we found a significant difference in favor of the horizontal target alignment compared with the vertical one in terms of the effective index of performance (IPe), confirming the work by Dennerlein et al. [Force feedback improves performance for steering and combined steering–targeting tasks, in: CHI ’00: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, ACM, New York, NY, USA, 2000, pp. 423–429].     

Multimodal feedback for the acquisition of small targets

This paper examines how multimodal feedback assists small-target acquisition in graphical user interfaces. All combinations of three feedback modes are analysed: non-speech audio; tactile; and pseudo-haptic "sticky" feedback. The tactile conditions used stimulation through vibration (rather than force-feedback), and the sticky conditions were implemented by dynamically reconfiguring mouse control-display gain as the cursor entered the target. Results show that for small, discretely located targets all feedback modes reduce targeting times, with stickiness providing substantial improvements. Furthermore, stickiness and tactile appear to combine well. However, the results of a more ecologically oriented menu-selection task show the need for caution, revealing that excessive feedback can damage interaction though "noise" that interferes with the acquisition of neighbouring targets.     

Teaching a pet-robot to understand user feedback through interactive virtual training tasks

In this paper, we present a human-robot teaching framework that uses “virtual” games as a means for adapting a robot to its user through natural interaction in a controlled environment. We present an experimental study in which participants instruct an AIBO pet robot while playing different games together on a computer generated playfield. By playing the games and receiving instruction and feedback from its user, the robot learns to understand the user’s typical way of giving multimodal positive and negative feedback. The games are designed in such a way that the robot can reliably predict positive or negative feedback based on the game state and explore its user’s reward behavior by making good or bad moves. We implemented a two-staged learning method combining Hidden Markov Models and a mathematical model of classical conditioning to learn how to discriminate between positive and negative feedback. The system combines multimodal speech and touch input for reliable recognition. After finishing the training, the system was able to recognize positive and negative reward with an average accuracy of 90.33%.