Steerable projection: exploring alignment in interactive mobile displays

Emerging smartphones and other handheld devices are now being fitted with a set of new embedded technologies such as pico-projection. They are usually designed with the pico-projector embedded in the top of the device. Despite the potential of personal mobile projection to support new forms of interactivity such as augmented reality techniques, these devices have not yet made significant impact on the ways in which mobile data is experienced. We suggest that this ‘traditional’ configuration of fixed pico-projectors within the device is unsuited to many projection tasks because it couples the orientation of the device to the management of the projection space, preventing users from easily and simultaneously using the mobile device and looking at the projection. We present a study which demonstrates this problem and the requirement for steerable projection behaviour and some initial users’ preferences for different projection coupling angles according to context. Our study highlights the importance of flexible interactive projections which can support interaction techniques on the device and on the projection space according to task. This inspires a number of interaction techniques that create different personal and shared interactive display alignments to suit a range of different mobile projection situations.     

MASHA: A multi-agent system handling user and device adaptivity of Web sites

A user that navigates on the Web using different devices should be characterized by a global profile, which represents his behaviour when using all these devices. Then, the user’s profile could be usefully exploited when interacting with a site agent that is able to provide useful recommendations on the basis of the user’s interests, on one hand, and to adapt the site presentation to the device currently exploited by the user, on the other hand. However, it is not suitable to construct such a global profile by a software running on the exploited device since this device (e.g., a mobile phone or a palmtop) may have limited resources. Therefore, in this paper, we propose a multi-agent architecture, called MASHA, handling user and device adaptivity of Web sites, in which each device is provided with a client agent that autonomously collects information about the user’s behaviour associated to just that device. However, the user profile contained in this client is continuously updated with information coming from a unique server agent, associated with the user. Such information is collected by the server agent from the different devices exploited by the user, and represents a global user profile. The third component of this architecture, called adapter agent, is capable to generate a personalized representation of the Web site, containing some useful recommendations derived by both an analysis of the user profile and the suggestions coming from other users exploiting the same device.     

Sports over a Distance

Sport is a domain full of movement-based interactions. These interactions typically have positive health effects as well as an impact on social bonding. We have investigated ways in which computer augmented devices can lead to new sport experiences and explored opportunities to combine physical activities with remote social bonding. Three prototypes have been implemented which showcase movement-based interaction in sports. “Breakout for Two” allows geographically distant users to play a physically exhausting ball game together. “FlyGuy” gives users a hang-glide experience controllable through body movement. “Push’N’Pull” uses isometric exercise equipment over a network to encourage users to complete a cooperative game whilst performing intense muscular actions. A comparison of these applications shows that such movement-based interaction in a networked environment allows players in different locations to achieve a work out and also to socialize. Based on these projects, we conclude with practical design implications for future Exertion Interfaces.     

Distributed user interfaces in public spaces using RFID-based panels

The combination and integration of services between mobile computing and context-aware applications responds to the use of mobile devices defining a wide range of distributed user interfaces to support social activities. In this paper, we propose a novel solution that combines social software with context awareness to improve users' interaction in public spaces. This approach is based on the concept of collaborative interactive panels where users share their opinions and ideas about environmental issues by performing natural gestures. And so, taking advantage of physical resources already available in public spaces combined with the use of well-known technologies, such as mobile devices and RFID, we extend the concept of social software from the Web to physical public scenarios, such as bus stations, squares, etc. As an example, we present a case of study that encourage citizens' participation in decisions related to the community environmental issues reducing the gap between the social software and users.     

Diversified users’ satisfaction with advanced mobile phone features

Mobile phones are widely used all over the world, and with their increasing number of value-added features, they are becoming far more than a mere mobile voice communication device. Rather, they provide a powerful platform for accessing information universally. This paper reports a study which scrutinized users’ preference levels with five new mobile phone design features facilitating universal information access through mobile phones: camera, colour screen, voice-activated dialling, Internet browsing, and wireless connectivity (e.g. Bluetooth, infrared, etc.). The survey study involved college students and investigated the degree to which each of the above features impacts the users’ overall satisfaction and enhances the potential of mobile phones to contribute to different aspects of universal access. Our results show that colour screen, voice-activated dialling, and Internet browsing feature can strongly predict users’ satisfaction level, and their preference levels together account for 22.7% of the variance of the users’ overall satisfaction. Users’ satisfaction levels are significantly different between models with colour screen feature and those without, and models with Internet browsing features and those without. Since mobile phone design needs to accommodate the needs and preferences of diversified user groups, the performed study also investigated the difference in users’ preference levels of the five new features for different ethnic groups, and difference in mobile phone owned by different genders. The result indicates that Asian female users in the U.S. market have higher preference level on colour screen feature than Caucasian female users. Significantly higher percentage of male users own phones with camera, Internet browsing, and wireless connectivity features than female users. The empirical study reported in this paper provides a comprehensive picture of how new design features can enhance the mobile phone as a universal access device, and what impact they have brought about. It can also help manufacturers adopt a universal design perspective in view of the differences in preference levels of users with different ethnicity and genders.     

Supporting device discovery and spontaneous interaction with spatial references

The RELATE interaction model is designed to support spontaneous interaction of mobile users with devices and services in their environment. The model is based on spatial references that capture the spatial relationship of a user’s device with other co-located devices. Spatial references are obtained by relative position sensing and integrated in the mobile user interface to spatially visualize the arrangement of discovered devices, and to provide direct access for interaction across devices. In this paper we discuss two prototype systems demonstrating the utility of the model in collaborative and mobile settings, and present a study on usability of spatial list and map representations for device selection.     

一种基于蓝牙的交互式系统设计与实现

随着无线互联技术和移动终端技术的发展,手机等移动终端用户对蓝牙产品的功能提出了更高的要求。蓝牙采用推送技术进行数据传输,用户只能选择是否接收,不能对蓝牙发送端的数据进行自主交互浏览。采用J2ME和蓝牙技术实现一种在移动蓝牙终端上使用的交互式系统,有效地利用了J2ME 跨平台的优势,以及蓝牙传送数据易用性和实用性等优点。根据对JSR82规范提供的Java蓝牙无线技术API的研究,给出了蓝牙交互式系统的设计方案和系统结构。通过J2ME的无线应用开发包和MIDlet编程模型,实现了交互式系统。     

Implicit interaction design for pervasive workflows

The automatic identification capabilities of mobile devices enable the use of implicit interactions to connect the physical world with digital services. For example, users can be provided with information and services just by approaching to them or pointing to them with a mobile device. This kind of interactions can improve business processes by reducing the gap between physical and digital spaces. This work presents Parkour, a design method for workflows that make use of implicit interactions. Parkour allows designers to indicate how implicit interactions can be orchestrated to support a workflow. Furthermore, reconfiguration techniques have been applied to adapt at run-time the degree in which these interactions intrude the user’s mind. Tool support has been developed to automate the validation of the workflow models and support the system reconfiguration. Finally, the proposal has been applied in the development of several workflows.     

Using ubiquitous computing in interactive mobile marketing

Unique features of handheld devices, including their mobility, personalization and location-awareness engender new types of applications for mobile commerce, such as mobile advertising. Mobile marketing and advertising applications deliver promotional information to consumers based on their preferences and location. In this paper, we present SMMART, a context-aware, adaptive and personalized m-commerce application designed to deliver targeted promotions to the users of mobile devices about the products they like while guarding the users’ identity and protecting them from any unsolicited messages. Promotions distributed by SMMART are personalized by performing intelligent matching of the user’s shopping interests to current promotions available at a retail site. SMMART can adapt to changing preferences of its user by inconspicuously monitoring his or her shopping habits. We describe a fully functional prototype of SMMART built for Pocket PCs running Windows CE with .NET Compact Framework. This paper also presents a study demonstrating end-user usability and economic viability of SMMART.     

M2S maps: supporting real-world navigation with mobile VR

Mobile devices are becoming increasingly integrated into our society. In addition to entertaining people with applications like pervasive games, mobile devices can also help to address cognitive challenges people face in the real world. This paper, by drawing on research findings from cognitive psychology and geography, explores a design to use mobile VR to help people overcome one cognitive barrier in navigation, which is to establish the correspondence between 2D spatial information found in maps and 3D entities they perceive from the real world. The design offers users multi-format, multi-scale, and semantic (M²S) maps, ranging from 2D maps to 3D immersive environments, and helps users to connect 2D maps to the real world through 3D environments which are equipped with semantic representation and animation techniques. Consequently, users can apply various kinds of spatial knowledge, 2D or 3D, in understanding the real world as well as assisting in navigation. This research enhances the design repertoire of mobile VR, and suggests a way to integrate virtual environments into people’s real-world life by examining the cognitive implications of 3D models on users’ activities.     

A framework for bimanual inter-device interactions

A shared interactive display (e.g., a tabletop) provides a large space for collaborative interactions. However, a public display lacks a private space for accessing sensitive information. On the other hand, a mobile device offers a private display and a variety of modalities for personal applications, but it is limited by a small screen. We have developed a framework that supports fluid and seamless interactions among a tabletop and multiple mobile devices. This framework can continuously track each user’s action (e.g., hand movements or gestures) on top of a tabletop and then automatically generate a unique personal interface on an associated mobile device. This type of inter-device interactions integrates a collaborative workspace (i.e., a tabletop) and a private area (i.e., a mobile device) with multimodal feedback. To support this interaction style, an event-driven architecture is applied to implement the framework on the Microsoft PixelSense tabletop. This framework hides the details of user tracking and inter-device communications. Thus, interface designers can focus on the development of domain-specific interactions by mapping user’s actions on a tabletop to a personal interface on his/her mobile device. The results from two different studies justify the usability of the proposed interaction.     

Secure mobile device structure for trust IoT

In the IoT environment, all devices are connected to each other, and mobile device is considered as key device. But hacking into mobile devices is increasing rapidly with the increase in mobile device users. As the market share of Android OS increases, hacking of mobile devices has focused on Android devices. Although there are many security solutions for mobile devices, they are fragmentary for mobile threats; that is, they are solutions for only several threats rather than comprehensive solutions. There is hence a limit to protecting user’s and company’s data stored or used on mobile devices from various types of hacking. To address this, we propose a mobile device protection technology based on domain isolation. Virtualization technology has emerged to increase CPU utilization in server-class PCs and to run various OSs in one system. As these virtualization technologies become lightweight, they are beginning to be applied to embedded devices. In this paper, we applied this lightweight embedded virtualization technology to mobile devices to divide mobile devices into two areas. Therefore, users can have hidden area from hacker attack in addition to Android OS area which can use same as existing mobile device. There is a hardware-based mobile security solution using an secure element, but this has to be reflected in the manufacturing process of the mobile device. However, since the domain separation technology using the virtualization, proposed in this paper, is a software solution, it has an advantage that it can be applied to a device that is already in use. In addition, to protect the hidden area, application authentication/authorization and user authentication technology were applied. And we use white-box cryptography to get root of trust of the key which is used for secure storage and data encryption/decryption. We believe this is a fundamental solution for protecting the mobile device users from hacking. We implemented and tested various mobile applications operating on a mobile device that incorporates our proposed structure based on domain isolation. There is some performance degradation caused by the domain separation, but it is negligible. According to https://www.wired.com/insights/2012/11/mobile-supercomputers/, the chips for mobile phones have evolved and mobile phones will soon become supercomputers. In this case, the addition of virtualization to the mobile device will have less impact on the computing power of the mobile device, and data protection stored in mobile devices and secure execution environment of security programs will become more important issues. Therefore, our TeeMo structure is a necessary technology to protect mobile device users.     

Broadcasting and personalization of user-generated contents in DVB-H mobile networks

During the last years, we have witnessed the boom of the digital market due to the proliferation of emergent audiovisual services and the increasing number of broadband networks. In this scenario, users insistently demand innovative services for exchanging and sharing their own audiovisual contents. In order to meet these needs, in this paper, we propose a system that broadcasts user-generated audiovisual contents for handheld devices in a mobile network based on the DVB-H broadcasting standard. Besides, our system offers diverse value-added services to these new active users, such as: (1) annotation, sharing and personalized distribution of audiovisual contents, (2) multi modal access, via Web or by client applications running locally in the handheld device, and (3) exploitation of return channels to transmit interactive contents that enhance the user’s experience. To achieve these goals, our system adopts well-known technologies for broadcasting and semantic annotation of audiovisual contents, as well as emergent technology from Web 2.0 and Semantic Web. We have carried out tests involving a group of students from the University of Vigo, who were satisfied with the personalization capabilities offered by our TV system for mobile settings.     

蓝牙4.2协议的一主多从防丢器APP设计

设计一款以蓝牙4.2协议为核心,可与多个从机进行通信的防丢器.将低功耗蓝牙芯片CC2541作为主控制器,结合在手机(主机)上开发的APP,通过蓝牙4.2相关协议,对配对成功的防丢器(从机)进行监控,当从机超出主机的监测范围后,主机和从机会同时启动报警装置,以便人们予以警觉.     

A distributed mobile database implementation on Pocket PC mobile devices communicating over Bluetooth

This paper describes a distributed database system implementation built on top of stand-alone mobile databases found on mobile devices. At the heart of the architecture are elected devices that take on the role of data directories which collect the schema of the databases and become the contact points for all nodes that wish to submit queries against the distributed database. The system is implemented on Pocket PCs that run the Microsoft WinCE operating system and communicate using Bluetooth, thus limiting the architecture to eight devices, which is a restriction imposed by piconets. Sample databases were configured on the devices that ran the SQL Server CE database engine, and a list of 170 sample queries of varying complexities were designed to conduct performance evaluation. This evaluation involved measurement of query response time, generated traffic, and device energy consumption. The obtained results indicated the feasibility of the system and its potential for providing mobile users with a framework for aggregating disparate data that are stored in mobile databases in the wireless ad hoc network.     

The narrative construction of our (social) world: steps towards an interactive learning environment for children with autism

Children with autism exhibit a deficit in narrative comprehension which adversely impacts upon their social world. The authors’ research agenda is to develop an interactive software system which promotes an understanding of narrative structure (and thus the social world) while addressing the needs of individual children. This paper reports the results from a longitudinal study, focussing on ‘primitive’ elements of narrative, presented as proto-narratives, in an interactive software game which adapts to the abilities of individual children. A correlation has been found with a real-world narrative comprehension task, and for most children a clear distinction in their understanding of narrative components.     

蓝牙技术及其在遥控器中的应用

蓝牙技术是一种短距离无线通讯技术，在计算机、信息家电以及各种移动设备中嵌入蓝牙技术，能够给用户带来极大的方便。同时，用蓝牙技术取代设备与设备之间的电缆，能够对无线移动数字通讯业务带来巨大的促进作用，本文简要介绍蓝牙技术，并且以蓝牙技术在遥控器中的应用说明如何实现蓝牙的应用模型。     

The design and realization of CoViD: a system for collaborative virtual 3D design

Many important decisions in the design process are made during fairly early on, after designers have presented initial concepts. In many domains, these concepts are already realized as 3D digital models. Then, in a meeting, the stakeholders for the project get together and evaluate these potential solutions. Frequently, the participants in this meeting want to interactively modify the proposed 3D designs to explore the design space better. Today’s systems and tools do not support this, as computer systems typically support only a single user and computer-aided design tools require significant training. This paper presents the design of a new system to facilitate a collaborative 3D design process. First, we discuss a set of guidelines which have been introduced by others and that are relevant to collaborative 3D design systems. Then, we introduce the new system, which consists of two main parts. The first part is an easy-to-use conceptual 3D design tool that can be used productively even by naive users. The tool provides novel interaction techniques that support important properties of conceptual design. The user interface is non-obtrusive, easy-to-learn, and supports rapid creation and modification of 3D models. The second part is a novel infrastructure for collaborative work, which offers an interactive table and several large interactive displays in a semi-immersive setup. It is designed to support multiple users working together. This infrastructure also includes novel pointing devices that work both as a stylus and a remote pointing device. The combination of the (modified) design tool with the collaborative infrastructure forms a new platform for collaborative virtual 3D design. Then, we present an evaluation of the system against the guidelines for collaborative 3D design. Finally, we present results of a preliminary user study, which asked naive users to collaborate in a 3D design task on the new system.     

An innovative mobile electronic tourist guide application

“Mobile tourism” represents a relatively new trend in the field of tourism and involves the use of mobile devices as electronic tourist guides. While much of the underlying technology is already available, there are still open challenges with respect to design, usability, portability, functionality and implementation aspects. Most existing “mobile tourism” solutions either represent of-the-shelf applications with rigidly defined content or involve portable devices with networking capabilities that access tourist content with the requirement of constant airtime, i.e., continuous wireless network coverage. This paper presents the design and implementation issues of a “mobile tourism” research prototype, which brings together the main assets of the two aforementioned approaches. Namely, it enables the creation of portable tourist applications with rich content that matches user preferences. The users may download these personalized applications (optimized for their specific device’s model) either directly to their mobile device or first to a PC and then to a mobile terminal (through infrared or bluetooth). Thereafter, network coverage is not further required as the applications execute in standalone mode and may be updated when the user returns online. The dynamically created tourist applications also incorporate a “push model”, wherein new tourist content is forwarded to the mobile terminal with minimal user intervention as soon as it is added or updated by the administrator. Our prototype has been developed on the top of Java 2 Micro Edition (J2ME) which offers an ideal platform for the development of full-fledged, interactive and portable applications tailored for resource-constrained mobile devices. The paper presents our development experiences with J2ME and highlights its main advantages and shortcomings in relation to the implementation of such kind of applications. Finally, an empirical evaluation of user experience with the mobile application prototype is presented.

VCode—Pervasive Data Transfer Using Video Barcode

In this paper, we describe a novel data transfer scheme that uses the camera in a smart phone as an alternative data channel. The data is encoded as a sequence of 2-D barcode images, displayed on a flat panel display, acquired by the camera, and decoded in real time by the software embedded in device. The decoded data is written to a file. Compared with existing data channels, such as CDMA/GPRS, cables, Bluetooth, and Infrared, our method relies on visual communication and does not require special hardware or data plans. Users only need to point the camera at a monitor displaying the VCode to download. Technical challenges to overcome include correction of perspective distortion, compensation for contrast variation, and efficient implementation of small footprint software into a mobile device. We address these challenges and present our solution in detail. We have implemented a prototype which allows users to download various types of files successfully, including pictures, ring tones and Java games onto camera phones running Symbian and Windows Mobile platforms. We discuss the limitations of our solution and outline future work to overcome these limitations.