Evolution towards smart home environments: empirical evaluation of three user interfaces

Smart home environments have evolved to the point where everyday objects and devices at home can be networked to give the inhabitants new means to control them. Familiar information appliances can be used as user interfaces (UIs) to home functions to achieve a more convenient user experience. This paper reports an ethnographic study of smart home usability and living experience. The purpose of the research was to evaluate three UIs—a PC, a media terminal, and a mobile phone—for smart home environments. The results show two main types of activity patterns, pattern control and instant control, which require different UI solutions. The results suggest that a PC can act as a central unit to control functions for activity patterns that can be planned and determined in advance. The mobile phone, on the other hand, is well suited for instant control. The mobile phone turned out to be the primary and most frequently used UI during the 6-month trial period in the smart apartment.     

Mixed reality participants in smart meeting rooms and smart home environments

Human–computer interaction requires modeling of the user. A user profile typically contains preferences, interests, characteristics, and interaction behavior. However, in its multimodal interaction with a smart environment the user displays characteristics that show how the user, not necessarily consciously, verbally and nonverbally provides the smart environment with useful input and feedback. Especially in ambient intelligence environments we encounter situations where the environment supports interaction between the environment, smart objects (e.g., mobile robots, smart furniture) and human participants in the environment. Therefore it is useful for the profile to contain a physical representation of the user obtained by multi-modal capturing techniques. We discuss the modeling and simulation of interacting participants in a virtual meeting room, we discuss how remote meeting participants can take part in meeting activities and they have some observations on translating research results to smart home environments.     

Problem solving,confidence and frustration when carrying out familiar tasks on non-familiar mobile devices

Smart mobile devices, which are hand-held electronic devices with an advanced operating system (such as the Android platform) connected via a wireless protocol, have become an integral and essential part of our everyday life, and support both social and workplace activities. However, adopting mobile technology within the workplace setting can give rise to challenges that impact user behaviour and performance. A study was carried out amongst 90 participants located in two countries, using internet connectivity as a case study. Confidence and frustration have previously been connected with technology competence, but this was not applied to a workplace scenario during problem-solving, when users are assigned an unfamiliar smart mobile device. This research focuses on identifying the link between workplace users' levels of confidence and frustration when seeking to independently solve problems whilst completing familiar tasks on new smart mobile devices. A detailed video analysis of users' attitudes and behaviour during problem-solving was conducted, emphasising a correlation between attitudes and behaviour towards completing a task.     

Development of a virtual aquarium system interacting with a smart device

New applications of smart devices interacting with other computing devices are recently providing interesting and feasible solutions in ubiquitous computing environments. In this study, we propose an interactive virtual aquarium system that interacts with a smart device as a user interface. We developed a virtual aquarium graphic system and a remote interaction application of a smart device for building an interactive virtual aquarium system. We performed an experiment that demonstrates the feasibility and the effectiveness of the proposed system as an example of a new type of interactive application of a smart display, where a smart device serves as a remote user interface.     

A remote control and media-sharing system using smart devices

The remote control and media sharing of electronic devices are key services in smart homes. The incorporation of mobile smart devices in these services has become a popular trend. Existing services require that these devices are located in the same local network. This paper presents the design and implementation of an integrated service architecture that supports the remote control of home appliances and the sharing of digital media between indoor and outdoor devices. The proposed design follows standards related to digital homes, and this study presents the details of its hardware and software components.     

Automatic adaptation of mobile applications to different user devices using modular mobile agents

Wearable, handheld, and embedded or standalone intelligent devices are becoming quite common and can support a diverse range of applications. In order to simplify development of applications which can adapt to a variety of mobile devices, we propose an adaptation framework which includes three techniques: follow‐me, context‐aware adaptation, and remote control scheme. For the first, we construct a personal agent capable of carrying its owner's applications. Second, we design a personal agent capable of carrying applications with an adaptable hierarchical structure. Then, applications can be adapted approximately to the context of evices by using an attribute‐based component decision algorithm. Finally, to achieve a remote control scheme, we distribute the computational load of applications on the resource‐restricted mobile devices. An application is divided into two parts that can be executed on a user device and a server separately. In short, this framework facilitates the development of widespread applications for ubiquitous computing environments. Furthermore, it enables the applications to follow their owners and automatically adapt to different devices. Copyright © 2005 John Wiley & Sons, Ltd.     

Architectural patterns for applications with external user interface elements

Coupling mobile devices and other remote interaction technology with software systems surrounding the user enables for building interactive environments under explicit user control. The realization of explicit interaction in ubiquitous or pervasive computing environments introduces a physical distribution of input devices, and technology embedded into the environment of the user. To fulfill the requirements of emerging trends in mobile interaction, common approaches for system design need adaptations and extensions. This paper presents the adaptation and extension of the Model-View-Controller approach to design applications of remote, complementary, duplicated and detached user interface elements.     

Design and implementation of a smart system for personalization and accurate selection of mobile services

Wireless mobile services are computing applications that run on handheld wireless devices. Such applications must work within the daunting constraints of the devices, which include memory, processing power, input capabilities, and size of display. It is therefore important that mobile services take into account the user’s context, optimize resource usage, and minimize input effort imposed on the user. In this paper, we present the design and implementation of a smart agent-enabled system for personalizing wireless mobile services and advertisements for Java 2 Micro Edition (J2ME) or Java ME, and Wireless Application Protocol (WAP) enabled devices. We use software agents for context filtering because such autonomous software entities have characteristics that can benefit mobile devices and the wireless environment, and the Composite Capability/Preference Profiles (CC/PP) standard for defining profiles for user preferences and device capabilities. The system incorporates the use of artificial neural networks to adaptively and iteratively learn to select the best available service based on contextual information. The system is evaluated using practical operating scenarios, as well as empirical data and results show an 87% success rate in the selection of the best available service.     

基于开源软硬件的智能家居系统设计与实现

针对目前智能家居系统中存在的问题．设计并实现一种基于开源软硬件的智能家居系统。采用树莓派作为系统的嵌入式网关和服务器．开源硬件Arduino控制器作为控制终端,实现对家居各种信息的检测和设备的控制。采用Wi—Fi,蓝牙等无线通信模块建立家庭内部无线网络．实现各模块之间的无线通信。同时设计网页和手机客户端,使用用户能够远程对家中的情况进行监控．同时对家电进行控制。     

Intelligent association selection of embedded agents in intelligent inhabited environments

Recent advances in technology and manufacturing have resulted in more powerful and smaller processors to be embedded in the various artefacts within smart environments. Most of these artefacts are network enabled and thanks to pervasive networking such artefacts can communicate and collaborate together to support our daily lives. Furthermore, these artefacts can also be equipped with embedded agents to provide intelligent reasoning, planning and learning capabilities. However, the multitude of interconnected devices and artefacts can result in major network and processing delays as well as creating inherent complexities in programming and configuring smart environments to personalise themselves to suit the individual needs. Hence, a major challenge to the design and use of smart environments involves finding the best set of device associations and interconnections that are most suitable to the environment and user needs. In this paper, we will present a novel intelligent method for reducing the number of associations and interconnections between the various devices and artefacts within smart environments to minimise the network and processing overheads while reducing the cognitive load associated with configuring and programming smart environments.     

Robust authentication and key agreement scheme preserving the privacy of secret key

In ubiquitous computing environments, people may obtain their services from application servers by using mobile devices at any time and anywhere. For convenience, most of those devices are small and of limited power and computation capacity. In this paper, we propose a robust user authentication and key agreement scheme suitable for ubiquitous computing environments. The main merits include: (1) a security-sensitive verification table is not required in the server; (2) the password can be chosen and changed freely by the clients and cannot be derived by the privileged administrator of the server; (3) all well-known security threats are solved in our proposed scheme; (4) the scheme does not have a serious time-synchronization problem; (5) the client and the server can establish a common session key; (6) the scheme is practical and efficient; (7) the scheme can preserve the privacy of the client’s secret key even if the secret information stored in a smart card is compromised.     

The smart phone: a ubiquitous input device

We show how modern mobile phones (Weiser's tabs) can interact with their environment, especially large situated displays (Weiser's boards). Smart phones' emerging capabilities are fueling a rise in the use of mobile phones as input devices to such resources as situated displays, vending machines, and home appliances. Mobile phones' prevalence gives them great potential to be the default physical interface for ubiquitous computing applications. We survey interaction techniques that use mobile phones as input devices to ubiquitous computing environments. We use smart phone to describe an enhanced mobile phone. Our analysis blurs the line between smart phones and PDAs such as the Palm Pilot because the feature sets continue to converge.     

DewSim: A trace-driven toolkit for simulating mobile device clusters in Dew computing environments

Dew computing is an emerging computing paradigm, which aims at minimizing the dependency over existing internetwork back-haul, ie, being dependent on processing resources offered by remote servers. Smartphones and tablets ubiquity and powerful computing hardware motivated researchers to investigate the way of providing Dew computing services by exploiting the aggregated capabilities of devices in a vicinity, a smart device cluster. Consequently, research on resource management is necessary to learn how to scavenge resources from such a cluster, deal with devices heterogeneity, limitations, and dynamic resource availability. Simulation is commonly practiced for studying resource management in other distributed computing research fields, specially due to the complexity involved in the set up of experiments. However, a free-to-use purpose specific toolkit for studying smart device clusters do not exist or have been documented. Current simulation efforts do not allow researchers to faithfully represent key singularities of such environment, which are energy depletion and nondedicated nature of computing resources. We propose a trace-based toolkit built on modular software artifacts to speed up research in resource management techniques in Dew environments. A trace-driven methodology is adopted to assure practical value of simulated scenarios. The toolkit comprises a device profiler application for Android to capture generic battery and CPU traces from real devices, a profile mixer to create user interaction baseline traces through generic ones, and an extensible engine to simulate the execution of workloads configurable via text files. Verification and validation tests were run to show correctness and reliability of our simulation approach.     

Self-organizable panel for assembling DIY ubiquitous computing

This paper proposes the concept of DIY (do-it-yourself) ubiquitous computing, an architecture allowing non-experts to establish ubiquitous computing environments in the real world. This concept has been implemented in the “u-Texture”, which is a self-organizable panel that works as a building block. While the traditional scheme attaches devices such as computers, sensors, and network equipments externally to make everyday objects smart, the u-Texture has these devices built in beforehand to assemble smart objects. The u-Texture can change its own behavior autonomously through recognition of its location, its angle of inclination, and surrounding environment by assembling these factors physically. This paper describes the design, the implementation, and various applications of u-Textures to confirm that the concept can contribute to establishment of ubiquitous computing environments in the real world without expert users.     

Effects of Feedforward in In-Air Remote Pointing

Remote pointing devices like the Wii remote have a wide range of applications and are becoming more important for the manipulation of and interactions with information on a distant display such as smart TVs. Because remote pointing devices are used without external support, however, muscular tremors and motional disparity between the display and motor space can result in usability problems of mouse jitters and instability. In this research, a solution is proposed to those problems using feedforward technology, where a user is provided with predictive information in multisensory modalities while approaching a target. Also, gender effect on the user experience of remote pointing devices is examined. By conducting two experiments and a survey, it was found that the feedforward signal plays a role of precue and is more effective than the typical feedback. It was also found that the modality variations in feedforward were impacted by the gender of the user. The findings can be used to improve user interfaces for remote pointing controllers.     

The reflective mobile agent paradigm implemented in a smart office environment

Ubiquitous systems will integrate computers invisibly and unobtrusively in everyday objects. Data will be catched from single or multi-sensor devices and will be used for context extraction. New location-based services will be adapted to user preferences. For this the ubiquitous system needs to know user profiles, likings, and habits. As the user moves, these information must be made available at the new location of the user. Either the user carries the data on wearable or portable computers or the smart environment takes responsibility for transporting them. The amount of new devices and services makes an efficient use by centralized systems very difficult. The idea presented in this paper is that a virtual reflection of the user represented by a mobile agent accompanying in the smart environment. Mobile agents offer a possibility to encapsulate information of a person and the person’s preferences and perform location-based services of the ubiquitous system in the name of the user. Security and privacy are major concerns of such an agent system. This paper describes a ubiquitous mobile agent system named UbiMAS which has security extensions to provide high protection of agents and significant personal data. UbiMAS is applied in the smart doorplate project as part of a smart office environment.     

Cryptanalysis and security enhancement of a ‘more efficient & secure dynamic ID-based remote user authentication scheme’

Remote user authentication is a method, in which remote server verifies the legitimacy of a user over an insecure communication channel. Currently, smart card-based remote user authentication schemes have been widely adopted due to their low computational cost and convenient portability for the authentication purpose. Recently, Wang et al. proposed a dynamic ID-based remote user authentication scheme using smart cards. They claimed that their scheme preserves anonymity of user, has the features of strong password chosen by the server, and protected from several attacks. However, in this paper, we point out that Wang et al.’s scheme has practical pitfalls and is not feasible for real-life implementation. We identify that their scheme: does not provide anonymity of a user during authentication, user has no choice in choosing his password, vulnerable to insider attack, no provision for revocation of lost or stolen smart card, and does provide session key agreement. To remedy these security flaws, we propose an enhanced authentication scheme, which covers all the identified weaknesses of Wang et al.’s scheme and is more secure and efficient for practical application environment.     

An anonymous mobile user authentication protocol using self-certified public keys based on multi-server architectures

As a smart phone becomes a daily necessity, mobile services are springing up. A mobile user should be authenticated and authorized before accessing these mobile services. Generally, mobile user authentication is a method which is used to validate the legitimacy of a mobile login user. As the rapid booming of computer networks, multi-server architecture has been pervasive in many network environments. Much recent research has been focused on proposing password-based remote user authentication protocols using smart cards for multi-server environments. To protect the privacy of users, many dynamic identity based remote user authentication protocols were proposed. In 2009, Hsiang and Shih claimed their protocol is efficient, secure, and suitable for the practical application environment. However, Sood et al. pointed out Hsiang et al.’s protocol is susceptible to replay attack, impersonation attack and stolen smart card attack. Moreover, the password change phase of Hsiang et al.’s protocol is incorrect. Thus, Sood et al. proposed an improved protocol claimed to be practical and computationally efficient. Nevertheless, Li et al. found that Sood et al.’s protocol is still vulnerable to leak-of-verifier attack, stolen smart card attack and impersonation attack and consequently proposed an improvement to remove the aforementioned weaknesses. In 2012, Liao et al. proposed a novel pairing-based remote user authentication protocol for multi-server environment, the scheme based on elliptic curve cryptosystem is more secure and efficient. However, through careful analyses, we find that Liao et al.’s protocol is still susceptible to the trace attack. Besides, Liao et al.’s protocol is inefficient since each service server has to update its ID table periodically. In this paper, we propose an improved protocol to solve these weaknesses. By enhancing the security, the improved protocol is well suited for the practical environment.     

A hierarchical, agent-based service oriented architecture for smart environments

Service Oriented Architectures, which allow for the integration of different subsystems and of applications running on different devices and platforms, may be very suitable to solve the problem of service personalization in large smart environments like cities, where the number of potential users and potentially available services is rapidly growing. Taking this into account, we have designed an agent-based service oriented architecture for smart spaces. Two of the greatest challenges in the design of such a solution are providing effective device, service and context federation and composition mechanisms and handling user mobility. For the first challenge, we have designed a hierarchical architecture and developed a set of inheritance, aggregation and access mechanisms for devices, services and context. To handle user mobility, we have followed three different strategies, using stationary, mobile or nomadic agents. In this paper, we describe the main aspects of our architecture and perform an experimental evaluation to determine the advantages and drawbacks of the different strategies.     

The Plan B OS for ubiquitous computing. Voice control, security, and terminals as case studies

The conventional approach to building pervasive environments relies on middleware to integrate different systems. Instead, we have built a system that can deal with these environments by exporting system resources through distributed virtual file systems. This requires no middleware, simplifies interoperation, and permits the application of general purpose tools to any system resource. A constraint-based file system import mechanism allows the system to adapt to changes in the environment and permits users to customize the environment and tailor adaptations according to their needs. The system has been in use for over a year to carry out our daily work and is underlying the smart space that we built for our department. The system, and some novel services, including ubiquitous voice interfaces, a distributed security architecture, and remote terminals for smart spaces, are also described in this paper.