Inferring contextual preferences using deep encoder-decoder learners

ABSTRACT

Context-aware systems enable the sensing and analysis of user context in order to provide personalised services. Our study is part of growing research efforts examining how high-dimensional data collected from mobile devices can be utilised to infer users’ dynamic preferences that are learned over time. We suggest novel methods for inferring the category of the item liked in a specific contextual situation, by applying encoder-decoder learners (long short-term memory networks and auto encoders) on mobile sensor data. In these approaches, the encoder-decoder learners reduce the dimensionality of the contextual features to a latent representation which is learned over time. Given new contextual sensor data from a user, the latent patterns discovered from each deep learner is used to predict the liked item’s category in the given context. This can greatly enhance a variety of services, such as mobile online advertising and context-aware recommender systems. We demonstrate our contribution with a point of interest (POI) recommender system in which we label contextual situations with the items’ categories. Empirical results utilising a real world data set of contextual situations derived from mobile phones sensors log show a significant improvement (up to 73% improvement) in prediction accuracy compared with state of the art classification methods.     

Content information desired by Chinese users for effective use of information appliances

Many studies have been conducted to determine how to present information on mobile phones in such a way as to enhance usability, but those studies have not considered what information users need for effective decision making. To investigate what users want from mobile phones, we carried out a content preparation study. A questionnaire was constructed based on prior results from website content research, traditional usability studies on consumer electronics, and extensive study on various types of mobile phones; it was completed by 375 Chinese industrial participants. The statistical results revealed nine major factors of mobile phone content: information relating to input/search, functions, operation, multimedia functions, stored files, phone calls, help/service, accessory functions, and messages. Also, users of different ages and genders evidenced different requirements for mobile phone content, especially concerning accessory and multimedia functions. This study suggests guidelines for mobile phone designers targeted at the Chinese market and provides a base for content study of other information appliances.     

Classifier-Guided Visual Correction of Noisy Labels for Image Classification Tasks

Training data plays an essential role in modern applications of machine learning. However, gathering labeled training data is time-consuming. Therefore, labeling is often outsourced to less experienced users, or completely automated. This can introduce errors, which compromise valuable training data, and lead to suboptimal training results. We thus propose a novel approach that uses the power of pretrained classifiers to visually guide users to noisy labels, and let them interactively check error candidates, to iteratively improve the training data set. To systematically investigate training data, we propose a categorization of labeling errors into three different types, based on an analysis of potential pitfalls in label acquisition processes. For each of these types, we present approaches to detect, reason about, and resolve error candidates, as we propose measures and visual guidance techniques to support machine learning users. Our approach has been used to spot errors in well-known machine learning benchmark data sets, and we tested its usability during a user evaluation. While initially developed for images, the techniques presented in this paper are independent of the classification algorithm, and can also be extended to many other types of training data.     

Cost effective data wiping methods for mobile phone

Thanks to the supply of smartphones, mobile phones have come to store increasingly more personal information of users. To protect users’ information, the information stored in mobile phones should not only be managed safety but also be deleted not to allow restoration. In the filesystems that have been applied to existing mobile phones, even when users have deleted information, data are not completely wiped from the storage. Therefore, the data can be easily recovered by using forensics tools. Considering that mobile phones are always exposed to the risk of robberies and loss, this situation can be misused for personal information spill. The present paper points out problems in the methods of data deleting from smartphones, proposes an efficient data deleting method considering mobile device environments with limitations in battery and hardware performance such as smartphones, and analyzes the efficiency of the techniques in relation to the types of filesystems and data formats.     

MViewer: mobile phone spatiotemporal data viewer

Nowadays movement patterns and people’s behavioral models are needed for traffic engineers and city planners. These observations could be used to reason about mobility and its sustainability and to support decision makers with reliable information. The very same knowledge about human diaspora and behavior extracted from these data is also valuable to the urban planner, so as to localize new services, organize logistics systems and to detect changes as they occur in the movement behavior. Moreover, it is interesting to investigate movement in places like a shopping area or a working district either for commercial purposes or for improving the service quality. These kinds of tracking data are made available by wireless and mobile communication technologies. It is now possible to record and collect a large amount of mobile phone calls in a city. Technologies for object tracking have recently become affordable and reliable and hence we were able to collect mobile phone data from a city in China from January 1, 2008 to December 31, 2008. The large amount of phone call records from mobile operators can be considered as life mates and sensors of persons to inform howmany people are present in any given area and how many are entering or leaving. Each phone call record usually contains the caller and callee IDs, date and time, and the base station where the phone calls are made. As mobile phones are widely used in our daily life, many human behaviors can be revealed by analyzing mobile phone data. Through mobile phones, we can learn the information about locations, communications between mobile phone users during their daily lives. In this work, we propose a comprehensive visual analysis system named as MViewer, Mobile phone spatiotemporal data Viewer, which is the first system to visualize and analyze the population’smobility patterns from millions of phone call records. Our system consists of three major components: 1) visual analysis of user groups in a base station; 2) visual analysis of the mobility patterns on different user groups making phone calls in certain base stations; 3) visual analysis of handoff phone call records. Some well-established visualization techniques such as parallel coordinates and pixelbased representations have been integrated into our system. We also develop a novel visualization schemes, Voronoidiagram-based visual encoding to reveal the unique features of mobile phone data. We have applied our system to real mobile phone datasets that are kindly provided by our project partners and obtained some interesting findings regarding people’s mobility patterns.     

An assisting,constrained 3D navigation technique for multiscale virtual 3D city models

Virtual 3D city models serve as integration platforms for complex geospatial and georeferenced information and as medium for effective communication of spatial information. In order to explore these information spaces, navigation techniques for controlling the virtual camera are required to facilitate wayfinding and movement. However, navigation is not a trivial task and many available navigation techniques do not support users effectively and efficiently with their respective skills and tasks. In this article, we present an assisting, constrained navigation technique for multiscale virtual 3D city models that is based on three basic principles: users point to navigate, users are lead by suggestions, and the exploitation of semantic, multiscale, hierarchical structurings of city models. The technique particularly supports users with low navigation and virtual camera control skills but is also valuable for experienced users. It supports exploration, search, inspection, and presentation tasks, is easy to learn and use, supports orientation, is efficient, and yields effective view properties. In particular, the technique is suitable for interactive kiosks and mobile devices with a touch display and low computing resources and for use in mobile situations where users only have restricted resources for operating the application. We demonstrate the validity of the proposed navigation technique by presenting an implementation and evaluation results. The implementation is based on service-oriented architectures, standards, and image-based representations and allows exploring massive virtual 3D city models particularly on mobile devices with limited computing resources. Results of a user study comparing the proposed navigation technique with standard techniques suggest that the proposed technique provides the targeted properties, and that it is more advantageous to novice than to expert users.     

Infomages: Embedding Data into Thematic Images

Recent studies have indicated that visually embellished charts such as infographics have the ability to engage viewers and positively affect memorability. Fueled by these findings, researchers have proposed a variety of infographic design tools. However, these tools do not cover the entire design space. In this work, we identify a subset of infographics that we call infomages. Infomages are casual visuals of data in which a data chart is embedded into a thematic image such that the content of the image reflects the subject and the designer's interpretation of the data. Creating an effective infomage, however, can require a fair amount of design expertise and is thus out of reach for most people. In order to also afford non-artists with the means to design convincing infomages, we first study the principled design of existing infomages and identify a set of key chart embedding techniques. Informed by these findings we build a design tool that links web-scale image search with a set of interactive image processing tools to empower novice users with the ability to design a wide variety of infomages. As the embedding process might introduce some amount of visual distortion of the data our tool also aids users to gauge the amount of this distortion, if any. We experimentally demonstrate the usability of our tool and conclude with a discussion of infomages and our design tool.     

Don't Peek at My Chart: Privacy-preserving Visualization for Mobile Devices

Data visualizations have been widely used on mobile devices like smartphones for various tasks (e.g., visualizing personal health and financial data), making it convenient for people to view such data anytime and anywhere. However, others nearby can also easily peek at the visualizations, resulting in personal data disclosure. In this paper, we propose a perception-driven approach to transform mobile data visualizations into privacy-preserving ones. Specifically, based on human visual perception, we develop a masking scheme to adjust the spatial frequency and luminance contrast of colored visualizations. The resulting visualization retains its original information in close proximity but reduces visibility when viewed from a certain distance or farther away. We conducted two user studies to inform the design of our approach (N=16) and systematically evaluate its performance (N=18), respectively. The results demonstrate the effectiveness of our approach in terms of privacy preservation for mobile data visualizations.     

Pedestrian navigation using the sense of touch

Haptics is a feedback technology that takes advantage of the human sense of touch by applying forces, vibrations, and/or motions to a haptic-enabled user device such as a mobile phone. Historically, human–computer interaction has been visual, data, or images on a screen. Haptic feedback can be an important modality in Mobile Location-Based Services like – knowledge discovery, pedestrian navigation and notification systems. In this paper we describe a methodology for the implementation of haptics in four distinct prototypes for pedestrian navigation. Prototypes are classified based on the user’s navigation guidance requirements, the user type (based on spatial skills), and overall system complexity. Here haptics is used to convey location, orientation, and distance information to users using pedestrian navigation applications. Initial user trials have elicited positive responses from the users who see benefit in being provided with a “heads up” approach to mobile navigation. We also tested the spatial ability of the user to navigate using haptics and landmark images based navigation. This was followed by a test of memory recall about the area. Users were able to successfully navigate from a given origin to a Destination Point without the use of a visual interface like a map. Results show the users of haptic feedback for navigation prepared better maps (better memory recall) of the region as compared to the users of landmark images based navigation.     

SOMAR: A SOcial Mobile Activity Recommender

A 2010 survey (Nielsen) showed that 22.7% of the time spent on the Internet is on a social network. Moreover, there is an increasing demand to access social networks by mobile phones, i.e., around 30% globally. Social networking has become a reality, and it generates an incredible amount of information that is sometimes difficult for users to process, especially from mobile phones. Several links, activities, and recommendations are proposed by networked friends every hour, which together are nearly impossible to manage. There is a need to filter and make accessible such information to users, which is the motivation behind developing a mobile recommender that exploits social network information. Thus, in this paper, we propose the design and the implementation of a SOcial Mobile Activity Recommender (SOMAR) that can integrate Facebook social network mobile data and sensor data to propose activities to the user (e.g., concert or computer science seminar). The recommendations are completely calculated in situ in the mobile device with an embedded data mining component. We analyze how to compute and update the social graph in case of changes in social relationships or user context. The paper also presents a case study to analyze the performance of the method.     

On the analysis of statistics of mobile visitors

This paper discusses location-based mobile services. The problem of counting mobile users (mobile phones) in a selected area is considered. The information available from the analysis of wire-less protocols (Wi-Fi, Bluetooth) is used for the calculation. The aim of the study is to construct an analog of systems of web statistics operating with real mobile subscribers (instead of data on web page visits as in web statistics). As a result, we obtain information about traffic, identification and analysis of trends in user traffic, search for the core of regular visitors, and detection of its dynamics. The paper presents algorithms for calculating network proximity and examples of use.     

Wireless advertising's challenges and opportunities

Rapidly developing wireless net-working technology and the growing mobile-device user base have fueled interest in activities that deliver advertisements to mobile devices over a wireless network. Studies by wireless media research companies indicate that delivering permission-based alerts to wireless phones captures consumers' attention, drives response actions, and builds brand awareness. Wireless devices are accessible, personal, and location aware. These characteristics allow for highly targeted, flexible, and dynamic wireless advertisements. Yet the target audience is vast, and users must be able to search for information, issue inquiries, and make purchases at any mobile location.     

Age differences in information finding tasks: Performance and visual exploration strategy with different web page layouts

Recent research suggests that older Internet users seem to find it more difficult to locate navigation links than to find information content in web pages. One possibility is that older Internet users’ visual exploration of web pages is more linear in nature, even when this type of processing is not appropriate for the task. In the current study, the eye movements of young and older Internet users were recorded using an ecological version of the web pages or a discursive version designed to induce a linear exploration. The older adults found more targets when performing content-oriented compared to navigation-oriented searches, thus replicating previous results. Moreover, they performed less well than young people only when required to locate navigation links and tended to outperform the younger participants in content-oriented searches. Although the type of search task and type of web page resulted in different visual strategies, little or no support was found for the hypothesis that older participants explore web pages in a more linear way in cases where this strategy was not appropriate. The main conclusion is that differences in visual exploration do not seem to mediate the specific difficulty older adults experience in navigation-oriented searches in web pages.     

Efficient techniques for time-constrained information dissemination using location-based social networks

Social networks have undergone an explosive growth in recent years. They constitute a central part of users׳ everyday lives as they are used as major tools for the spread of information, ideas and notifications among the members of the network. In this work we investigate the use of location-based social networks as a medium of emergency notification, for efficient dissemination of emergency information among members of the social network under time constraints. Our objective is the following: given a location-based social network comprising a number of mobile users, the social relationships among the users, the set of recipients, and the corresponding timeliness requirements, our goal is to select an appropriate subset of users so that the spread of information is maximized, time constraints are satisfied and costs are considered. We propose LATITuDE, our system that investigates the interactions among the members of the social network to infer their social relationships, and develop scalable dissemination mechanisms that select the most efficient set of users to initiate the dissemination process in order to maximize the information reach among the appropriate receivers within a time window. Our detailed experimental results illustrate that our approach is practical, effectively addresses the problem of informing the appropriate set of users within a deadline when an emergency event occurs, uses a small number of messages, and consistently outperforms its competitors.     

The sensor internet at work: Locating everyday items using mobile phones

We present a system for monitoring and locating everyday items using mobile phones. The system is based on phones which are enhanced with the capability to detect electronically tagged objects in their vicinity. It supports various functionalities: On the one hand, phones can store the context in which users leave registered items and thus help to locate them later on. On the other hand, object owners can search for their objects using the infrastructure of mobile phones carried by other users. We describe the design of our object location system and provide an algorithm which can be used to search for lost or misplaced items efficiently by selecting the most suitable sensors based on arbitrary domain knowledge. Furthermore, we demonstrate the practicability of such wide-area searching by means of user-held sensors in a series of simulations complemented by a real-world experiment.     

Quantitative Comparison of Time-Dependent Treemaps

Rectangular treemaps are often the method of choice to visualize large hierarchical datasets. Nowadays such datasets are available over time, hence there is a need for (a) treemaps that can handle time-dependent data, and (b) corresponding quality criteria that cover both a treemap's visual quality and its stability over time. In recent years a wide variety of (stable) treemapping algorithms has been proposed, with various advantages and limitations. We aim to provide insights to researchers and practitioners to allow them to make an informed choice when selecting a treemapping algorithm for specific applications and data. To this end, we perform an extensive quantitative evaluation of rectangular treemaps for time-dependent data. As part of this evaluation we propose a novel classification scheme for time-dependent datasets. Specifically, we observe that the performance of treemapping algorithms depends on the characteristics of the datasets used. We identify four potential representative features that characterize time-dependent hierarchical datasets and classify all datasets used in our experiments accordingly. We experimentally test the validity of this classification on more than 2000 datasets, and analyze the relative performance of 14 state-of-the-art rectangular treemapping algorithms across varying features. Finally, we visually summarize our results with respect to both visual quality and stability to aid users in making an informed choice among treemapping algorithms. All datasets, metrics, and algorithms are openly available to facilitate reuse and further comparative studies.     

How information within the perceptual span guides visual search and aids perception

We explore the guiding effect of task-related information appearing within the perceptual span on visual search based on elements such as the range of effective information attention within the perceptual span and the orientation of task-related objects. In the first experiment, we recorded reaction time and eye movement data of 48 subjects performing a visual search task for target information in the navigation interface and divided them into four experimental groups according to the perceptual span (within or outside the range) and orientation (left or right) of the task-related information. The experimental results showed that: (a) task-related information appearing within the perceptual span had a guiding effect on target search; and (b) the right side closer to the target in task search have better performance in guiding. In the second experiment, eye-movement experiments of the navigation interface based on the eye-movement processing theoretical model of linguistics showed that: (a) within the perceptual span, the shifting of attention causing eye movements (saccade) and brings gaze into the area of interest, and (b) the perceptual span moves with attention shifts. This study provides evidence for cognitive processing of information responses within the perceptual span and dynamic adjustment of perceptual span location and provides a reliable method for effective information presentation and visual layout in navigation interfaces, with a view to providing reference and learning for related studies.     

Learning by foraging: The impact of individual knowledge and social tags on web navigation processes

The World Wide Web provides a tremendously large quantity of information. When users search for information or products on the Web, they will presumably be inclined to choose their path of navigation on the basis of their prior knowledge. In those cases in which the prior knowledge of users is incorrect, however, this navigation process is assumed to lead to suboptimal search results. In an experimental study with 180 participants, we examined to what extent both the users’ prior knowledge and social tags - which capture the collective knowledge of a Web community in tag clouds - influenced the navigation of users and triggered incidental learning processes during the Web search. The results supported the assumption that the users’ prior knowledge is indeed crucial for navigation, as users followed those tags which corresponded to their internal associations. Moreover, we found that social tags also affected the navigation behavior of users, as a strong collective association of social tags led to a high selection rate for these tags. Finally, the results showed that social tags triggered incidental learning processes, as those internal associations which corresponded to tags with a strong collective association were strengthened during navigation. The implications of these findings for further research are discussed.