Unobtrusive touch‐free interaction on mobile devices in dirty working environments

In this work, we propose a new mode of interaction using hand gestures captured by the back camera of a mobile device. Using a simple intuitive two‐finger picking gesture, the user can perform mouse click and drag‐and‐drop operations from the back of the device, which provides an unobstructed touch‐free interaction. This method allows users to operate on a tablet and maintain full awareness of the display, which is especially suitable for certain working environments, such as the machine shop, garage, kitchen, gym, or construction site, where people's hands may be dirty, wet, or in gloves. The speed, accuracy, and error rate of this interaction have been evaluated and compared with the typical touch interaction in an empirical study. The results show that, although this method is not, in general, as efficient and accurate as direct touch, participants still consider it an effective and intuitive method of interacting with mobile devices in environments where direct touch is impractical.     

On‐the‐Fly Power‐Aware Rendering

Power saving is a prevailing concern in desktop computers and, especially, in battery‐powered devices such as mobile phones. This is generating a growing demand for power‐aware graphics applications that can extend battery life, while preserving good quality. In this paper, we address this issue by presenting a real‐time power‐efficient rendering framework, able to dynamically select the rendering configuration with the best quality within a given power budget. Different from the current state of the art, our method does not require precomputation of the whole camera‐view space, nor Pareto curves to explore the vast power‐error space; as such, it can also handle dynamic scenes. Our algorithm is based on two key components: our novel power prediction model, and our runtime quality error estimation mechanism. These components allow us to search for the optimal rendering configuration at runtime, being transparent to the user. We demonstrate the performance of our framework on two different platforms: a desktop computer, and a mobile device. In both cases, we produce results close to the maximum quality, while achieving significant power savings.     

Hyperlinking reality via camera phones

A novel user interface concept for camera phones, called “Hyperlinking Reality via Camera Phones”, that we present in this article, provides a solution to one of the main challenges facing mobile user interfaces, that is, the problem of selection and visualization of actions that are relevant to the user in her current context. Instead of typing keywords on a small and inconvenient keypad of a mobile device, a user of our system just snaps a photo of her surroundings and objects in the image become hyperlinks to information. Our method commences by matching a query image to reference panoramas depicting the same scene that were collected and annotated with information beforehand. Once the query image is related to the reference panoramas, we transfer the relevant information from the reference panoramas to the query image. By visualizing the information on the query image and displaying it on the camera phone’s (multi-)touch screen, the query image augmented with hyperlinks allows the user intuitive access to information.     

Sketch‐Based Editing Tools for Tumour Segmentation in 3D Medical Images

In the past years sophisticated automatic segmentation algorithms for various medical image segmentation problems have been developed. However, there are always cases where automatic algorithms fail to provide an acceptable segmentation. In these cases the user needs efficient segmentation editing tools, a problem which has not received much attention in research. We give a comprehensive overview on segmentation editing for three‐dimensional (3D) medical images. For segmentation editing in two‐dimensional (2D) images, we discuss a sketch‐based approach where the user modifies the segmentation in the contour domain. Based on this 2D interface, we present an image‐based as well as an image‐independent method for intuitive and efficient segmentation editing in 3D in the context of tumour segmentation in computed tomography (CT). Our editing tools have been evaluated on a database containing 1226 representative liver metastases, lung nodules and lymph nodes of different shape, size and image quality. In addition, we have performed a qualitative evaluation with radiologists and technical experts, proving the efficiency of our tools.     

An Improved Geometric Approach for Palette‐based Image Decomposition and Recoloring

Palette‐based image decomposition has attracted increasing attention in recent years. A specific class of approaches have been proposed basing on the RGB‐space geometry, which manage to construct convex hulls whose vertices act as palette colors. However, such palettes do not guarantee to have the representative colors which actually appear in the image, thus making it less intuitive and less predictable when editing palette colors to perform recoloring. Hence, we proposed an improved geometric approach to address this issue. We use a polyhedron, but not necessarily a convex hull, in the RGB space to represent the color palette. We then formulate the task of palette extraction as an optimization problem which could be solved in a few seconds. Our palette has a higher degree of representativeness and maintains a relatively similar level of accuracy compared with previous methods. For layer decomposition, we compute layer opacities via simple mean value coordinates, which could achieve instant feedbacks without precomputations. We have demonstrated our method for image recoloring on a variety of examples. In comparison with state‐of‐the‐art works, our approach is generally more intuitive and efficient with fewer artifacts.     

Component‐wise Controllers for Structure‐Preserving Shape Manipulation

Recent shape editing techniques, especially for man‐made models, have gradually shifted focus from maintaining local, low‐level geometric features to preserving structural, high‐level characteristics like symmetry and parallelism. Such new editing goals typically require a pre‐processing shape analysis step to enable subsequent shape editing. Observing that most editing of shapes involves manipulating their constituent components, we introduce component‐wise controllers that are adapted to the component characteristics inferred from shape analysis. The controllers capture the natural degrees of freedom of individual components and thus provide an intuitive user interface for editing. A typical model usually results in a moderate number of controllers, allowing easy establishment of semantic relations among them by automatic shape analysis supplemented with user interaction. We propose a component‐wise propagation algorithm to automatically preserve the established inter‐relations while maintaining the defining characteristics of individual controllers and respecting the user‐specified modeling constraints. We extend these ideas to a hierarchical setup, allowing the user to adjust the tool complexity with respect to the desired modeling complexity. We demonstrate the effectiveness of our technique on a wide range of man‐made models with structural features, often containing multiple connected pieces.     

Viewfinder Alignment

The viewfinder of a digital camera has traditionally been used for one purpose: to display to the user a preview of what is seen through the camera's lens. High quality cameras are now available on devices such as mobile phones and PDAs, which provide a platform where the camera is a programmable device, enabling applications such as online computational photography, computer vision‐based interactive gaming, and augmented reality. For such online applications, the camera viewfinder provides the user's main interaction with the environment. In this paper, we describe an algorithm for aligning successive viewfinder frames. First, an estimate of inter‐frame translation is computed by aligning integral projections of edges in two images. The estimate is then refined to compute a full 2D similarity transformation by aligning point features. Our algorithm is robust to noise, never requires storing more than one viewfinder frame in memory, and runs at 30 frames per second on standard smartphone hardware. We use viewfinder alignment for panorama capture, low‐light photography, and a camera‐based game controller.     

Communicating Through Location: The Understood Meaning of the Foursquare Check‐In

Location‐based social networks (LBSNs) are mobile applications that allow people to share their physical location with friends through their mobile device. The sharing of location is a relatively new form of computer‐mediated communication, and there is a lack of existing research examining the coordination practices of people using social location sharing services. This article reports on qualitative interviews with frequent users of the LBSN Foursquare to show both how LBSNs complicate views of the relative “placelessness” of traditional mobile communication and how the design of Foursquare complicates its utility as a mobile coordination tool.     

基于Android的智能图像共享系统的研究

Android以其开源性、易用性和良好的用户体验,成为智能手机及平板电脑主流的操作系统和移动应用开发平台。对Android系统架构以及基于此平台的应用程序的设计和实现过程进行研究,设计并实现了智能图像共享系统。该系统顺应3G移动互联网技术和微博蓬勃发展的趋势,实现了移动终端对图像的上传、特效编辑、即拍即传、地理信息共享等功能。     

Bubble-sensing: Binding sensing tasks to the physical world

We propose bubble-sensing, a new sensor network abstraction that allows mobile phone users to create a binding between sensing tasks and the physical world at locations of interest, that remains active for a duration set by the user. We envision mobile phones being able to affix sensing task bubbles at places of interest and then receive sensed data as it becomes available in a delay-tolerant fashion, in essence, creating a living documentary of places of interest in the physical world. The system relies on other mobile phones that opportunistically pass through bubble-sensing locations to acquire tasks and do the sensing on behalf of the initiator, and deliver the data to the bubble-sensing server for retrieval by the user who initiated the task.We describe an implementation of the bubble-sensing system using sensor-enabled mobile phones, specifically, Nokia’s N80 and N95 (with GPS, accelerometers, microphone, camera). Task bubbles are maintained at locations through the interaction of “bubble carriers”, which carry the sensing task into the area of interest, and “bubble anchors”, which maintain the task bubble in the area when the bubble carrier is no longer present. In our implementation, bubble carriers and bubble anchors implement a number of simple mobile phone based protocols that refresh the task bubble state as new mobile phones move through the area. Phones communicate using the local Ad-Hoc 802.11g radio to transfer task state and maintain the task in the region of interest. This task bubble state is ephemeral and times out when no bubble carriers or bubble anchors are in the area. Our design is resilient to periods when no mobiles pass through the bubble area and is capable of “reloading” the task into the bubble region. In this paper, we describe the bubble-sensing system and a simple proof-of-concept experiment.     

ScribbleBoost: Adding Classification to Edge‐Aware Interpolation of Local Image and Video Adjustments

One of the most common tasks in image and video editing is the local adjustment of various properties (e.g., saturation or brightness) of regions within an image or video. Edge‐aware interpolation of user‐drawn scribbles offers a less effort‐intensive approach to this problem than traditional region selection and matting. However, the technique suffers a number of limitations, such as reduced performance in the presence of texture contrast, and the inability to handle fragmented appearances. We significantly improve the performance of edge‐aware interpolation for this problem by adding a boosting‐based classification step that learns to discriminate between the appearance of scribbled pixels. We show that this novel data term in combination with an existing edge‐aware optimization technique achieves substantially better results for the local image and video adjustment problem than edge‐aware interpolation techniques without classification, or related methods such as matting techniques or graph cut segmentation.     

Gesture-based interaction via finger tracking for mobile augmented reality

The goal of this research is to explore new interaction metaphors for augmented reality on mobile phones, i.e. applications where users look at the live image of the device’s video camera and 3D virtual objects enrich the scene that they see. Common interaction concepts for such applications are often limited to pure 2D pointing and clicking on the device’s touch screen. Such an interaction with virtual objects is not only restrictive but also difficult, for example, due to the small form factor. In this article, we investigate the potential of finger tracking for gesture-based interaction. We present two experiments evaluating canonical operations such as translation, rotation, and scaling of virtual objects with respect to performance (time and accuracy) and engagement (subjective user feedback). Our results indicate a high entertainment value, but low accuracy if objects are manipulated in midair, suggesting great possibilities for leisure applications but limited usage for serious tasks.     

In‐field evaluation of the modulation transfer function and the signal‐to‐noise ratio of electronic‐display devices

Abstract— This paper describes a charged‐coupled device (CCD) camera, which was developed for in‐field evaluation of the image quality of electronic‐display devices [such as cathode‐ray tubes (CRTs) and liquid‐crystal displays (LCDs)] used for medical applications. Contrary to traditional cameras for display‐image‐quality evaluation, this CCD camera does not require a sophisticated x‐y‐z translation stage for mounting and adjustment. Instead, it is handheld and pressed by gentle pressure against the display screen. It is controlled by a software package which was originally developed for display calibration according to the DICOM 14 gray‐scale standard display function (GSDF). This software package controls the camera gain when measurements are made at different display luminance, display test patterns, performs image analysis and displays the results of the measurements and calculations. The work concentrated on the measurement of modulation transfer function (MTF) and of signal‐to‐noise ratio (SNR) per display pixel. The MTF is derived from the Fourier transform of the line spread function (LSF). The single‐display‐pixel SNR is derived from the integration of the noise power spectrum (NPS) of a camera image taken of a display with a uniform luminance. It is demonstrated that the device can produce repeatable results in terms of MTF and SNR. MTFs were measured on three monochrome CRTs and five monochrome LCDs in order to study repeatability and similar quantities. The MTF was measured on a 5‐Mpixel LCD yielding values that lie within 3.5% of the average MTF at the Nyquist frequency and 4.0% of the maximum total sharpness (∫ MTF² df). The MTF was also measured on a 9‐Mpixel LCD, yielding values that lie within 9.0% of the average MTF at the Nyquist frequency and 8.0% of the maximum total sharpness. The SNR was measured eight times on a 3‐Mpixel monochrome LCD at nine digital driving levels (DDLs). At a DDL of 185, the mean SNR was 15.694 and the standard deviation (Stdv) was 0.587. At a DDL of 65, the mean SNR was 5.675 and Stdv was 0.120.     

Design conditions for attractive reality in mobile‐type 3‐D display

Abstract— An autostereoscopic 3‐D display suitable for the mobile environment is prototyped and evaluated. First, the required conditions for a 3‐D display in a mobile environment are considered, and the three major requirements are clarified: small size, viewing‐position flexibility, and application support. An application of a mobile‐type 3‐D display should be different from that of a large‐sized 3‐D display because a mobile‐type 3‐D display cannot realize the feeling of immersion while large‐sized 3‐D displays can realize it easily. From this assumption, it is considered that it is important to realize the feeling to handle a 3‐D image. Three types of 3‐D displays are developed to satisfy these requirements. They are subjectively evaluated to confirm their attractiveness. Results of the tests show that intuitive interaction can increase the reality of the 3‐D image in the sense of unity and also can improve the solidity and depth impression of the 3‐D image.     

Making in‐Front‐of Cars Transparent: Sharing First‐Person‐Views via Dashcam

Visual obstruction caused by a preceding vehicle is one of the key factors threatening driving safety. One possible solution is to share the first‐person‐view of the preceding vehicle to unveil the blocked field‐of‐view of the following vehicle. However, the geometric inconsistency caused by the camera‐eye discrepancy renders view sharing between different cars a very challenging task. In this paper, we present a first‐person‐perspective image rendering algorithm to solve this problem. Firstly, we contour unobstructed view as the transferred region, then by iteratively estimating local homography transformations and performing perspective‐adaptive warping using the estimated transformations, we are able to locally adjust the shape of the unobstructed view so that its perspective and boundary could be matched to that of the occluded region. Thus, the composited view is seamless in both the perceived perspective and photometric appearance, creating an impression as if the preceding vehicle is transparent. Our system improves the driver's visibility and thus relieves the burden on the driver, which in turn increases comfort. We demonstrate the usability and stability of our system by performing its evaluation with several challenging data sets collected from real‐world driving scenarios.     

Articulated‐Motion‐Aware Sparse Localized Decomposition

Compactly representing time‐varying geometries is an important issue in dynamic geometry processing. This paper proposes a framework of sparse localized decomposition for given animated meshes by analyzing the variation of edge lengths and dihedral angles (LAs) of the meshes. It first computes the length and dihedral angle of each edge for poses and then evaluates the difference (residuals) between the LAs of an arbitrary pose and their counterparts in a reference one. Performing sparse localized decomposition on the residuals yields a set of components which can perfectly capture local motion of articulations. It supports intuitive articulation motion editing through manipulating the blending coefficients of these components. To robustly reconstruct poses from altered LAs, we devise a connection‐map‐based algorithm which consists of two steps of linear optimization. A variety of experiments show that our decomposition is truly localized with respect to rotational motions and outperforms state‐of‐the‐art approaches in precisely capturing local articulated motion.     

Crowd sculpting: A space‐time sculpting method for populating virtual environments

We introduce “Crowd Sculpting”: a method to interactively design populated environments by using intuitive deformation gestures to drive both the spatial coverage and the temporal sequencing of a crowd motion. Our approach assembles large environments from sets of spatial elements which contain inter‐connectible, periodic crowd animations. Such a “Crowd Patches” approach allows us to avoid expensive and difficult‐to‐control simulations. It also overcomes the limitations of motion editing, that would result into animations delimited in space and time. Our novel methods allows the user to control the crowd patches layout in ways inspired by elastic shape sculpting: the user creates and tunes the desired populated environment through stretching, bending, cutting and merging gestures, applied either in space or time. Our examples demonstrate that our method allows the space‐time editing of very large populations and results into endless animation, while offering real‐time, intuitive control and maintaining animation quality.     

Dual‐lenticular‐lens‐based 2‐D/3‐D convertible autostereoscopic display

Abstract— A 2‐D/3‐D convertible display using two lenticular lenses has been developed. It shows 2‐D pictures in full resolution and 3‐D autostereoscopic pictures in half resolution by moving one lens relative to the other. The lens assembly consists of thin metal frames, two lenticular lenses, and two shape‐memory‐alloy (SMA) wires used as actuators. While this assembly is applicable to flat‐panel displays of any kind, its simple structure and low power consumption make it best suited to mobile terminals, such as PDAs and mobile phones. Here, we describe its structure and present evaluation results.     

一种快速的QR码图像去模糊方法

具有摄像功能的手机已经成为一种便捷的二维条码阅读器,但手机因为本身的限制造成拍摄图像模糊,影响了后续的条码识别。本文提出了一种适合在手机平台上的图像盲恢复方法,以QR码识别为应用目的,利用QR码图像的灰度分布特点,通过模糊图像的边缘响应计算得到PSF函数,再进行图像恢复。实验证明,本文提出的方法具有去模糊效果好、计算简单的特点,适合手机等嵌入式平台的实时应用。     

Generic interactive pixel‐level image editing

Several image editing methods have been proposed in the past decades, achieving brilliant results. The most sophisticated of them, however, require additional information per‐pixel. For instance, dehazing requires a specific transmittance value per pixel, or depth of field blurring requires depth or disparity values per pixel. This additional per‐pixel value is obtained either through elaborated heuristics or through additional control over the capture hardware, which is very often tailored for the specific editing application. In contrast, however, we propose a generic editing paradigm that can become the base of several different applications. This paradigm generates both the needed per‐pixel values and the resulting edit at interactive rates, with minimal user input that can be iteratively refined. Our key insight for getting per‐pixel values at such speed is to cluster them into superpixels, but, instead of a constant value per superpixel (which yields accuracy problems), we have a mathematical expression for pixel values at each superpixel: in our case, an order two multinomial per superpixel. This leads to a linear least‐squares system, effectively enabling specific per‐pixel values at fast speeds. We illustrate this approach in three applications: depth of field blurring (from depth values), dehazing (from transmittance values) and tone mapping (from brightness and contrast local values), and our approach proves both favorably interactive and accurate in all three. Our technique is also evaluated with a common dataset and compared favorably.