From PhotoWork to PhotoUse: exploring personal digital photo activities

People accumulate large collections of digital photos, which they use for individual, social, and utilitarian purposes. In order to provide suitable technologies for enjoying our expanding photo collections, it is essential to understand how and to what purpose these collections are used. Contextual interviews with 12 participants in their homes explored the use of digital photos, incorporating new photo activities that are offered by new technologies. Based on the qualitative analysis of the collected data, we give an overview of current photo activities, which we term PhotoUse. We introduce a model of PhotoUse, which emphasises the purpose of photo activities rather than the tools to support them. We argue for the use of our model to design tools to support the user’s individual and social goals pertaining to PhotoUse.     

Fog effect for photography using stereo vision

Fog is an important factor in photography with a special aesthetic, emotional, or compositional meaning. We present a fog-simulation method for photo editing using binocular stereo vision. Given a stereo pair, we estimate the depth information by stereo matching followed by a process to refine depth results for the given photo editing purpose. Then, depth-aware fog effects can be applied on the base image, with optional interaction for control purposes. Besides homogeneous fog, we provide three tools to control the density of the fog media. Thus, various kinds of heterogeneous atmospheric effects can also been simulated. Experiments show that the proposed method can achieve more natural-looking results than manually drawn fog, our results are very close to the appearance of fog in the real world.     

Geotagging in multimedia and computer vision—a survey

Geo-tagging is a fast-emerging trend in digital photography and community photo sharing. The presence of geographically relevant metadata with images and videos has opened up interesting research avenues within the multimedia and computer vision domains. In this paper, we survey geo-tagging related research within the context of multimedia and along three dimensions: (1) Modalities in which geographical information can be extracted, (2) Applications that can benefit from the use of geographical information, and (3) The interplay between modalities and applications. Our survey will introduce research problems and discuss significant approaches. We will discuss the nature of different modalities and lay out factors that are expected to govern the choices with respect to multimedia and vision applications. Finally, we discuss future research directions in this field.     

ON MACHINE VISION AND CAST

Building machines that “see” is a complex systems engineering task. While, at present, universal vision machines (with performance comparable to the human visual system) appear to be impractical, special purpose machines have been devised successfully to operate in restricted application domains. Designing such a dedicated system usually requires detailed knowledge of the application domain, experience with available processing techniques, and a large experimentation effort. Despite the peculiarities of different vision domains, many of them share similar processing techniques. We propose to develop computerized tools to support the development of application-specific vision systems in order to make this process less time-consuming and costly. We discuss some of the fundamental problems of machine vision and examine why current computer-aided systems theory (CAST) technology is not sufficient for this task. On the other hand it seems likely that developments in the Held of machine vision could have a significant impact upon other CAST applications of similar complexity.     

State of the Art on Monocular 3D Face Reconstruction,Tracking, and Applications

The computer graphics and vision communities have dedicated long standing efforts in building computerized tools for reconstructing, tracking, and analyzing human faces based on visual input. Over the past years rapid progress has been made, which led to novel and powerful algorithms that obtain impressive results even in the very challenging case of reconstruction from a single RGB or RGB‐D camera. The range of applications is vast and steadily growing as these technologies are further improving in speed, accuracy, and ease of use. Motivated by this rapid progress, this state‐of‐the‐art report summarizes recent trends in monocular facial performance capture and discusses its applications, which range from performance‐based animation to real‐time facial reenactment. We focus our discussion on methods where the central task is to recover and track a three dimensional model of the human face using optimization‐based reconstruction algorithms. We provide an in‐depth overview of the underlying concepts of real‐world image formation, and we discuss common assumptions and simplifications that make these algorithms practical. In addition, we extensively cover the priors that are used to better constrain the under‐constrained monocular reconstruction problem, and discuss the optimization techniques that are employed to recover dense, photo‐geometric 3D face models from monocular 2D data. Finally, we discuss a variety of use cases for the reviewed algorithms in the context of motion capture, facial animation, as well as image and video editing.     

Comprehensive Cardiovascular Image Analysis Using MR and CT at Siemens Corporate Research

At Siemens Corporate Research we have created a set of tools for the analysis of MR and CT cardiovascular images in the applications Argus, Vessel View, and Proteus. Argus is designed to assess cardiovascular function by reporting measures of morphology and tissue health using a 2-D approach. Vessel View, a 3-D application, is capable of quantifying vascular integrity and provides tools for segmenting vessels. Lastly, Proteus has functionality for registering 3-D cardiac data sets (e.g., MR and CT). Taken together, these applications allow for a comprehensive analysis of MR and CT cardiovascular studies. Throughout this paper we will illustrate the capabilities of our tools via their application to an actual clinical case. Our contribution lies in combining several computer vision technologies and applying them to practical, real world problems.     

The Minimal Structure and Motion Problems with Missing Data for 1D Retina Vision

In this paper we investigate the structure and motion problem for calibrated one-dimensional projections of a two-dimensional environment. The theory of one-dimensional cameras are useful in several areas, e.g. within robotics, autonomous guided vehicles, projection of lines in ordinary vision and vision of vehicles undergoing so called planar motion. In a previous paper the structure and motion problem for all cases with non-missing data was classified and solved. Our aim is here to classify all structure and motion problems, even those with missing data, and to solve them. In the classification we introduce the notion of a prime problem. A prime problem is a minimal problem that does not contain a minimal problem as a sub-problem. We further show that there are infinitely many such prime problems. We give solutions to four prime problems, and using the duality of Carlsson these can be extended to solutions of seven prime problems. Finally we give some experimental results based on synthetic data.     

General-purpose vision chip architecture for real-time machine vision

To solve the I/O bottleneck problem in existing vision systems and to realize versatile processing adaptive to various and changing environments, we propose a new vision chip architecture for applications such as robot vision. The chip has general-purpose processing elements (PEs) with each PE being directly connected to a photo detector (PD) and can implement various visual processing algorithms. We developed and simulated some sample programs for the chip and proved that they can be processed within 1 ms/frame, a rate that is high enough for high-speed visual feedback for robot control. Aiming to complete the chip, we are now developing test chips based on the architecture. The latest design has 8 x 8 PEs and PDs in an area 3.3 mm x 3.0 mm using a 0.8 μm CMOS process.     

基于谱图理论的特征匹配方法研究和改进

图像匹配问题是计算机视觉领域的一个基本问题,广泛地应用于很多领域,如:模式识别,自动导航,医学诊断,计算机视觉,图像三维重构等领域。将所研究的问题转化为数学问题,再利用数学工具解决这一问题,成为当今研究的一种重要手段。在这里,将图像匹配中的图像转化为数学-图论中的图,利用谱图理论解决图像匹配问题,从而形成了一类比较流行而新颖的方法,针对这一方法进行了较为系统的探究并做出了改进。     

Multi-modal and multi-scale photo collection summarization

With the proliferation of digital cameras and mobile devices, people are taking much more photos than ever before. However, these photos can be redundant in content and varied in quality. Therefore there is a growing need for tools to manage the photo collections. One efficient photo management way is photo collection summarization which segments the photo collection into different events and then selects a set of representative and high quality photos (key photos) from those events. However, existing photo collection summarization methods mainly consider the low-level features for photo representation only, such as color, texture, etc, while ignore many other useful features, for example high-level semantic feature and location. Moreover, they often return fixed summarization results which provide little flexibility. In this paper, we propose a multi-modal and multi-scale photo collection summarization method by leveraging multi-modal features, including time, location and high-level semantic features. We first use Gaussian mixture model to segment photo collection into events. With images represented by those multi-modal features, our event segmentation algorithm can generate better performance since the multi-modal features can better capture the inhomogeneous structure of events. Next we propose a novel key photo ranking and selection algorithm to select representative and high quality photos from the events for summarization. Our key photo ranking algorithm takes the importance of both events and photos into consideration. Furthermore, our photo summarization method allows users to control the scale of event segmentation and number of key photos selected. We evaluate our method by extensive experiments on four photo collections. Experimental results demonstrate that our method achieves better performance than previous photo collection summarization methods.     

Control theory for stochastic distributed parameter systems,an engineering perspective

The main purpose of this paper is to survey some recent progresses on control theory for stochastic distributed parameter systems, i.e., systems governed by stochastic differential equations in infinite dimensions, typically by stochastic partial differential equations. We will explain the new phenomenon and difficulties in the study of controllability and optimal control problems for one dimensional stochastic parabolic equations and stochastic hyperbolic equations. In particular, we shall see that both the formulation of corresponding stochastic control problems and the tools to solve them may differ considerably from their deterministic/finite-dimensional counterparts. More importantly, one has to develop new tools, say, the stochastic transposition method introduced in our previous works, to solve some problems in this field.     

Concurrent photo sequence organization

Personal photo album organization is a highly demanding domain where advanced tools are required to manage large photo collections. In contrast to many previous works, that try to solve the problem of organizing a single user photo sequence, we present a new technique to account for the concurrent photo sequence organization problem, that is the problem of organizing multiple photo sequences taken during the same event. Given a set of sequences acquired at the same place during the same temporal window by several users using different cameras, our framework is intended to capture the evolution of the event and groups photos based on temporal proximity and visual content. The method automatically organizes the reference sequence in a tree capturing the event structure. Such a structure is then used to align the remaining photo sequences to the reference one. We tested our approach on the publicly available Gallagher dataset and on a new dataset we collected; this new dataset is composed of four photo sequences taken by four users at a public event. Results demonstrate the effectiveness of our method.     

Reality Television as a Model for Online Behavior: Blogging, Photo, and Video Sharing

This research explores traditional mass media as an antecedent to nondirected self-disclosure online. New Internet-based tools allow users to communicate with global audiences, and to make intimate personal information available to this audience. At the same time, a culture that rewards the public performance of private thoughts and emotions is increasingly evident in "reality" television (RTV) programming. This study used survey data to examine RTV consumption, authoritarianism, and users' offline social context as potential antecedents for nondirected self-disclosure via blogs, online photo sharing, and online video sharing. RTV consumption correlated with blogging and video sharing, but not photo sharing. Social support network size was a significant correlate of photo sharing, indicating that photo sharing may be a more relational activity.     

Acyclic belief propagation for stereo matching

Most of the works for solving early vision problems, such as stereo, rely on inference techniques and cyclic graphical models. Although Loopy Belief Propagation is one of the most interesting tools as it applies to that research field, loops are still a major challenge to face: the main problem arising with them is the introduction of approximation issues which could affect the accuracy of the final results. Therefore, breaking loops could be the winning strategy to overcome these problems. In this paper, we describe a way to consider an MRF grid as composed by a suitable set of singly connected graphs. Then, an acyclic propagation scheme, based on that set, is given accordingly. Experiments performed on different stereo pairs will prove that our approach is up to the current state-of-art and it spreads several new research issues for the future.     

Intelligent Visual Media Processing: When Graphics Meets Vision

The computer graphics and computer vision communities have been working closely together in recent years, and a variety of algorithms and applications have been developed to analyze and manipulate the visual media around us. There are three major driving forces behind this phenomenon: 1) the availability of big data from the Internet has created a demand for dealing with the ever-increasing, vast amount of resources; 2) powerful processing tools, such as deep neural networks, provide effective ways for learning how to deal with heterogeneous visual data; 3) new data capture devices, such as the Kinect, the bridge between algorithms for 2D image understanding and 3D model analysis. These driving forces have emerged only recently, and we believe that the computer graphics and computer vision communities are still in the beginning of their honeymoon phase. In this work we survey recent research on how computer vision techniques benefit computer graphics techniques and vice versa, and cover research on analysis, manipulation, synthesis, and interaction. We also discuss existing problems and suggest possible further research directions.     

Authenticating corrupted photo images based on noise parameter estimation

Photo image authentication is an interesting and demanding field in the computer vision and image processing community. This research is motivated by its wide range of applications, which include smart card authentication systems, biometric passport systems, etc. In this paper, we propose a method of authenticating corrupted photo images based on noise parameter estimation. The proposed method first generates corrupted images by adjusting the noise parameters in the initial training phase. This set of corrupted images and the noise parameters can be represented by a linear combination of the prototypes of the corrupted images and the noise parameters. In the testing phase, the noise parameters of the corrupted photo image can be estimated with a corrupted image and an original image. Finally, we can make a synthesized photo image from the original photo image using the estimated noise parameters and verify it with the corrupted photo image. The experimental results show that the proposed method can estimate the noise parameters accurately and improve the performance of photo image authentication.     

Developing a generic augmented-reality interface

We seek to design an AR interface that provides users with interactivity so rich it would merge the physical space in which we live and work with the virtual space in which we store and interact with digital information. In this single augmented space, computer-generated entities would become first-class citizens of the physical environment. We would use these entities just as we use physical objects, selecting and manipulating them with our hands instead of with a special-purpose device such as a mouse or joystick. Interaction would then be intuitive and seamless because we would use the same tools to work with digital and real objects. Tiles is an AR interface that moves one step closer to this vision. It allows effective spatial composition, layout, and arrangement of digital objects in the physical environment. The system facilitates seamless two-handed, three-dimensional interaction with both virtual and physical objects, without requiring any special-purpose input devices     

Modeling the World from Internet Photo Collections

There are billions of photographs on the Internet, comprising the largest and most diverse photo collection ever assembled. How can computer vision researchers exploit this imagery? This paper explores this question from the standpoint of 3D scene modeling and visualization. We present structure-from-motion and image-based rendering algorithms that operate on hundreds of images downloaded as a result of keyword-based image search queries like “Notre Dame” or “Trevi Fountain.” This approach, which we call Photo Tourism, has enabled reconstructions of numerous well-known world sites. This paper presents these algorithms and results as a first step towards 3D modeling of the world’s well-photographed sites, cities, and landscapes from Internet imagery, and discusses key open problems and challenges for the research community.     

A combinatory framework of Web 2.0 mashup tools,OWL-S and UDDI

The increasing number of Web 2.0 applications, such as wikis or social networking sites, indicates a movement to large-scale collaborative and social Web activities. Users can share information, add value to Web applications by using them or aggregate data from different sources creating Web applications (mashups) using specialized tools (mashup tools). However, Web 2.0 is not a new technology, but it rather embraces a new philosophy, treating the Internet as a platform. Several issues related to the Semantic Web vision, such as interoperability or machine understandable data semantics, are not tackled by Web 2.0. In this paper, we present our effort to combine semantic Web services (SWS) discovery frameworks, UDDI repositories and existing mashup tools in order to enhance the procedure of developing mashups with semantic mashup discovery capabilities. Towards this end, we introduce a social-oriented extension of OWL-S advertisements, their mapping algorithm on UDDI repositories and a semantic mashup discovery algorithm. Finally, we elaborate on the way our framework has been realized using the Yahoo Pipes mashup tool.     

Image denoising by anisotropic diffusion with inter-scale information fusion

Anisotropic partial differential equations (PDEs) based schemes for denoising digital images are fast becoming an indispensable tool in computer vision problems. In this paper we propose to denoise noisy images via such multiscale anisotropic diffusion. In general, digital images contain objects of multiple scales and denoising them without destroying edges is one of the main objective in early computer vision problems. Unlike the previous approaches, which discard the multiple scale based images produced by anisotropic PDE, we utilize information contained in them. By effectively combining the inter-scale details, the proposed scheme improves upon the noise removal and detail preservation properties over other schemes. Numerical results indicate that the scheme achieves good denoising with edge preservation on a variety of images.