Estimating Photometric Properties from Image Collections

We address the problem of jointly estimating the scene illumination, the radiometric camera calibration and the reflectance properties of an object using a set of images from a community photo collection. The highly ill-posed nature of this problem is circumvented by using appropriate representations of illumination, an empirical model for the nonlinear function that relates image irradiance with intensity values and additional assumptions on the surface reflectance properties. Using a 3D model recovered from an unstructured set of images, we estimate the coefficients that represent the illumination for each image using a frequency framework. For each image, we also compute the corresponding camera response function. Additionally, we calculate a simple model for the reflectance properties of the 3D model. A robust non-linear optimization is proposed exploiting the high sparsity present in the problem.     

Rephotography Using Image Collections

This paper proposes a novel system that “rephotographs” a historical photograph with a collection of images. Rather than finding the accurate viewpoint of the historical photo, users only need to take a number of photographs around the target scene. We adopt the structure from motion technique to estimate the spatial relationship among these photographs, and construct a set of 3D point cloud. Based on the user‐specified correspondences between the projected 3D point cloud and historical photograph, the camera parameters of the historical photograph are estimated. We then combine forward and backward warping images to render the result. Finally, inpainting and content‐preserving warping are used to refine it, and the photograph at the same viewpoint of the historical one is produced by this photo collection.     

AutoStyle: Automatic Style Transfer from Image Collections to Users' Images

Stylizing photos, to give them an antique or artistic look, has become popular in recent years. The available stylization filters, however, are usually created manually by artists, resulting in a narrow set of choices. Moreover, it can be difficult for the user to select a desired filter, since the filters’ names often do not convey their functions. We investigate an approach to photo filtering in which the user provides one or more keywords, and the desired style is defined by the set of images returned by searching the web for those keywords. Our method clusters the returned images, allows the user to select a cluster, then stylizes the user's photos by transferring vignetting, color, and local contrast from that cluster. This approach vastly expands the range of available styles, and gives each filter a meaningful name by default. We demonstrate that our method is able to robustly transfer a wide range of styles from image collections to users’ photos.     

Comparing Personal Image Collections with PICTuReVis

Digital image collections contain a wealth of information, which for instance can be used to trace illegal activities and investigate criminal networks. We present a method that enables analysts to reveal relations among people, based on the patterns in their collections. Similar temporal and spatial patterns can be found using a parameterized algorithm, visualization is used to choose the right parameters and to inspect the patterns found. The visualization shows relations between image properties: the person it belongs to, the concepts in the image, its time stamp and location. We demonstrate the method with image collections of 10, 000 people containing 460, 000 images in total.     

Improving Image Retrieval Effectiveness via Multiple Queries

Conventional approaches to image retrieval are based on the assumption that relevant images are physically near the query image in some feature space. This is the basis of the cluster hypothesis. However, semantically related images are often scattered across several visual clusters. Although traditional Content-based Image Retrieval (CBIR) technologies may utilize the information contained in multiple queries (gotten in one step or through a feedback process), this is often only a reformulation of the original query. As a result most of these strategies only get the images in some neighborhood of the original query as the retrieval result. This severely restricts the system performance. Relevance feedback techniques are generally used to mitigate this problem. In this paper, we present a novel approach to relevance feedback which can return semantically related images in different visual clusters by merging the result sets of multiple queries. We also provide experimental results to demonstrate the effectiveness of our approach.Xiangyu Jin received his B.S. and M.E. in Computer Science from the Nanjing University, China, in 1999 and 2002, respectively. He has a visiting student in Microsoft Research Asia (2001) and now is a Ph.D. candidate in the Department of Computer Science at the University of Virginia. His current research interest includes multimedia information retrieval and user interface study. He had the authored or co-authored about 20 publications in these areas.James French is currently a Research Associate Professor in the Department of Computer Science at the University of Virginia. He received a B.A. in Mathematics and M.S. and Ph.D. (1982) degrees in Computer Science, all at the University of Virginia. After several years in industry, he returned to the University of Virginia in 1987 as a Senior Scientist in the Institute for Parallel Computation and joined the Department of Computer Science in 1990. His current research interests include content-based retrieval and information retrieval in widely distributed information systems. He is the editor of five books, and the author or co-author of one book and over 75 papers and book chapters. Professor French is a member of the ACM, the IEEE Computer Society, ASIST, and Sigma Xi. At the time of this work he was on a leave of absence from the University of Virginia serving as a program director at the U.S. National Science Foundation.     

基于非监督纹理分割下的图像查询

随着图像数据库的广泛应用，基于图像纹理的查询已经变成了一项重要也是复杂的工作。为了更简便地查询图像，文中基于图像纹理的非监督分割，提出了新的图像分类方法。通过对每一纹理图像进行这种非监督纹理分割，提取纹理的特征向量，然后基于这些参数，运用假设检验进行纹理的分类和融合。通过对MT Vistex和Brodatz数据库中选取的纹理图像进行分类测试，证明该方法是十分有效的。     

基于非监督纹理分割下的图像查询

随着图像数据库的广泛应用,基于图像纹理的查询已经变成了一项重要也是复杂的工作。为了更简便地查询图像,文中基于图像纹理的非监督分割,提出了新的图像分类方法。通过对每一纹理图像进行这种非监督纹理分割,提取纹理的特征向量,然后基于这些参数,运用假设检验进行纹理的分类和融合。通过对MT Vistex和Brodatz数据库中选取的纹理图像进行分类测试,证明该方法是十分有效的。     

Guiding Image Manipulations using Shape‐appearance Subspaces from Co‐alignment of Image Collections

We propose a system to restrict the manipulation of shape and appearance in an image to a valid subspace which we learn from a collection of exemplar images. To this end, we automatically co‐align a collection of images and learn a subspace model of shape and appearance using principal components. As finding perfect image correspondences for general images is not feasible, we build an approximate partial alignment and improve bad alignments leveraging other, more successful alignments. Our system allows the user to change appearance and shape in real‐time and the result is “projected” onto the subspace of meaningful changes. The change in appearance and shape can either be locked or performed independently. Additional applications include suggestion of alternative shapes or appearance.     

An Image Database System with Support for Traditional Alphanumeric Queries and Content-Based Queries by Example

Large image databases are commonly employed in applications like criminal records, customs, plant root databases, and voters' registration databases. Efficient and convenient mechanisms for database organization and retrieval are essential. A quick and easy-to-use interface is needed which should also mesh naturally with the overall image management system. In this paper we describe the design and implementation of an integrated image database system. This system offers support for both alphanumeric query, based on alphanumeric data attached to the image file, and content-based query utilizing image examples. Content-based retrieval, specifically Query by Image Example, is made possible by the SHOSLIF approach. Alphanumeric query is implemented by a collection of parsing and query modules. All these are accessible from within a user-friendly GUI.     

图像检索中一种支持多例查询的新方法

基于内容的图像检索的关键问题之一是高层语义和低层图像特征之间的差异,相关反馈技术是缩短这个"语义鸿沟"的有效方法。本文提出了一种新的相关反馈算法,通过分析正例图像在特征空间中的散布来构造该类图像的投影空间,该空间对应于一个语义类在特征空间中分布密集的子空间,在投影空间中计算相似图像。同时根据每次反馈的信息不断修正投影空间来提高系统的检索性能。在Corel大图像库中的实验结果表明,该算法对多例图像查询有较好的检索效果。     

Fast Feature‐Oriented Visual Connection for Large Image Collections

Deriving the visual connectivity across large image collections is a computationally expensive task. Different from current image‐oriented match graph construction methods which build on pairwise image matching, we present a novel and scalable feature‐oriented image matching algorithm for large collections. Our method improves the match graph construction procedure in three ways. First, instead of building trees repeatedly, we put the feature points of the input image collection into a single kd‐tree and select the leaves as our anchor points. Then we construct an anchor graph from which each feature can intelligently find a small portion of related candidates to match. Finally, we design a new form of adjacency matrix for fast feature similarity measuring, and return all the matches in different photos across the whole dataset directly. Experiments show that our feature‐oriented correspondence algorithm can explore visual connectivity between images with significant improvement in speed.     

XML查询模式发掘

随着XML的广泛应用，如何高效地处理XML查询受到越来越多的关注．发掘频繁使用的查询模式，并缓存其查询结果是提高查询效率的有效手段之一．由于XML查询可以表示为树，因而可以使用频繁结构挖掘(Frequent Structure Mining)的技术来发掘频繁查询模式．提出一种高效的频繁查询模式发掘方法,同以前的类似工作相比,所做工作是严格地基于XML查询语义的，因而发掘结果曼为准确，也更易于使用．     

基于互联网图像集的室外场景建模技术综述

目前在互联网上有海量的室外场景照片,它们不仅内容覆盖面广,对同一场景有大量不同视点和光照条件下的采样,而且获取成本很低.如何利用这些照片中包含的丰富场景信息,快速、方便地构造各种逼真的虚拟场景,是互联网迅猛发展给虚拟现实、计算机图形学和计算机视觉带来的新的研究课题之一.文中分析、总结了近年来国内外互联网图像的真实场景几何建模、自然光照建模和材质反射属性建模的最新研究进展,并对其未来的发展趋势提出了一些看法.     

Restricted Information from Nonadaptive Queries to NP

We investigate classes of sets that can be decided by bounded truth-table reductions to an NP set in which evaluators donothave full access to the answers to the queries but get only restricted information such as the number of queries that are in the oracle set or even just this number modulom, for somem?2. We also investigate the case in which evaluators are nondeterministic. We show that when we vary the information that the evaluators get, this can change the resulting power of the evaluators. We locate all these classes within levels of the Boolean hierarchy which allows us to compare the complexity of such classes.     

Modeling the World from Internet Photo Collections

There are billions of photographs on the Internet, comprising the largest and most diverse photo collection ever assembled. How can computer vision researchers exploit this imagery? This paper explores this question from the standpoint of 3D scene modeling and visualization. We present structure-from-motion and image-based rendering algorithms that operate on hundreds of images downloaded as a result of keyword-based image search queries like “Notre Dame” or “Trevi Fountain.” This approach, which we call Photo Tourism, has enabled reconstructions of numerous well-known world sites. This paper presents these algorithms and results as a first step towards 3D modeling of the world’s well-photographed sites, cities, and landscapes from Internet imagery, and discusses key open problems and challenges for the research community.