A pictorial index mechanism for model-based matching

We are currently developing unified query processing strategies for image databases. To perform this task, model-based representations of images by content are being used, as well as a hierarchical generalization of a relatively new object-recognition technique called data-driven indexed hypotheses. As the name implies, it is index-based, from which its efficiency derives. Earlier approaches to data-driven model-based object recognition techniques were not capable of handling complex image data containing overlapping, partially visible, and touching objects due to the limitations of the features used for building models. Recently, a few data-driven techniques capable of handling complex image data have been proposed. In these techniques, as in traditional databases, iconic index structures are employed to store the image and shape representation in such a way that searching for a given shape or image feature can be conducted efficiently. Some of these techniques handle the insertion and deletion of shapes and/or image representations very efficiently and with very little influence on the overall system performance. However, the main disadvantage of all previous data-driven implementations is that they are main memory based. In the present paper, we describe a secondary memory implementation of data-driven indexed hypotheses along with some performance studies we have conducted.     

A Logical Approach to Quality of Service Specification in Video Databases

Quality of Service (QoS) is defined as a set of perceivable attributes expressed in a user-friendly language with parameters that may be objective or subjective. Objective parameters are those related to a particular service and are measurable and verifiable. Subjective parameters are those based on the opinions of the end-users. We believe that quality of service should become an integral part of multimedia database systems and users should be able to query by requiring a quality of service from the system. The specification and enforcement of QoS presents an interesting challenge in multimedia systems development. A deal of effort has been done on QoS specification and control at the system and the network levels, but less work has been done at the application/user level. In this paper, we propose a language, in the style of constraint database languages, for formal specification of QoS constraints. The satisfaction by the system of the user quality requirements can be viewed as a constraint satisfaction problem, and the negotiation can be viewed as constraint optimization. We believe this paper represents a first step towards the development of a database framework for quality of service management in video databases. The contribution of this paper lies in providing a logical framework for specifying and enforcing quality of service in video databases. To our knowledge, this work is the first from a database perspective on quality of service management.     

A data model and a query languagefor multimedia documents databases

A multimedia application involves information that may be in a form of video, images, audio, text and graphics, need to be stored, retrieved and manipulated in large databases. In this paper, we propose an object-oriented database schema that supports multimedia documents and their temporal, spatial and logical structures. We present a document example and show how the schema can adress all the structures described. We also present a multimedia query specification language that can be used to describe a multimedia content portion to be retrieved from the database. The language provides means by which the user can specify the information on the media as well as the temoral and spatial relationships among these media.     

Integrity constraint integration in heterogeneous databases: An enhanced methodology for schema integration

In today's technologically diverse corporate environment, it is common to find several different databases being used to accomplish the organization's operational data management functions. Providing interoperability among these databases is important to the successful operation of the organization. One approach to providing interoperability among heterogeneous database systems, is to define one or more schemas which represent a coherent view of the underlying databases. In the past, most approaches have used schematic knowledge about the underlying databases to generate integrated representations of the databases. In this paper we present a seven step methodology for utilizing integrity constraint knowledge from heterogeneous databases. Specifically, we describe how we can generate a set of integrity constraints applicable at the integrated level from constraints specified on local databases. We introduce the concept of constraint-based relationships between objects in heterogeneous databases and describe the role that these relationships play in integrity constraint integration. Finally, we describe how the integrated set of constraints generated using our methodology can be used to facilitate semantic query processing in a heterogeneous database environment.     

基于complex moment的圆度测量和图像检索

提出了一种complex moment的圆度测量方法。作为图像的形状特征,该方法易于计算,并可以减少形状提取时的大量计算,对噪声和狭窄区域有很好的鲁棒性。使用图像结构相似度度量方法 FSIM,结合complexmoment的圆度特征、图像的相位特征、相位不变性特征,既弥补相位不变性(PC)对图像对比度不变的影响,又能充分利用圆度测量计算简单,旋转、放缩不变性,计算量小的优点。为了验证该算法效率,采用图像库Shape7,彩色物体库COL、VIEW、ILL进行图像检索实验。实验表明,该方法对图像的形状和明度变化有很好的检索效果。     

An Optimization Approach to Improving Collections of Shape Maps

Finding an informative, structure‐preserving map between two shapes has been a long‐standing problem in geometry processing, involving a variety of solution approaches and applications. However, in many cases, we are given not only two related shapes, but a collection of them, and considering each pairwise map independently does not take full advantage of all existing information. For example, a notorious problem with computing shape maps is the ambiguity introduced by the symmetry problem — for two similar shapes which have reflectional symmetry there exist two maps which are equally favorable, and no intrinsic mapping algorithm can distinguish between them based on these two shapes alone. Another prominent issue with shape mapping algorithms is their relative sensitivity to how “similar” two shapes are — good maps are much easier to obtain when shapes are very similar. Given the context of additional shape maps connecting our collection, we propose to add the constraint of global map consistency, requiring that any composition of maps between two shapes should be independent of the path chosen in the network. This requirement can help us choose among the equally good symmetric alternatives, or help us replace a “bad” pairwise map with the composition of a few “good” maps between shapes that in some sense interpolate the original ones. We show how, given a collection of pairwise shape maps, to define an optimization problem whose output is a set of alternative maps, compositions of those given, which are consistent, and individually at times much better than the original. Our method is general, and can work on any collection of shapes, as long as a seed set of good pairwise maps is provided. We demonstrate the effectiveness of our method for improving maps generated by state‐of‐the‐art mapping methods on various shape databases.     

Extracting semantic knowledge from web context for multimedia IR: a taxonomy,survey and challenges

Since its invention, the Web has evolved into the largest multimedia repository that has ever existed. This evolution is a direct result of the explosion of user-generated content, explained by the wide adoption of social network platforms. The vast amount of multimedia content requires effective management and retrieval techniques. Nevertheless, Web multimedia retrieval is a complex task because users commonly express their information needs in semantic terms, but expect multimedia content in return. This dissociation between semantics and content of multimedia is known as the semantic gap. To solve this, researchers are looking beyond content-based or text-based approaches, integrating novel data sources. New data sources can consist of any type of data extracted from the context of multimedia documents, defined as the data that is not part of the raw content of a multimedia file. The Web is an extraordinary source of context data, which can be found in explicit or implicit relation to multimedia objects, such as surrounding text, tags, hyperlinks, and even in relevance-feedback. Recent advances in Web multimedia retrieval have shown that context data has great potential to bridge the semantic gap. In this article, we present the first comprehensive survey of context-based approaches for multimedia information retrieval on the Web. We introduce a data-driven taxonomy, which we then use in our literature review of the most emblematic and important approaches that use context-based data. In addition, we identify important challenges and opportunities, which had not been previously addressed in this area.

     

Computer vision based on a hypothesization and verification scheme by parallel relaxation

Shape recovery from a monocular image is addressed. It is often said that the information conveyed by an image is insufficient to reconstruct 3D shapes of objects in the image. This implies that shape recovery from an image necessitates the use of additional plausible constraints on typical structures and features of the objects in an ordinary scene. We propose a hypothesization and verification method for 3D shape recovery based on geometrical constraints peculiar to man-made objects. The objective is to increase the robustness of computer vision systems. One difficulty with this method lies in the mutual dependency between proper assignment of constraints to the regions in a given image and recovery of a consistent 3D shape. A concurrent mechanism has been implemented which is based on energy minimization using a parallel network for relaxation. This mechanism is capable of maintaining consistency between constraint assignment and shape recovery.     

Sparse Representation Shape Models

It is well-known that, during shape extraction, enrolling an appropriate shape constraint model could effectively improve locating accuracy. In this paper, a novel deformable shape model, Sparse Representation Shape Models (SRSM), is introduced. Rather than following commonly utilized statistical shape constraints, our model constrains shape appearance based on a morphological structure, the convex hull of aligned training samples, i.e., only shapes that could be linearly represented by aligned training samples with the sum of coefficients equal to one, are defined as qualified. This restriction strictly controls shape deformation modes to reduce extraction errors and prevent extremely poor outputs. This model is realized based on sparse representation, which ensures during shape regularization the maximum valuable shape information could be reserved. Besides, SRSM is interpretable and hence helpful to further understanding applications, such as face pose recognition. The effectiveness of SRSM is verified on two publicly available face image datasets, the FGNET and the FERET.     

An extended algebra for constraint databases

Constraint relational databases use constraints to both model and query data. A constraint relation contains a finite set of generalized tuples. Each generalized tuple is represented by a conjunction of constraints on a given logical theory and, depending on the logical theory and the specific conjunction of constraints, it may possibly represent an infinite set of relational tuples. For their characteristics, constraint databases are well suited to model multidimensional and structured data, like spatial and temporal data. The definition of an algebra for constraint relational databases is important in order to make constraint databases a practical technology. We extend the previously defined constraint algebra (called generalized relational algebra). First, we show that the relational model is not the only possible semantic reference model for constraint relational databases and we show how constraint relations can be interpreted under the nested relational model. Then, we introduce two distinct classes of constraint algebras, one based on the relational algebra, and one based on the nested relational algebra, and we present an algebra of the latter type. The algebra is proved equivalent to the generalized relational algebra when input relations are modified by introducing generalized tuple identifiers. However, from a user point of view, it is more suitable. Thus, the difference existing between such algebras is similar to the difference existing between the relational algebra and the nested relational algebra, dealing with only one level of nesting. We also show how external functions can be added to the proposed algebra     

Detecting the most unusual part of two- and three-dimensional digital images

The purpose of this paper is to introduce an algorithm that can detect the most unusual part of a digital image in probabilistic setting. The most unusual part of a given shape is defined as a part of the image that has the maximal distance to all non-intersecting shapes with the same form. The method is tested on two- and three-dimensional images and has shown very good results without any predefined model. A version of the method independent of the contrast of the image is considered and is found to be useful for finding the most unusual part (and the most similar part) of the image conditioned on given image.The results can be used to scan large image databases, as for example medical databases.     

A content based image retrieval system for a biological specimen collection

Digital photography and decreasing cost of storing data in digital form has led to an explosion of large digital image repositories. Since the number of images in image databases can be large (millions in some cases) it is important to develop automated tools to search them. In this paper, we present a content based image retrieval system for a database of parasite specimen images. Unlike most content based image retrieval systems, where the database consists of objects that vary widely in shape and size, the objects in our database are fairly uniform. These objects are characterized by flexible body shapes, but with fairly rigid ends. We define such shapes to be FleBoRE (Flexible Body Rigid Extremities) objects, and present a shape model for this class of objects. We have defined similarity functions to compute the degree of likeness between two FleBoRE objects and developed automated methods to extract them from specimen images. The system has been tested with a collection of parasite images from the Harold W. Manter Laboratory for Parasitology. Empirical and expert-based evaluations show that query by shape approach is effective in retrieving specimens of the same class.     

Structural feature indexing for retrieval of partially visible shapes

Efficient and robust information retrieval from large image databases is an essential functionality for the reuse, manipulation, and editing of multimedia documents. Structural feature indexing is a potential approach to efficient shape retrieval from large image databases, but the indexing is sensitive to noise, scales of observation, and local shape deformations. It has now been confirmed that efficiency of classification and robustness against noise and local shape transformations can be improved by the feature indexing approach incorporating shape feature generation techniques (Nishida, Comput. Vision Image Understanding 73 (1) (1999) 121-136). In this paper, based on this approach, an efficient, robust method is presented for retrieval of model shapes that have parts similar to the query shape presented to the image database. The effectiveness is confirmed by experimental trials with a large database of boundary contours obtained from real images, and is validated by systematically designed experiments with a large number of synthetic data.     

On Similarity Measures for Multimedia Database Applications

A multimedia database query consists of a set of fuzzy and boolean (or crisp) predicates, constants, variables, and conjunction, disjunction, and negation operators. The fuzzy predicates are evaluated based on different media criteria, such as color, shape, layout, keyword. Since media-based evaluation yields similarity values, results to such a query is defined as an ordered set. Since many multimedia applications require partial matches, query results also include tuples which do not satisfy all predicates. Hence, any fuzzy semantics which extends the boolean semantics of conjunction in a straight forward manner may not be desirable for multimedia databases. In this paper, we focus on the problem of ‘given a multimedia query which consists of multiple fuzzy and crisp predicates, how to provide the user with a meaningful overall ranking.’ More specifically, we study the problem of merging similarity values in queries with multiple fuzzy predicates. We describe the essential multimedia retrieval semantics, compare these with the known approaches, and propose a semantics which captures the retrieval requirements in multimedia databases. Received 13 August 1999 / Revised 13 May 2000 / Accepted in revised form 26 July 2000     

基于单张照片的三维人脸重建优化算法

传统人脸三维重建算法难以确定人脸形状,并且计算复杂。针对此问题,提出一种以水平集方法获取人脸轮廓并结合明暗恢复形状(SFS)算法重建三维模型的方法,该方法仅需单张正面人脸照片。首先采用主动形状模型确定人脸轮廓,将其作为水平集的初始演化曲线,分割出完整的人脸形状;然后对人脸区域进行灰度变换,求出灰度图像;最后通过SFS算法重建已知光照条件的人脸图像的三维模型,将该模型作为参考与灰度图像匹配,进而确定其光照条件和三维模型。实验结果表明,与基于网格模型的算法相比,该方法可快速地重建具有完整形状的人脸模型。