Image database design based on 9D-SPA representation for spatial relations

Spatial relationships between objects are important features for designing a content-based image retrieval system. We propose a new scheme, called 9D-SPA representation, for encoding the spatial relations in an image. With this representation, important functions of intelligent image database systems such as visualization, browsing, spatial reasoning, iconic indexing, and similarity retrieval can be easily achieved. The capability of discriminating images based on 9D-SPA representation is much more powerful than any spatial representation method based on minimum bounding rectangles or centroids of objects. The similarity measures using 9D-SPA representation provide a wide range of fuzzy matching capability in similarity retrieval to meet different user's requirements. Experimental results showed that our system is very effective in terms of recall and precision. In addition, the 9D-SPA representation can be incorporated into a two-level index structure to help reduce the search space of each query processing. The experimental results also demonstrated that, on average, only 0.1254 percent /spl sim/ 1.6829 percent of symbolic pictures (depending on various degrees of similarity) were accessed per query in an image database containing 50,000 symbolic pictures.     

图像中物体空间关系的表示及在检索中的应用

基于图像中物体之间的空间关系的图像检索往往受困于待处理的图像中物体种类和空间位置难以自动准确地获取。文中基于物体识别算法的输出,提出一种对物体空间关系的三元组表示法,给出基于这种表示方法对图像索引、相似度计算和检索排序的方法及允许用户使用查询词和空间关系表达查询需求的二维输入界面,并实现原型系统。这种表示法具有良好的鲁棒性,可容忍物体识别算法一定程度的误差,将物体识别得到的置信度加入三元组表示法置信度计算和排序算法中,减少物体识别结果误差对检索性能的影响。在原型系统上的实验表明,该系统在实验中对包含物体位置关系的检索给出更准确的结果,在NDCG@m、MAP、F@m上均优于现有系统。     

Unified framework for representing and ranking

In the database retrieval and nearest neighbor classification tasks, the two basic problems are to represent the query and database objects, and to learn the ranking scores of the database objects to the query. Many studies have been conducted for the representation learning and the ranking score learning problems, however, they are always learned independently from each other. In this paper, we argue that there are some inner relationships between the representation and ranking of database objects, and try to investigate their relationships by learning them in a unified way. To this end, we proposed the Unified framework for Representation and Ranking (UR²) of objects for the database retrieval and nearest neighbor classification tasks. The learning of representation parameter and the ranking scores are modeled within one single unified objective function. The objective function is optimized alternately with regard to representation parameter and the ranking scores. Based on the optimization results, iterative algorithms are developed to learn the representation parameter and the ranking scores on a unified way. Moreover, with two different formulas of representation (feature selection and subspace learning), we give two versions of UR². The proposed algorithms are tested on two challenging tasks – MRI image based brain tumor retrieval and nearest neighbor classification based protein identification. The experiments show the advantage of the proposed unified framework over the state-of-the-art independent representation and ranking methods.     

Harmonizing Hierarchical Manifolds for Multimedia Document Semantics Understanding and Cross-Media Retrieval

In this paper, we consider the problem of multimedia document (MMD) semantics understanding and content-based cross-media retrieval. An MMD is a set of media objects of different modalities but carrying the same semantics and the content-based cross-media retrieval is a new kind of retrieval method by which the query examples and search results can be of different modalities. Two levels of manifolds are learned to explore the relationships among all the data in the level of MMD and in the level of media object respectively. We first construct a Laplacian media object space for media object representation of each modality and an MMD semantic graph to learn the MMD semantic correlations. The characteristics of media objects propagate along the MMD semantic graph and an MMD semantic space is constructed to perform cross-media retrieval. Different methods are proposed to utilize relevance feedback and experiment shows that the proposed approaches are effective.     

基于浅层语义分析技术的语义检索

在信息检索领域,语义检索技术较传统的关键字检索,无论在检索效果还是用户体验方面,都有诸多优势。语义检索融合了信息检索、语义分析以及信息融合等诸多方法,已成为现阶段该领域研究的一项重要技术。在Lu-cene索引技术基础之上,提出了语义检索的方法,即对语句进行语义分析,获得一种描述语句浅层语义信息的形式化表示,并对这种形式化表示建立索引;将表述语义联系的多层次相似度通过信息融合技术进行融合,并将其映射成查询语句与索引数据之间的相似度,达到语义检索的目的。     

Spatial reasoning and similarity retrieval for image database systems based on RS-strings

In this paper, we propose a rotation-invariant spatial knowledge representation called RS-string. Then we present the string generation algorithm to automatically generate RS-strings for segmented pictures. We also propose the spatial reasoning and similarity retrieval algorithms based on RS-strings. The similarity retrieval algorithm is much more flexible than all previous 2D string representations because our approach can consider every possible view of a query picture. Thus the system does not require the user to provide a query picture which must have the same orientation as that of a database picture. Finally, we provide several examples to demonstrate the capabilities of spatial reasoning and similarity retrieval based on the RS-string representation.     

Spatial match retrieval based on direction signatures using multiple key hashing scheme

The spatial relationships among pictorial objects are important spatial characteristics in image database systems. Based on the concept of 9-DLT representation, we propose a new method for effective storage utilization and picture retrieval. First, we transform a symbolic picture into a set of triples associated with pairwise spatial relationships among objects. Then, an associated normalized signature record with nine attributes is specified. Finally, each normalized record of this image database can be well allocated by using our proposed multiple key hashing scheme and can be fast accessed for spatial match retrieval. The performance formula for spatial picture retrieval is presented and the theoretically optimal solutions to the bit lengths of all the attributes are derived in certain case. Also, we give a systematic method to solve the feasible bit lengths for the attributes based on the method proposed by Chang. © 1997 John Wiley & Sons, Inc.     

Reasoning about pictures and similarity retrieval for image information systems based on SK-set knowledge representation

Spatial reasoning and similarity retrieval are two important functions of any image information system. Good spatial knowledge representation for images is necessary to adequately support these two functions. In this paper, we propose a new spatial knowledge representation, called the SK-set based on morphological skeleton theories. Spatial reasoning algorithms which achieve more accurate results by directly analysing skeletons are described. SK-set facilitates browsing and progressive visualization. We also define four new types of similarity measures and propose a similarity retrieval algorithm for performing image retrieval. Moreover, using SK-set as a spatial knowledge representation will reduce the storage space required by an image database significantly.     

Spatial arrangement of color in retrieval by visual similarity

In image search based on chromatic similarity, the effectiveness of retrieval can be improved by taking into account the spatial arrangement of colors. This can serve both to distinguish images with the same colors in different arrangement, and to capture the similarity between images with different colors but similar arrangements. We propose a model of representation and comparison which attains this goal by partitioning the image in separate entities and by associating them with individual chromatic attributes and with mutual spatial relationships. The effectiveness of the proposed model is assessed in a user-based evaluation. Experimental results show the capability of the model to join and balance chromatic and spatial similarity, thus improving the effectiveness of retrieval with respect to representations based on a global histogram.     

A probabilistic framework for semantic video indexing, filtering,and retrieval

Semantic filtering and retrieval of multimedia content is crucial for efficient use of the multimedia data repositories. Video query by semantic keywords is one of the most difficult problems in multimedia data retrieval. The difficulty lies in the mapping between low-level video representation and high-level semantics. We therefore formulate the multimedia content access problem as a multimedia pattern recognition problem. We propose a probabilistic framework for semantic video indexing, which call support filtering and retrieval and facilitate efficient content-based access. To map low-level features to high-level semantics we propose probabilistic multimedia objects (multijects). Examples of multijects in movies include explosion, mountain, beach, outdoor, music etc. Semantic concepts in videos interact and to model this interaction explicitly, we propose a network of multijects (multinet). Using probabilistic models for six site multijects, rocks, sky, snow, water-body forestry/greenery and outdoor and using a Bayesian belief network as the multinet we demonstrate the application of this framework to semantic indexing. We demonstrate how detection performance can be significantly improved using the multinet to take interconceptual relationships into account. We also show how the multinet can fuse heterogeneous features to support detection based on inference and reasoning     

Indexing and Retrieving Motions of Characters in Close Contact

Human motion indexing and retrieval are important for animators due to the need to search for motions in the database which can be blended and concatenated. Most of the previous researches of human motion indexing and retrieval compute the Euclidean distance of joint angles or joint positions. Such approaches are difficult to apply for cases in which multiple characters are closely interacting with each other, as the relationships of the characters are not encoded in the representation. In this research, we propose a topology-based approach to index the motions of two human characters in close contact. We compute and encode how the two bodies are tangled based on the concept of rational tangles. The encoded relationships, which we define as {it TangleList}, are used to determine the similarity of the pairs of postures. Using our method, we can index and retrieve motions such as one person piggy-backing another, one person assisting another in walking, and two persons dancing / wrestling. Our method is useful to manage a motion database of multiple characters. We can also produce motion graph structures of two characters closely interacting with each other by interpolating and concatenating topologically similar postures and motion clips, which are applicable to 3D computer games and computer animation.     

Iconic indexing by 2-d strings

In this paper, we describe a new way of representing a symbolic picture by a two-dimensional string. A picture query can also be specified as a 2-D string. The problem of pictorial information retrieval then becomes a problem of 2-D subsequence matching. We present algorithms for encoding a symbolic picture into its 2-D string representation, reconstructing a picture from its 2-D string representation, and matching a 2-D string with another 2-D string. We also prove the necessary and sufficient conditions to characterize ambiguous pictures for reduced 2-D strings as well as normal 2-D strings. This approach thus allows an efficient and natural way to construct iconic indexes for pictures.     

基于小波多尺度分析的彩色图像检索方法

多媒体技术的普及和Internet技术的实施导致了大量图像信息的出现，基于文本关键词的传统检索方法已不能适应图像信息检索的要求，这使得基于内容的图像检索技术逐渐成为目前的研究热点。基于内容检索技术中必不可少的关键步骤就是图像特征的提取，其中可提取的特征有颜色、纹理和形状等。但是，由于图像的每种特征只能抓住图像相似性的某一个方面，因此如何能更好地表示图像就成为基于内容图像检索中一个重要的研究方向。针对该问题，提出了一种基于图像颜色和纹理特征的图像检索方法，其中颜色特征采用HSV颜色空间的直方图，纹理特征采用图像小波多尺度表示方法中细节信息的方差统计量，这样就充分利用了颜色的丰富表现性和小波变换的多分辨性及其变换系数的统计特性。通过对不同类型图像使用不同特征组合进行图像检索查准率的对比实验结果表明，这种图像检索方法是行之有效的。     

Ontology-based topological representation of remote-sensing images

This article proposes an ontology-based topological representation of remote-sensing images. Semantics, especially related to the topological relationships between the objects represented, are not explicit in remote-sensing images and this fact limits spatial analysis. Our aim is to provide an explicit ontological definition of the topological relations between objects in the image using the Quadtree data structure for spatial indexing. This structure is explicitly defined in an ontology allowing the automatic interpretation of the representations obtained, taking into account the topological relations and increasing the spatial analytical capabilities. This representation has been validated by a case study of semantic retrieval based on the normalized difference vegetation index (NDVI), taking into account the topological relations between NDVI regions in images. In the experiments, we compare the effectiveness of results from eight queries using four traditional supervised image classification algorithms and the proposal representation. The experimental results show the feasibility of the proposal, supporting the concept of the image retrieval process providing a semantic complement to remote-sensing images. The proposed representation contributes to incorporation of semantics into geographical data, especially to remote-sensing images, and it can be used to develop applications in the Geospatial Semantic Web.     

SENEX: a computer-based representation of cellular signal transduction processes in the central nervous system

The SENEX project is exploring knowledge representation in the neurobiology of ageing through object-oriented programming. SENEX is built from a classification structure of biologic entities and significant relationships among them. For example, an enzyme is an entity and an enzymatic reaction is a relationship among enzyme, cofactor(s), substrate(s) and product(s). There are currently 2600 classes of entities and 50 classes of relationships in SENEX. The class structure serves several functions. One function is to interrelate general and specific categories of molecular and morphologic entities. For example, tyrosine kinase and serine/threonine kinase are specific types of the more general class of protein kinase enzymes. Another function of the class structure is to serve as a network through which inheritance of attributes may occur. For example, the attribute 'subunits' is inherited by all subclasses of the general class multisubunit protein. Information may be accessed through links established in the class structure and through links relating one object as part of another. Relationships form the basis of separate modules within SENEX. This paper describes the types of relationships currently used and planned in the representation of age-related changes in cellular signal transduction processes of mammalian central nervous systems. We also describe tools for specific retrieval of relationships and for tracing links in complex reaction cascades. Application of these tools to identifying possible signal transduction pathways to guide further exploration through experimentation is discussed.