Enabling context-aware multimedia annotation by a novel?generic semantic problem-solving platform

Automatic generation of metadata, facilitating the retrieval of multimedia items, potentially saves large amounts of manual work. However, the high specialization degree of feature extraction algorithms makes them unaware of the context they operate in, which contains valuable and often necessary information. In this paper, we show how Semantic Web technologies can provide a context that algorithms can interact with. We propose a generic problem-solving platform that uses Web services and various knowledge sources to find solutions to complex requests. The platform employs a reasoner-based composition algorithm, generating an execution plan that combines several algorithms as services. It then supervises the execution of this plan, intervening in case of errors or unexpected behavior. We illustrate our approach by a use case in which we annotate the names of people depicted in a photograph.     

Graph-based multi-space semantic correlation propagation for?video retrieval

By introducing the concept detection results to the retrieval process, concept-based video retrieval (CBVR) has been successfully used for semantic content-based video retrieval application. However, how to select and fuse the appropriate concepts for a specific query is still an important but difficult issue. In this paper, we propose a novel and effective concept selection method, named graph-based multi-space semantic correlation propagation (GMSSCP), to explore the relationship between the user query and concepts for video retrieval application. Compared with traditional methods, GMSSCP makes use of a manifold-ranking algorithm to collectively explore the multi-layered relationships between the query and concepts, and the expansion result is more robust to noises. Parallel to this, GMSSCP has a query-adapting property, which can enhance the process of concept correlation propagation and selection with strong pertinence of query cues. Furthermore, it can dynamically update the unified propagation graph by flexibly introducing the multi-modal query cues as additional nodes, and is not only effective for automatic retrieval but also appropriate for the interactive case. Encouraging experimental results on TRECVID datasets demonstrate the effectiveness of GMSSCP over the state-of-the-art concept selection methods. Moreover, we also apply it to the interactive retrieval system??VideoMap and gain an excellent performance and user experience.     

Image retrieval ++—web image retrieval with an enhanced multi-modality ontology

In this paper we present an enhanced multi-modality ontology-based approach for web image retrieval step by step. Several ontology-based approaches have been made in the field of multimedia retrieval. Our multi-modality approach is one of the earliest attempts to integrate information from different modalities and apply the model in a complex domain. In order to develop the model, we need to answer the following questions: (1) how to find the proper structure and construct an ontology which can integrate information from different modalities; (2) how to quantify the matching degree (concept similarity) and provide an independent ranking mechanism; (3) how to ensure the scalability of this approach when applied to large domains. The first question has been answered by our multi-modality ontology which has been discussed in Wang et al. (Does ontology help in image retrieval? In: Asia-Pacific workshop on visual information processing, 2006) and its extension (Wang et al., Does ontology help in image retrieval?—a comparison between keyword, text ontology and multi-modality ontology approaches, ACM Press, New York, NY, USA, pp 109–112, 2006). More details about this work is given later. The main focus of this paper is that we propose a new ranking mechanism using Spearman’s ranking correlation to measure the similarity of concepts in the ontology. We take the priorities of information from different modalities into consideration. This algorithm gives the answer of the second question. The semantic matchmaking result is quantized and the degree of similarity between concepts is calculated. For the third question, importing of ontology will resolve the scalability issue but computing concept similarity and identify relationships when integrating different ontologies will be beyond the scope of this paper. To convince readers that our multi-modality ontology and concept similarity ranking is the right step forward, we decided to work on the animal kingdom. We believe this domain is challenging as demonstrated by images depict animals in a wide range of aspects, pose, configurations and appearances. We experimented with a data sets of 4,000 web images. Based on ground truth, we analyze the image content and text information, build up the enhanced multi-modality ontology and compare the retrieval results. Results show that we can even classify close animal species which share similar appearances and we can infer their hidden relationships from the canine family graph. By assigning a ranking to the semantic relationships we show unequivocal evidence that our improved model achieves good accuracy and performs comparable result with the Google re-ranking result in our previous work.     

Drag-and-drop multimedia: ¶an interface framework for digital libraries

In this paper, we describe a new interface for querying multimedia digital libraries and an interface building framework. The interface employs a drag-and-drop style of interaction and combines a structured natural-language style query specification with reusable multimedia objects. We call this interface DanDMM, short for “drag-and-drop multimedia”. DanDMM interfaces capture the syntax of the underlying query language, and dynamically reconfigure to reflect the contents of the data repository. A distinguishing feature of DanDMM is its ability to synthesize integrated interfaces that incorporate both example-based specification using multimedia objects, and traditional techniques including keyword, attribute, and free text-based search. We describe the DanDMM-builder, a framework for synthesizing DanDMM interfaces, and give several examples of interfaces that have been constructed using DanDMM-builder, including a remote-sensing library application and a video digital library. Received: 15 December 1997 / Revised: June 1999     

HealthFinland—A national semantic publishing network and portal for health information

Providing citizens with reliable, up-to-date and individually relevant health information on the Web is done by governmental, non-governmental, business and other organizations. Currently the information is published with little co-ordination and co-operation between the publishers. For publishers, this means duplicated work and costs due to publishing same information twice on many websites. Also maintaining links between websites requires work. From the citizens point of view, finding content is difficult due to e.g. differences in layman’s vocabularies compared to medical terminology and difficulties in aggregating information from several sites.To solve these problems, we present a national scale semantic publishing system HealthFinland which consists of (1) a centralized content infrastructure of health ontologies and services with tools, (2) a distributed semantic content creation channel based on several health organizations, and (3) an intelligent semantic portal aggregating and presenting the contents from intuitive and health promoting end-user perspectives for human users as well as for other websites and portals.     

Geotagging in multimedia and computer vision—a survey

Geo-tagging is a fast-emerging trend in digital photography and community photo sharing. The presence of geographically relevant metadata with images and videos has opened up interesting research avenues within the multimedia and computer vision domains. In this paper, we survey geo-tagging related research within the context of multimedia and along three dimensions: (1) Modalities in which geographical information can be extracted, (2) Applications that can benefit from the use of geographical information, and (3) The interplay between modalities and applications. Our survey will introduce research problems and discuss significant approaches. We will discuss the nature of different modalities and lay out factors that are expected to govern the choices with respect to multimedia and vision applications. Finally, we discuss future research directions in this field.     

Highly parallel inference engine PIE —Goal rewriting model and machine architecture—

Logic programming is expected to make knowledge information processing feasible. However, conventional Prolog systems lack both processing power and flexibility for solving large problems. To overcome these limitations, an approach is developed in which natural execution features of logic programs can be represented using Proof Diagrams. AND/ OR parallel processing based on a goal-rewriting model is examined. Then the abstract architecture of a highly parallel inference engine (PIE) is described. PIE makes it possible to achieve logic/control separation in machine architecture. The architecture proposed here is discussed from the viewpoint of its high degree of parallelism and flexibility in problem solving in comparison with other approaches.     

Automatic image–text alignment for large-scale web image indexing and retrieval

In this paper, an automatic image–text alignment algorithm is developed to achieve more effective indexing and retrieval of large-scale web images by aligning web images with their most relevant auxiliary text terms or phrases. First, a large number of cross-media web pages (which contain web images and their auxiliary texts) are crawled and segmented into a set of image–text pairs (informative web images and their associated text terms or phrases). Second, near-duplicate image clustering is used to group large-scale web images into a set of clusters of near-duplicate images according to their visual similarities. The near-duplicate web images in the same cluster share similar semantics and are simultaneously associated with a same or similar set of auxiliary text terms or phrases which co-occur frequently in the relevant text blocks, thus performing near-duplicate image clustering can significantly reduce the uncertainty on the relatedness between the semantics of web images and their auxiliary text terms or phrases. Finally, random walk is performed over a phrase correlation network to achieve more precise image–text alignment by refining the relevance scores between the web images and their auxiliary text terms or phrases. Our experiments on algorithm evaluation have achieved very positive results on large-scale cross-media web pages.     

Robotic batting system — an architecture for learning and dynamic pattern generation

The robot map-making of a scene containing unknown objects of arbitrary shapes is considered. A specific formulation of the terrain acquisition problem due to Lumelsky et al. is investigated. The aim is to develop efficient new algorithms and to understand the basics of the problem. A new, generalized algorithm-GenTer-is developed. GenTer produces a family of algorithms through the variation of a scalar parameter . A version, Terl, of the generalized algorithm may offer a better performance, on the average, than the existing algorithm, the Sightseer Strategy. Terl offers a new feature called hierarchical map-making, through which a good appropriate map can be efficiently created.     

SAS—an experimental tool for dynamic program structure acquisition and analysis

This paper describes an experimental microprocessor-based tool, SAS (Software Analysis System), which has been developed to enable dynamic program structure acquistion and analysis to be made on digital computing machines.The system uses a universal hardware extraction technique to obtain branch vectors which are used to analyse and display the structure of the software being monitored. A display, especially designed for small instrument screens, is used to present this structure. Emphasis has been directed towards development of methods with high degrees of machine independence and it is envisaged that such techniques could either be integrated into the new generation of logic analysers or form part of a universal tool for computer programmers. Initial research has been guided towards the application of these techniques to compiled, assembled, or machine coded systems and in this context a number of techniques are described.The motivation for this research has been provided by the present escalating software costs, in particular those in post development which account for approximately 75% of the total software expenditure.     

SHAPE—an approach for self-healing and self-protection in complex distributed networks

Increasing complexity of large scale distributed systems is creating problem in managing faults and security attacks because of the manual style adopted for management. This paper proposes a novel approach called SHAPE to self-heal and self-protect the system from various kinds of faults and security attacks. It deals with hardware, software, and network faults and provides security against DDoS, R2L, U2L, and probing attacks. SHAPE is implemented and evaluated against various standard metrics. The results are provided to support the approach.     

Proxy signatures and buyer–seller watermarking protocols for the protection of multimedia content

Watermarking techniques are developed to deal with multimedia distribution, authentication and copyright protection. It is usually the seller who embeds a watermark in multimedia content to identify the buyer. The embedded watermark can then be used to trace the traitors identity if unauthorized copies are found. However, repudiation and framing issues might arise in this approach. To solve these problems, buyer–seller watermarking protocols have been proposed based on watermarking in the encrypted domain. Such watermarks combine encryption, digital watermarking, and fingerprinting to preserve digital rights of both the buyer and the seller. Unfortunately, most existing watermarking techniques do not provide convincing proofs to ensure that they achieve the claimed level of security and informal proofs abound in the literature. In this paper, we propose a buyer–seller watermarking protocol based on proxy signatures and homomorphic encryption. Formal proofs are provided to show that in the proposed protocol, watermarks are generated such that the seller is unable to fabricate piracy, but he can trace copyright violators. The protocol further protects anonymity of the buyer until he is adjudicated to be guilty. Moreover, we solve the conspiracy problem without imposing any unrealistic assumptions about thrust-worthiness of the parties involved.     

Magix—an ICAE system for Problem-Based Learning

Intelligent Computer Aided Education (ICAE) systems are software, aimed at assisting in the teaching and learning of students. Magix is a prototype ICAE system for use in Problem-Based Learning (PBL). In this system the principles of constructivism, user-driven interaction, knowledge-based systems and metacognition are integrated. The fact that the use of a standard algorithm in problem-based learning problem-solving cannot be enforced, greatly complicates the development of the student model. Magix features a student model that consists of a cognitive component supplemented by a metacognitive component. The aim is to externalize the student model in order to facilitate reflection and encourage critical, creative and corrective self-monitoring. The system consists of two main subsystems, Magix-Ed the hypertext-based student component and Magix-An the expert system that gives an analysis of the problem-solving attempt performed in Magix-Ed. Magix-Ed presents the problem to be solved in a virtual world. The world consists of objects that visually represent the problem and tools that could be used to solve the problem. Magix-An analyses a student's problem-solving attempts. An evaluation report is produced that aims to encourage certain thinking dispositions, while enriching the student's repertoire of strategies through autoregulation.     

A comparison between phonetic engine and GMM–UBM classifier for language identification tasks

Microsystem Technologies - In this paper, two automatic language identification (LID) systems are compared. One of the systems is the Hidden Markov Model (HMM) based phonetic engine (PE), and the...     

Development of hybrid hydraulic–electric power units for field and service robots

Energetic autonomy of a hydraulic-based mobile robot requires a power source capable of both hydraulic and electrical power generation. The hydraulic power is used for locomotion, and the electric power is used for the control computer, sensors and other peripherals. In addition, the power source must be lightweight and quiet. This study presents several designs of internal combustion engine-based power units. Each power unit is evaluated with a Ragone plot which shows its performance over a wide range of operation times. Several hydraulic–electric power units (HEPUs) were built and successfully demonstrated on the Berkeley lower extremity exoskeleton (BLEEX). The best-performing design of the HEPUs, based upon the Ragone plot analysis, is described in detail. This HEPU produces constant pressure hydraulic power and constant voltage electric power. The pressure and voltage are controlled on board the power unit by a computer. A novel characteristic of this power unit is its cooling system in which hydraulic fluid is used to cool the engine cylinders. The prototype power unit weighs 27 kg and produces 2.3 kW (3.0 hp) hydraulic power at 6.9 MPa (1000 p.s.i.) and 220 W of electric power at 15 V DC.     

The modelling of an engine injector mechanism as a dynamic gain element and its application†

The recent availability of extensive experimental results on the frequency response testing of a diesel engine has lead to an investigation of the influence of the injector mechanism on the dynamic behaviour. It is shown that this mechanism can be modelled as a dynamic gain element which, together with the authors' previously developed models for this type of engine, shows why difficulties were experienced in obtaining the frequency response plots. Additionally, the developed theory shows why diesel engines tend to exhibit unstable behaviour when lightly loaded.     

Mímir: An open-source semantic search framework for interactive information seeking and discovery

Semantic search is gradually establishing itself as the next generation search paradigm, which meets better a wider range of information needs, as compared to traditional full-text search. At the same time, however, expanding search towards document structure and external, formal knowledge sources (e.g. LOD resources) remains challenging, especially with respect to efficiency, usability, and scalability.This paper introduces Mímir—an open-source framework for integrated semantic search over text, document structure, linguistic annotations, and formal semantic knowledge. Mímir supports complex structural queries, as well as basic keyword search.Exploratory search and sense-making are supported through information visualisation interfaces, such as co-occurrence matrices and term clouds. There is also an interactive retrieval interface, where users can save, refine, and analyse the results of a semantic search over time. The more well-studied precision-oriented information seeking searches are also well supported.The generic and extensible nature of the Mímir platform is demonstrated through three different, real-world applications, one of which required indexing and search over tens of millions of documents and fifty to hundred times as many semantic annotations. Scaling up to over 150 million documents was also accomplished, via index federation and cloud-based deployment.