Multimedia data mining: state of the art and challenges

Advances in multimedia data acquisition and storage technology have led to the growth of very large multimedia databases. Analyzing this huge amount of multimedia data to discover useful knowledge is a challenging problem. This challenge has opened the opportunity for research in Multimedia Data Mining (MDM). Multimedia data mining can be defined as the process of finding interesting patterns from media data such as audio, video, image and text that are not ordinarily accessible by basic queries and associated results. The motivation for doing MDM is to use the discovered patterns to improve decision making. MDM has therefore attracted significant research efforts in developing methods and tools to organize, manage, search and perform domain specific tasks for data from domains such as surveillance, meetings, broadcast news, sports, archives, movies, medical data, as well as personal and online media collections. This paper presents a survey on the problems and solutions in Multimedia Data Mining, approached from the following angles: feature extraction, transformation and representation techniques, data mining techniques, and current multimedia data mining systems in various application domains. We discuss main aspects of feature extraction, transformation and representation techniques. These aspects are: level of feature extraction, feature fusion, features synchronization, feature correlation discovery and accurate representation of multimedia data. Comparison of MDM techniques with state of the art video processing, audio processing and image processing techniques is also provided. Similarly, we compare MDM techniques with the state of the art data mining techniques involving clustering, classification, sequence pattern mining, association rule mining and visualization. We review current multimedia data mining systems in detail, grouping them according to problem formulations and approaches. The review includes supervised and unsupervised discovery of events and actions from one or more continuous sequences. We also do a detailed analysis to understand what has been achieved and what are the remaining gaps where future research efforts could be focussed. We then conclude this survey with a look at open research directions.     

Multimodal fusion for multimedia analysis: a survey

This survey aims at providing multimedia researchers with a state-of-the-art overview of fusion strategies, which are used for combining multiple modalities in order to accomplish various multimedia analysis tasks. The existing literature on multimodal fusion research is presented through several classifications based on the fusion methodology and the level of fusion (feature, decision, and hybrid). The fusion methods are described from the perspective of the basic concept, advantages, weaknesses, and their usage in various analysis tasks as reported in the literature. Moreover, several distinctive issues that influence a multimodal fusion process such as, the use of correlation and independence, confidence level, contextual information, synchronization between different modalities, and the optimal modality selection are also highlighted. Finally, we present the open issues for further research in the area of multimodal fusion.     

On burst rescheduling in OBS networks with partial wavelength conversion capability

The core nodes in an optical burst switching (OBS) network are normally equipped with wavelength converters (WCs) to reduce the burst loss probability. Since WCs are expensive and still immature technologically, it is desirable to reduce the number of WCs in the network and to have partial wavelength conversion capability at the core nodes. Nevertheless, a majority of algorithms in the literature are proposed under the full wavelength conversion assumption. As a result, they do not consider the burst loss caused by insufficient WCs, i.e., bursts dropped due to the unavailability of free WCs to convert them to unused wavelengths. In this paper, we demonstrate how to use burst rescheduling to decrease the burst loss due to insufficient WCs and hence cut down on the overall burst loss probability in OBS networks. Two burst rescheduling algorithms are proposed. Their effectiveness in reducing the overall burst loss probability is verified through simulation experiments.     

IIR system identification using cat swarm optimization

Conventional derivative based learning rule poses stability problem when used in adaptive identification of infinite impulse response (IIR) systems. In addition the performance of these methods substantially deteriorates when reduced order adaptive models are used for such identification. In this paper the IIR system identification task is formulated as an optimization problem and a recently introduced cat swarm optimization (CSO) is used to develop a new population based learning rule for the model. Both actual and reduced order identification of few benchmarked IIR plants is carried out through simulation study. The results demonstrate superior identification performance of the new method compared to that achieved by genetic algorithm (GA) and particle swarm optimization (PSO) based identification.     

Distributed evolutionary multi-objective mesh-partitioning algorithm for parallel finite element computations

Majority of the mesh-partitioning algorithms attempt to optimise the interprocessor communications, while balancing the computational load among the processors. However, it is desirable to simultaneously optimise the submesh aspect ratios in order to significantly improve the convergence characteristics of the domain decomposition based Preconditioned-conjugate-gradient algorithms, being used extensively in the state-of-the-art parallel finite element codes. Keeping this in view, a new distributed multi-objective mesh-partitioning algorithm using evolutionary computing techniques is proposed in this paper. Effectiveness of the proposed distributed mesh-partitioning algorithm is demonstrated by solving several unstructured meshes of practical-engineering problems and also benchmark problems.     

Generating human-like soccer primitives from human data

Recently, interest in analysis and generation of human and human-like motion has increased in various areas. In robotics, in order to operate a humanoid robot, it is necessary to generate motions that have strictly dynamic consistency. Furthermore, human-like motion for robots will bring advantages such as energy optimization.This paper presents a mechanism to generate two human-like motions, walking and kicking, for a biped robot using a simple model based on observation and analysis of human motion. Our ultimate goal is to establish a design principle of a controller in order to achieve natural human-like motions. The approach presented here rests on the principle that in most biological motor learning scenarios some form of optimization with respect to a physical criterion is taking place. In a similar way, the equations of motion for the humanoid robot systems are formulated in such a way that the resulting optimization problems can be solved reliably and efficiently.The simulation results show that faster and more accurate searching can be achieved to generate an efficient human-like gait. Comparison is made with methods that do not include observation of human gait. The gait has been successfully used to control Robo-Erectus, a soccer-playing humanoid robot, which is one of the foremost leading soccer-playing humanoid robots in the RoboCup Humanoid League.     

A compact printed multistubs loaded resonator rectangular monopole antenna design for multiband wireless systems

In the present article, a compact triple‐band multistubs loaded resonator printed monopole antenna is proposed. The antenna consists of a quarter wavelength two asymmetrical inverted L‐shaped stubs to excite two resonant modes for 3.5/5.5 GHz bands and one integrated horizontally T‐shaped stub with inverted long L‐shaped stub to excite resonant mode for 2.5 GHz band. By loading these stub resonators along y‐axis with distinct gaps, the antenna resonates at three frequencies 2.57/3.52/5.51 GHz covering the desired bands while keeping compact size of 24 × 30 mm² (0.2 × 0.25 ). The proposed antenna is fabricated on Rogers RT/duroid 5880 substrate with thickness 0.79 mm and its performance experimentally verified. The measured results reveal that the antenna has the impedance bandwidths of about 210 MHz (2.50‐2.71 GHz), 260 MHz (3.37‐3.63 GHz), and 650 MHz (5.20‐5.85 GHz), for 2.5/3.5/5.5 GHz WiMAX and 5.2/5.8 GHz WLAN band systems. The antenna provides omnidirectional radiation patterns and flat antenna gains over the three operating bands. In addition, the design approach and effects of multistubs resonator lengths on the operating bands are also examined and discussed in detail.     

Concept-based near-duplicate video clip detection for novelty re-ranking of web video search results

State-of-the-art near-duplicate video clip (NDVC) detection for novelty re-ranking uses non-semantic low-level features (color/texture) to detect and eliminate “content-based NDVC” and increases content level novelty in the top results. However, humans may perceive a video as near duplicate from a semantic perspective as well. In this paper, we propose concept-based near-duplicate video clip (CBNDVC) detection technique for novelty re-ranking. We identify “semantic NDVC”, making use of the semantic features (events/concepts) and re-rank the top results to increase the content as well as semantic novelty. Videos are represented as a multivariate time series of confidence values of relevant concepts and thereafter discovery of CBNDVC clusters is achieved by conceptual clustering. Obtained results show higher precision and recall from the user’s perspective.     

Grounded discovery of symbols as concept–language pairs

In human designer usage, symbols have a rich semantics, grounded on experience, which permits flexible usage — e.g. design ideation is improved by meanings triggered by contrastive words. In computational usage however, symbols are syntactic tokens whose semantics is mostly left to the implementation, resulting in brittle failures in many knowledge-based systems. Here we ask if one may define symbols in computational design as {label,meaning} pairs, as opposed to merely the label. We consider three questions that must be answered to bootstrap a symbol learning process: (a) which concepts are most relevant in a given domain, (b) how to define the semantics of such symbols, and (c) how to learn labels for these so as to form a grounded symbol. We propose that relevant symbols may be discovered by learning patterns of functional viability. The stable patterns are information-conserving codes, also called chunks in cognitive science, which relate to the process of acquiring expertise in humans. Regions of a design space that contain functionally superior designs can be mapped to a lower-dimensional manifold; the inter-relations of the design variables discovered thus constitute the chunks. Using these as the initial semantics for symbols, we show how the system can acquire labels for them by communicating with human designers. We demonstrate the first steps in this process in our baby designer approach, by learning two early grounded symbols, tight and loose.