Multimodal fusion for multimedia analysis: a survey

This survey aims at providing multimedia researchers with a state-of-the-art overview of fusion strategies, which are used for combining multiple modalities in order to accomplish various multimedia analysis tasks. The existing literature on multimodal fusion research is presented through several classifications based on the fusion methodology and the level of fusion (feature, decision, and hybrid). The fusion methods are described from the perspective of the basic concept, advantages, weaknesses, and their usage in various analysis tasks as reported in the literature. Moreover, several distinctive issues that influence a multimodal fusion process such as, the use of correlation and independence, confidence level, contextual information, synchronization between different modalities, and the optimal modality selection are also highlighted. Finally, we present the open issues for further research in the area of multimodal fusion.     

Editors’ note

Editorial Note

Editors’ note     

Haar invariant signatures and spatial recognition using omnidirectional visual information only

This paper describes a method for spatial representation, place recognition and qualitative self-localization in dynamic indoor environments, based on omnidirectional images. This is a difficult problem because of the perceptual ambiguity of the acquired images, and their weak robustness to noise, geometrical and photometric variations of real world scenes. The spatial representation is built up invariant signatures using Invariance Theory where we suggest to adapt Haar invariant integrals to the particular geometry and image transformations of catadioptric omnidirectional sensors. It follows that combining simple image features in a process of integration over visual transformations and robot motion, can build discriminant percepts about robot spatial locations. We further analyze the invariance properties of the signatures and the apparent relation between their similarity measures and metric distances. The invariance properties of the signatures can be adapted to infer a hierarchical process, from global room recognition to local and coarse robot localization.     

Collaborative user modeling with user-generated tags for social recommender systems

With the popularity of social media services, the sheer amount of content is increasing exponentially on the Social Web that leads to attract considerable attention to recommender systems. Recommender systems provide users with recommendations of items suited to their needs. To provide proper recommendations to users, recommender systems require an accurate user model that can reflect a user’s characteristics, preferences and needs. In this study, by leveraging user-generated tags as preference indicators, we propose a new collaborative approach to user modeling that can be exploited to recommender systems. Our approach first discovers relevant and irrelevant topics for users, and then enriches an individual user model with collaboration from other similar users. In order to evaluate the performance of our model, we compare experimental results with a user model based on collaborative filtering approaches and a vector space model. The experimental results have shown the proposed model provides a better representation in user interests and achieves better recommendation results in terms of accuracy and ranking.     

A new expert system for learning management systems evaluation based on neutrosophic sets

There has been a sudden increase in the usage of Learning Management Systems applications to support learner's learning process in higher education. Many studies in learning management system evaluation are implemented under complete information, while the real environment has uncertainty aspects. As these systems were described by development organizations with uncertainty terms such as vague, imprecise, ambiguity and inconsistent, that is why traditional evaluation methods may not be effective. This paper suggests neutrosophic logic as a better option to simulate human thinking than fuzzy logic because unlike fuzzy logic, it is able to handle indeterminacy of information which expresses the percentage of unknown parameters. As previous studies suggested neutrosophic decision maker and neutrosophic expert systems as future work in ecommerce and e‐learning applications, this paper presents neutrosphic expert system for learning management systems evaluation. Information for building and validating the neutrosophic expert system is collected from five experts using surveys, and then analysis is done by using Fuzzytech 5.54d software. Finally, the comparison between fuzzy expert system and neutrosophic expert system results show that the neutrosophic logic is capable of representing uncertainty in human thinking for evaluating Learning Management Systems.     

Disassembly Sequencing Using Tabu Search

End-of-life disassembly has developed into a major research area within the sustainability paradigm, resulting in the emergence of several algorithms and structures proposing heuristics techniques such as Genetic Algorithm (GA), Ant Colony Optimization (ACO) and Neural Networks (NN). The performance of the proposed methodologies heavily depends on the accuracy and the flexibility of the algorithms to accommodate several factors such as preserving the precedence relationships during disassembly while obtaining near- optimal and optimal solutions. This paper improves a previously proposed Genetic Algorithm model for disassembly sequencing by utilizing a faster metaheuristic algorithm, Tabu search, to obtain the optimal solution. The objectives of the proposed algorithm are to minimize (1) the traveled distance by the robotic arm, (2) the number of disassembly method changes, and (3) the number of robotic arm travels by combining the identical-material components together and hence eliminating unnecessary disassembly operations. In addition to improving the quality of optimum sequence generation, a comprehensive statistical analysis comparing the previous Genetic Algorithm and the proposed Tabu Search Algorithm is also included     

A two-stage gene selection scheme utilizing MRMR filter and GA wrapper

Gene expression data usually contain a large number of genes, but a small number of samples. Feature selection for gene expression data aims at finding a set of genes that best discriminates biological samples of different types. In this paper, we propose a two-stage selection algorithm for genomic data by combining MRMR (Minimum Redundancy–Maximum Relevance) and GA (Genetic Algorithm). In the first stage, MRMR is used to filter noisy and redundant genes in high-dimensional microarray data. In the second stage, the GA uses the classifier accuracy as a fitness function to select the highly discriminating genes. The proposed method is tested for tumor classification on five open datasets: NCI, Lymphoma, Lung, Leukemia and Colon using Support Vector Machine (SVM) and Naïve Bayes (NB) classifiers. The comparison of the MRMR-GA with MRMR filter and GA wrapper shows that our method is able to find the smallest gene subset that gives the most classification accuracy in leave-one-out cross-validation (LOOCV).     

Validating the ISO/IEC 15504 measure of software requirementsanalysis process capability

ISO/IEC 15504 is an emerging international standard on software process assessment. It defines a number of software engineering processes and a scale for measuring their capability. One of the defined processes is software requirements analysis (SRA). A basic premise of the measurement scale is that higher process capability is associated with better project performance (i.e., predictive validity). The paper describes an empirical study that evaluates the predictive validity of SRA process capability. Assessments using ISO/IEC 15504 were conducted on 56 projects world-wide over a period of two years. Performance measures on each project were also collected using questionnaires, such as the ability to meet budget commitments and staff productivity. The results provide strong evidence of predictive validity for the SRA process capability measure used in ISO/IEC 15504, but only for organizations with more than 50 IT staff. Specifically, a strong relationship was found between the implementation of requirements analysis practices as defined in ISO/IEC 15504 and the productivity of software projects. For smaller organizations, evidence of predictive validity was rather weak. This can be interpreted in a number of different ways: that the measure of capability is not suitable for small organizations or that the SRA process capability has less effect on project performance for small organizations     

Optimal structure for heat and cold protection under transient heat conduction

A suitable combination of materials for sheltering a system from a sudden change of environmental temperature has been theoretically studied. The protective composite wall consists of two materials. An insulating material is placed on the outer surface, while, for the inner surface, materials that have good heat storage properties but negligible heat transfer resistance are chosen. The results show that by replacing some of the insulation material with a heat storage material, the temperature of the protected system can be maintained at a considerably lower level. Although the optimal thickness ratio X depends on the Biot number, Fourier number, and on the heat capacity ratio K _C, for a large number of thermal protection cases, the approximation X = 0.45 yields practically the minimum progress of the transient. If the Biot number is sufficiently small, it is better to replace all of the insulation material with a good heat storage material.     

Curve/surface representation and evolution using vector level sets with application to the shape-based segmentation problem

In this paper, we revisit the implicit front representation and evolution using the vector level set function (VLSF) proposed in (H. E. Abd El Munim, et al., Oct. 2005). Unlike conventional scalar level sets, this function is designed to have a vector form. The distance from any point to the nearest point on the front has components (projections) in the coordinate directions included in the vector function. This kind of representation is used to evolve closed planar curves and 3D surfaces as well. Maintaining the VLSF property as the distance projections through evolution will be considered together with a detailed derivation of the vector partial differential equation (PDE) for such evolution. A shape-based segmentation framework will be demonstrated as an application of the given implicit representation. The proposed level set function system will be used to represent shapes to give a dissimilarity measure in a variational object registration process. This kind of formulation permits us to better control the process of shape registration, which is an important part in the shape-based segmentation framework. The method depends on a set of training shapes used to build a parametric shape model. The color is taken into consideration besides the shape prior information. The shape model is fitted to the image volume by registration through an energy minimization problem. The approach overcomes the conventional methods problems like point correspondences and weighing coefficients tuning of the evolution (PDEs). It is also suitable for multidimensional data and computationally efficient. Results in 2D and 3D of real and synthetic data will demonstrate the efficiency of the framework