Scene-adaptive accurate and fast vertical crowd counting via joint using depth and color information

Reliable and real-time crowd counting is one of the most important tasks in intelligent visual surveillance systems. Most previous works only count passing people based on color information. Owing to the restrictions of color information influences themselves for multimedia processing, they will be affected inevitably by the unpredictable complex environments (e.g. illumination, occlusion, and shadow). To overcome this bottleneck, we propose a new algorithm by multimodal joint information processing for crowd counting. In our method, we use color and depth information together with a ordinary depth camera (e.g. Microsoft Kinect). Specifically, we first detect each head of the passing or still person in the surveillance region with adaptive modulation ability to varying scenes on depth information. Then, we track and count each detected head on color information. The characteristic advantage of our algorithm is that it is scene adaptive, which means the algorithm can be applied into all kinds of different scenes directly without additional conditions. Based on the proposed approach, we have built a practical system for robust and fast crowd counting facing complicated scenes. Extensive experimental results show the effectiveness of our proposed method.     

Pan-sharpening:a fast variational fusion approach

A fast variational fusion model based on partial differential equations (PDEs) is presented for pansharpening. The functional framework consists of several energy terms. The gradient energy term is created by calculating the gradient vector field of the panchromatic image and the geometry of the pan is injected into the multi-spectral bands. The radiometric reduction energy term and the channel correlation energy term are defined to decrease the radiometric distortion and preserve the correlation of multi-spectral channels while enforcing the inter-bands coherence. Inspired by the shock-filtering model, an inverse diffusion term for image enhancement is put to PDEs which are deduced by minimizing the energy functional. In comparison with the state-of-the-art fusion approaches based on `a trous wavelet and non-sampled contourlet, our model can obtain the fused image with a high spatial or spectral quality by adjusting the weight coefficients of the energy terms. It can also achieve a rather good trade-off between the spatial resolution improvement and the spectral quality preserving. Our model’s computational complexity for one time step is only O(N).     

Multi-object tracking via MHT with multiple information fusion in surveillance video

Tracking multiple objects is critical to automatic video content analysis and virtual reality. The major problem is how to solve data association problem when ambiguous measurements are caused by objects in close proximity. To tackle this problem, we propose a multiple information fusion-based multiple hypotheses tracking algorithm integrated with appearance feature, local motion pattern feature and repulsion–inertia model for multi-object tracking. Appearance model based on HSV–local binary patterns histogram and local motion pattern based on optical flow are adopted to describe objects. A likelihood calculation framework is proposed to incorporate the similarities of appearance, dynamic process and local motion pattern. To consider the changes in appearance and motion pattern over time, we make use of an effective template updating strategy for each object. In addition, a repulsion–inertia model is adopted to explore more useful information from ambiguous detections. Experimental results show that the proposed approach generates better trajectories with less missing objects and identity switches.     

An adaptive PNN-DS approach to classification using multi-sensor information fusion

In this paper, an adaptive neural network approach to classification which combines modified probabilistic neural network and D-S evidence theory (PNN-DS) is proposed. It attempts to deal with the drawbacks of information uncertainty and imprecision using single classification algorithm. This PNN-DS approach firstly adopts a modified PNN to obtain posteriori probabilities and make a primary classification decision in feature-level fusion. Then posteriori probabilities are transformed to masses noting the evidence of the D-S evidential theory. Finally advanced D-S evidential theory is utilized to gain more accurate classification results in the last decision-level fusion. In order to implement PNN-DS, covariance matrices are firstly employed in the modified PNN module to replace the singular smoothing factor in the PNN’s kernel function, and linear function is utilized in the pattern of summation layer. Secondly, the whole scheme of the proposed approach is explained in depth. Thirdly, three classification experiments are carried out on the proposed approach and a large amount of comparable analyses are done to demonstrate the effectiveness and robustness of the proposed approach. Experiments reveal that the PNN-DS outperforms BPNN-DS, which provides encouraging results in terms of classification accuracy and the speed of learning convergence.     

An objective-oriented approach to program comprehension using multiple information sources

Program comprehension is a key activity throughout software maintenance and reuse. The knowledge acquired through comprehending programs can guide engineers to perform various kinds of software maintenance and reuse tasks. The effective comprehension strategy and the associated efficient approach, as well as the sophisticated tool support, are the indispensable elements for an entire solution to program comprehension to reduce the high costs of this nontrivial activity. This paper presents an objective-oriented comprehension strategy, contrasting to the traditional comprehensive understanding strategy in the literature. It is a kind of on-demand understanding for specific tasks and more effective in practice. In addition, using multiple information sources to understand programs is proposed with the corresponding framework. From these two points of views, we propose a feature-oriented program comprehension approach using requirement documentation. This approach aims at a specific category of feature-related software maintenance and reuse tasks. Case studies are conducted to evaluate the proposed solution. Results from the studied cases show that the experimental prototype provides more explicit advices for software engineers when performing these tasks.     

Recognizing activities in multiple views with fusion of frame judgments

This paper focuses on activity recognition when multiple views are available. In the literature, this is often performed using two different approaches. In the first one, the systems build a 3D reconstruction and match that. However, there are practical disadvantages to this methodology since a sufficient number of overlapping views is needed to reconstruct, and one must calibrate the cameras. A simpler alternative is to match the frames individually. This offers significant advantages in the system architecture (e.g., it is easy to incorporate new features and camera dropouts can be tolerated). In this paper, the second approach is employed and a novel fusion method is proposed. Our fusion method collects the activity labels over frames and cameras, and then fuses activity judgments as the sequence label. It is shown that there is no performance penalty when a straightforward weighted voting scheme is used. In particular, when there are enough overlapping views to generate a volumetric reconstruction, our recognition performance is comparable with that produced by volumetric reconstructions. However, if the overlapping views are not adequate, the performance degrades fairly gracefully, even in cases where test and training views do not overlap.     

多传感器信息融合伺服的移动机器人快速跟随

针对移动机器人快速跟随任务的需要,采取了多传感器信息融合的伺服控制,用双目摄像机对移动机器人进行视觉导航,利用激光雷达和超声波传感器完成移动机器人的避障设计,通过位于机器人轮部的光电编码器反馈信息处理实现机器人的自定位,语音交互和手势交互共同完成移动机器人的人机交互,制定了移动机器人快速跟随整体策略并提出了软硬件系统集成方案.在实验环境中通过实验和实践表明了移动机器人能够快速顺利的完成跟随任务.     

An integrated approach to high-level information fusion

In today’s fast paced military operational environment, vast amounts of information must be sorted out and fused not only to allow commanders to make situation assessments, but also to support the generation of hypotheses about enemy force disposition and enemy intent. Current information fusion technology has the following two limitations. First, current approaches do not consider the battlefield context as a first class entity. In contrast, we consider situational context in terms of terrain analysis and inference. Second, there are no integrated and implemented models of the high-level fusion process. This paper describes the HiLIFE (High-Level Information Fusion Environment) computational framework for seamless integration of high levels of fusion (levels 2, 3 and 4). The crucial components of HiLIFE that we present in this paper are: (1) multi-sensor fusion algorithms and their performance results that operate in heterogeneous sensor networks to determine not only single targets but also force aggregates, (2) computational approaches for terrain-based analysis and inference that automatically combine low-level terrain features (such as forested areas, rivers, etc.) and additional information, such as weather, and transforms them into high-level militarily relevant abstractions, such as NO-GO, SLOW-GO areas, avenues of approach, and engagement areas, (3) a model for inferring adversary intent by mapping sensor readings of opponent forces to possible opponent goals and actions, and (4) sensor management for positioning intelligence collection assets for further data acquisition. The HiLIFE framework closes the loop on information fusion by specifying how the different components can computationally work together in a coherent system. Furthermore, the framework is inspired by a military process, the Intelligence Preparation of the Battlefield, that grounds the framework in practice. HiLIFE is integrated with a distributed military simulation system, OTBSAF, and the RETSINA multi-agent infrastructure to provide agile and sophisticated reasoning. In addition, the paper presents validation results of the automated terrain analysis that were obtained through experiments using military intelligence Subject Matter Experts (SMEs).     

An approach for measuring semantic similarity between words using multiple information sources

Semantic similarity between words is becoming a generic problem for many applications of computational linguistics and artificial intelligence. This paper explores the determination of semantic similarity by a number of information sources, which consist of structural semantic information from a lexical taxonomy and information content from a corpus. To investigate how information sources could be used effectively, a variety of strategies for using various possible information sources are implemented. A new measure is then proposed which combines information sources nonlinearly. Experimental evaluation against a benchmark set of human similarity ratings demonstrates that the proposed measure significantly outperforms traditional similarity measures.     

Behavior monitoring for assistive environments using multiple views

This work presents an approach to behavior understanding using multiple cameras. This approach is appropriate for monitoring people in an assistive environment for the purpose of issuing alerts in cases of abnormal behavior. The output of multiple classifiers is used to model and extract abnormal behavior from both the target trajectory and the target short-term activity (i.e., walking, running, abrupt motion, etc.). Spatial information is obtained after an offline camera registration using homography information. The proposed approach is verified experimentally in an indoor environment. The experiments are performed with a single moving target; however, the method can be generalized to multiple moving targets, which may occlude each other, due to the use of multiple cameras.     

New approach to information fusion steady-state Kalman filtering

By the modern time series analysis method, based on the autoregressive moving average (ARMA) innovation model, a unified and general information fusion steady-state Kalman filtering approach is presented for the general multisensor systems with different local dynamic models and correlated noises. It can handle the filtering, smoothing, and prediction fusion problems for state or signal. The optimal fusion rule weighted by matrices is re-derived as a weighted least squares (WLS) fuser, and is reviewed. An optimal fusion rule weighted by diagonal matrices is presented, which is equivalent to the optimal fusion rule weighted by scalars for components, and it realizes a decoupled fusion. The new algorithms of the steady-state Kalman estimator gains are presented. In order to compute the optimal weights, the formulas of computing the cross-covariances among local estimation errors by Lyapunov equations are presented. The exponential convergence of the iterative solution of Lyapunov equation is proved. It is proved that the optimal fusion estimators under three weighted fusion rules are locally optimal, but are globally suboptimal. The proposed steady-state Kalman fusers can reduce the on-line computational burden, and are suitable for real-time applications. A simulation example for the 3-sensor steady-state Kalman tracking fusion estimators shows their effectiveness and correctness, and gives the accuracy comparison of the fusion rules.     

Managing information technology investments using a real-options approach

Investments in IT have become a dominant part of the capital budgets of many organizations. While the costs seem readily identifiable, many of the benefits are elusive. In this paper we develop a formal and practical methodology to evaluate information technology infrastructure investments. Our experience in using this approach has shown that it not only impacts the outcome but also improved understanding of how to align operating drivers with business capabilities and investment decisions. Thus, the real value of the real options approach is in how it informs the management process of IT investments.     

Combining multiple clusterings using fast simulated annealing

This paper presents a fast simulated annealing framework for combining multiple clusterings based on agreement measures between partitions, which are originally used to evaluate a clustering algorithm. Although we can follow a greedy strategy to optimize these measures as the objective functions of clustering ensemble, it may suffer from local convergence and simultaneously incur too large computational cost. To avoid local optima, we consider a simulated annealing optimization scheme that operates through single label changes. Moreover, for the measures between partitions based on the relationship (joined or separated) of pairs of objects, we can update them incrementally for each label change, which ensures that our optimization scheme is computationally feasible. The experimental evaluations demonstrate that the proposed framework can achieve promising results.     

A multiple layer fusion approach on keystroke dynamics

In this paper, we present a novel keystroke dynamic recognition system by means of a novel two-layer fusion approach. First, we extract four types of keystroke latency as the feature from our dataset. The keystroke latency will be transformed into similarity scores via Gaussian Probability Density Function (GPD). We also propose a new technique, known as Direction Similarity Measure (DSM), which measures the absolute difference between two sets of latency. Last, four fusion approaches coupled with six fusion rules are applied to improve the final result by combining the scores that are produced by GPD and DSM. Best result with equal error rate of 1.401% is obtained with our two-layer fusion approach.     

Deep learning based origin-destination prediction via contextual information fusion

Origin-Destination (OD) prediction which aims to predict the number of passenger’s travel demands from one region to another, is critically important to many real applications including intelligent transportation systems and public safety. The challenges of this problem lie in both the dynamic patterns of the human mobility data and data sparsity in issue in some regions. Thus it is difficult to model the complex spatio-temporal correlations of the human mobility data to predict the OD of their trips. Meanwhile, the crowd flows in different regions of a city and the context features (e.g. holiday, weather and POIs) are potentially useful to alleviate the data sparsity issue and improve the OD prediction, but are largely ignored by existing works. In this paper, we propose a deep spatio-temporal framework which named Auxiliary-tasks Enhanced Spatio-Temporal Network (AEST) to more effectively address the OD prediction problem. AEST trains a model to conduct OD inference via learning crowd flow and external data as auxiliary task. The novel Hierarchical Convolutional LSTM (HC-LSTM) Network is proposed which combines CNN, GCN and LSTM to effectively capture spatiao-temporal correlations. In addition, we design a Contextual Network (ContextNet) which learns representations of contextual information to assist OD prediction. We conduct extensive experiments over bike and taxicab trip datasets in New York. The results show that our method is superior to the state-of-art approaches.

     

Visual information retrieval system via content-based approach

In this paper a content-based image retrieval method that can search large image databases efficiently by color, texture, and shape content is proposed. Quantized RGB histograms and the dominant triple (hue, saturation, and value), which are extracted from quantized HSV joint histogram in the local image region, are used for representing global/local color information in the image. Entropy and maximum entry from co-occurrence matrices are used for texture information and edge angle histogram is used for representing shape information. Relevance feedback approach, which has coupled proposed features, is used for obtaining better retrieval accuracy. A new indexing method that supports fast retrieval in large image databases is also presented. Tree structures constructed by k-means algorithm, along with the idea of triangle inequality, eliminate candidate images for similarity calculation between query image and each database image. We find that the proposed method reduces calculation up to average 92.2 percent of the images from direct comparison.     

RGBD co-saliency detection via multiple kernel boosting and fusion

Multimedia Tools and Applications - RGBD co-saliency detection, which aims at extracting common salient objects from a group of RGBD images with the additional depth information, has become an...     

A general approach to the fusion of imprecise information

We consider the problem of fusion of multiple information sources, particularly in environments when the sensor observations are imprecise. The concept of a combinability relationship is introduced to enable the inclusion in the fusion process of information about the appropriateness of fusing different elements from the observation space. This idea allows for the use of an expert knowledge base, containing information about the domain of the particular problem, in the fusion process and leads to a more intelligent aggregation. We show that if we use a combinability relationship that only allows fusion of identical elements then the only idempotent fusion of any collection of fuzzy observations is their intersection. Using the idea of the fuzzy measure we considered situations in which we allow a partial collection of the observations determine the fused value. © 1997 John Wiley & Sons, Inc.