Variational Multi-Valued Velocity Field Estimation for Transparent Sequences

Motion estimation in sequences with transparencies is an important problem in robotics and medical imaging applications. In this work we propose a variational approach for estimating multi-valued velocity fields in transparent sequences. Starting from existing local motion estimators, we derive a variational model for integrating in space and time such a local information in order to obtain a robust estimation of the multi-valued velocity field. With this approach, we can indeed estimate multi-valued velocity fields which are not necessarily piecewise constant on a layer—each layer can evolve according to a non-parametric optical flow. We show how our approach outperforms existing methods; and we illustrate its capabilities on challenging experiments on both synthetic and real sequences.     

A Database and Evaluation Methodology for Optical Flow

The quantitative evaluation of optical flow algorithms by Barron et al. (1994) led to significant advances in performance. The challenges for optical flow algorithms today go beyond the datasets and evaluation methods proposed in that paper. Instead, they center on problems associated with complex natural scenes, including nonrigid motion, real sensor noise, and motion discontinuities. We propose a new set of benchmarks and evaluation methods for the next generation of optical flow algorithms. To that end, we contribute four types of data to test different aspects of optical flow algorithms: (1) sequences with nonrigid motion where the ground-truth flow is determined by tracking hidden fluorescent texture, (2) realistic synthetic sequences, (3) high frame-rate video used to study interpolation error, and (4) modified stereo sequences of static scenes. In addition to the average angular error used by Barron et al., we compute the absolute flow endpoint error, measures for frame interpolation error, improved statistics, and results at motion discontinuities and in textureless regions. In October 2007, we published the performance of several well-known methods on a preliminary version of our data to establish the current state of the art. We also made the data freely available on the web at . Subsequently a number of researchers have uploaded their results to our website and published papers using the data. A significant improvement in performance has already been achieved. In this paper we analyze the results obtained to date and draw a large number of conclusions from them.     

Discriminative semi-supervised learning of dynamical systems for motion estimation

We introduce novel discriminative semi-supervised learning algorithms for dynamical systems, and apply them to the problem of 3D human motion estimation. Our recent work on discriminative learning of dynamical systems has been proven to achieve superior performance than traditional generative learning approaches. However, one of the main issues of learning the dynamical systems is to gather labeled output sequences which are typically obtained from precise motion capture tools, hence expensive. In this paper we utilize a large amount of unlabeled (input) video data to improve the prediction performance of the dynamical systems significantly. We suggest two discriminative semi-supervised learning approaches that extend the well-known algorithms in static domains to the sequential, real-valued multivariate output domains: (i) self-training which we derive as coordinate ascent optimization of a proper discriminative objective over both model parameters and the unlabeled state sequences, (ii) minimum entropy approach which maximally reduces the model's uncertainty in state prediction for unlabeled data points. These approaches are shown to achieve significant improvement against the traditional generative semi-supervised learning methods. We demonstrate the benefits of our approaches on the 3D human motion estimation problems.     

Variational-Bayes Optical Flow

The Horn-Schunck (HS) optical flow method is widely employed to initialize many motion estimation algorithms. In this work, a variational Bayesian approach of the HS method is presented, where the motion vectors are considered to be spatially varying Student’s t-distributed unobserved random variables, i.e., the prior is a multivariate Student’s t-distribution, while the only observations available is the temporal and spatial image difference. The proposed model takes into account the residual resulting from the linearization of the brightness constancy constraint by Taylor series approximation, which is also assumed to be a spatially varying Student’s t-distributed observation noise. To infer the model variables and parameters we recur to variational inference methodology leading to an expectation-maximization (EM) framework with update equations analogous to the Horn-Schunck approach. This is accomplished in a principled probabilistic framework where all of the model parameters are estimated automatically from the data. Experimental results show the improvement obtained by the proposed model which may substitute the standard algorithm in the initialization of more sophisticated optical flow schemes.     

Non Uniform Multiresolution Method for Optical Flow and Phase Portrait Models: Environmental Applications

In this paper we define a complete framework for processing large image sequences for a global monitoring of short range oceanographic and atmospheric processes. This framework is based on the use of a non quadratic regularization technique for optical flow computation that preserves flow discontinuities. We also show that using an appropriate tessellation of the image according to an estimate of the motion field can improve optical flow accuracy and yields more reliable flows. This method defines a non uniform multiresolution approach for coarse to fine grid generation. It allows to locally increase the resolution of the grid according to the studied problem. Each added node refines the grid in a region of interest and increases the numerical accuracy of the solution in this region. We make use of such a method for solving the optical flow equation with a non quadratic regularization scheme allowing the computation of optical flow field while preserving its discontinuities. The second part of the paper deals with the interpretation of the obtained displacement field. For this purpose a phase portrait model used along with a new formulation of the approximation of an oriented flow field allowing to consider arbitrary polynomial phase portrait models for characterizing salient flow features. This new framework is used for processing oceanographic and atmospheric image sequences and presents an alternative to complex physical modeling techniques.     

Efficient preflow push algorithms

Algorithms for the maximum flow problem can be grouped into two categories: augmenting path algorithms [Ford LR, Fulkerson DR. Flows in networks. Princeton University Press: Princeton, NJ: 1962], and preflow push algorithms [Goldberg AV, Tarjan RE. A new approach to the maximum flow problem. In: Proceedings of the 18th annual ACM symposium on theory of computing, 1986; p. 136–46]. Preflow push algorithms are characterized by a drawback known as ping pong effect. In this paper we present a technique that allows to avoid such an effect and can be considered as an approach combining the augmenting path and preflow push methods. An extended experimentation shows the effectiveness of the proposed approach.     

Improving sub-pixel correspondence through upsampling

Many fundamental computer vision problems, including optical flow estimation and stereo matching, involve the key step of computing dense color matching among pixels. In this paper, we show that by merely upsampling, we can improve sub-pixel correspondence estimation. In addition, we identify the regularization bias problem and explore its relationship to image resolution. We propose a general upsampling framework to compute sub-pixel color matching for different computer vision problems. Various experiments were performed on motion estimation and stereo matching data. We are able to reduce errors by up to 30%, which would otherwise be very difficult to achieve through other conventional optimization methods.     

基于全自动生长分割的梯度彩色光流场计算

形变物体边界的准确定位是光流估计的难点之一,仅依靠改进光流算法收效甚微。提出了全自动生长分割,以准确提取形变运动物体,从而将视频分割结果与梯度彩色光流算法相结合来提高光流法对形变物体的检测准确法度     

Regularization of optical flow with M-band wavelet transform

The optical flow is an important tool for problems arising in the analysis of image sequences. Flow fields generated by various existing solving techniques are often noisy and partially incorrect, especially near occlusions or motion boundaries. Therefore, the additional information on the scene gained from a sequence of images is usually worse. In this paper, discrete wavelet transform has been adopted in order to enhance the reliability of optical flow estimation. A generalization of the well-known dyadic orthonormal wavelets to the case of the dilation scale factor M > 2 with N vanishing moments has been used, and it has proved to be a useful regularizing tool. The advantages in the computations have been shown by experimental results performed on real image sequences.     

On the design of high-performance algorithms for aligning multiple protein sequences on mesh-based multiprocessor architectures

In this paper, we address the problem of multiple sequence alignment (MSA) for handling very large number of proteins sequences on mesh-based multiprocessor architectures. As the problem has been conclusively shown to be computationally complex, we employ divisible load paradigm (also, referred to as divisible load theory, DLT) to handle such large number of sequences. We design an efficient computational engine that is capable of conducting MSAs by exploiting the underlying parallelism embedded in the computational steps of multiple sequence algorithms. Specifically, we consider the standard Smith–Waterman (SW) algorithm in our implementation, however, our approach is by no means restrictive to SW class of algorithms alone. The treatment used in this paper is generic to a class of similar dynamic programming problems. Our approach is recursive in the sense that the quality of solutions can be refined continuously till an acceptable level of quality is achieved. After first phase of computation, we design a heuristic scheme that renders the final solution for MSA. We conduct rigorous simulation experiments using several hundreds of homologous protein sequences derived from the Rattus Norvegicus and Mus Musculus databases of olfactory receptors. We quantify the performance based on speed-up metric. We compare our algorithms to serial or single machine processing approaches. We testify our findings by comparing with conventional equal load partitioning (ELP) strategy that is commonly used in the parallel processing literature. Based on our extensive simulation study, we observe that DLT paradigm offers an excellent speed-up characteristics and provides avenues for its use in several other biological sequence processing related problem. This study is a first time attempt in using the DLT paradigm to devise efficient strategies to handle large scale multiple protein sequence alignment problem on mesh-based multiprocessor systems.     

Ginisupport vector machines for segmental minimum Bayes risk decoding of continuous speech

We describe the use of support vector machines (SVMs) for continuous speech recognition by incorporating them in segmental minimum Bayes risk decoding. Lattice cutting is used to convert the Automatic Speech Recognition search space into sequences of smaller recognition problems. SVMs are then trained as discriminative models over each of these problems and used in a rescoring framework. We pose the estimation of a posterior distribution over hypotheses in these regions of acoustic confusion as a logistic regression problem. We also show that GiniSVMs can be used as an approximation technique to estimate the parameters of the logistic regression problem. On a small vocabulary recognition task we show that the use of GiniSVMs can improve the performance of a well trained hidden Markov model system trained under the Maximum Mutual Information criterion. We also find that it is possible to derive reliable confidence scores over the GiniSVM hypotheses and that these can be used to good effect in hypothesis combination. We discuss the problems that we expect to encounter in extending this approach to large vocabulary continuous speech recognition and describe initial investigation of constrained estimation techniques to derive feature spaces for SVMs.     

Motion estimation and segmentation

In the general structure-from-motion (SFM) problem involving several moving objects in a scene, the essential first step is to segment moving objects independently. We attempt to deal with the problem of optical flow estimation and motion segmentation over a pair of images. We apply a mean field technique to determine optical flow and motion boundaries and present a deterministic algorithm. Since motion discontinuities represented by line process are embedded in the estimation of the optical flow, our algorithm provides accurate estimates of optical flow especially along motion boundaries and handles occlusion and multiple motions. We show that the proposed algorithm outperforms other well-known algorithms in terms of estimation accuracy and timing.     

Estimation-based metaheuristics for the probabilistic traveling salesman problem

The probabilistic traveling salesman problem (PTSP) is a central problem in stochastic routing. Recently, we have shown that empirical estimation is a promising approach to devise highly effective local search algorithms for the PTSP. In this paper, we customize two metaheuristics, an iterated local search algorithm and a memetic algorithm, to solve the PTSP. This customization consists in adopting the estimation approach to evaluate the solution cost, exploiting a recently developed estimation-based local search algorithm, and tuning the metaheuristics parameters. We present an experimental study of the estimation-based metaheuristic algorithms on a number of instance classes. The results show that the proposed algorithms are highly effective and that they define a new state-of-the-art for the PTSP.     

Exploration and exploitation of scratch games

We consider a variant of the multi-armed bandit model, which we call scratch games, where the sequences of rewards are finite and drawn in advance with unknown starting dates. This new problem is motivated by online advertising applications where the number of ad displays is fixed according to a contract between the advertiser and the publisher, and where a new ad may appear at any time. The drawn-in-advance assumption is natural for the adversarial approach where an oblivious adversary is supposed to choose the reward sequences in advance. For the stochastic setting, it is functionally equivalent to an urn where draws are performed without replacement. The non-replacement assumption is suited to the sequential design of non-reproducible experiments, which is often the case in real world. By adapting the standard multi-armed bandit algorithms to take advantage of this setting, we propose three new algorithms: the first one is designed for adversarial rewards; the second one assumes a stochastic urn model; and the last one is based on a Bayesian approach. For the adversarial and stochastic approaches, we provide upper bounds of the regret which compare favorably with the ones of Exp3 and UCB1. We also confirm experimentally that these algorithms compare favorably with Exp3, UCB1 and Thompson Sampling by simulation with synthetic models and ad-serving data.     

Performance evaluation of proposed Differential Evolution and Particle Swarm Optimization algorithms for scheduling m-machine flow shops with lot streaming

We consider n-job, m-machine lot streaming problem in a flow shop with equal size sub lots where the objective is to minimize the makespan and total flow time. Lot streaming (Lot sizing) is a technique that splits a production lot consisting of identical items into sub lots to improve the performance of a multi stage production system by over lapping the sub lots on successive machines. There is a scope for efficient algorithms for scheduling problems in m-machine flow shop with lot streaming. In recent years, much attention is given to heuristics and search techniques. To solve this problem, we propose a Differential Evolution Algorithm (DEA) and Particle Swarm Optimization (PSO) to evolve best sequence for makespan/total flow time criterion for m-machine flow shop involved with lot streaming and set up time. In this research, we propose the DEA and PSO algorithms for discrete lot streaming with equal sub lots. The proposed methods are tested and the performances were evaluated. The computational results show that the proposed algorithms are very competitive for the lot streaming flow shop scheduling problem.     

大位移变分光流计算的快速算法

目的 多尺度方法的提出解决了传统HS（Horn Schunck）算法不能计算大位移光流的问题,但同时也增加了迭代运算的步数。为加快迭代收敛速度,研究大位移变分光流计算的快速算法,并分析其性能。方法 将用于加快变分图像处理迭代运算的Split Bregman方法、对偶方法和交替方向乘子法应用到大位移光流计算中。结果 分别进行了精度、迭代步数、运行时间的对比实验。引入3种快速方法的模型均能够在保证精度的同时,在较少时间内计算出图像序列的光流场,所需时间为传统方法的11%~42%。结论 将3种快速方法应用到大位移变分光流计算中,对于不同图像序列均可以较大地提高计算效率。     

Optimizing regenerator cost in traffic grooming

In optical networks, regenerators have to be placed on lightpaths every d consecutive nodes in order to regenerate the signal. In addition, grooming enables the use of the same regenerator by several lightpaths. Up to g (the grooming factor) lightpaths can use the same regenerator. In this work we consider the problem of minimizing the number of regenerators used in traffic grooming in optical networks. Starting from the 4-approximation algorithm of Flammini et al. (2010) [10] for d=1 and a path topology, we provide an approximation algorithm with the same approximation ratio for d=1and the ring and tree topologies. We present also a technique based on matching that leads to the same approximation ratio in tree topology and can be used to obtain approximation algorithms in other topologies. We provide an approximation algorithm for general topology that uses this technique. Finally, all the results are extended to the case of general d.     

Trajectory-Based Place-Recognition for Efficient Large Scale Localization

Place recognition is a core competency for any visual simultaneous localization and mapping system. Identifying previously visited places enables the creation of globally accurate maps, robust relocalization, and multi-user mapping. To match one place to another, most state-of-the-art approaches must decide a priori what constitutes a place, often in terms of how many consecutive views should overlap, or how many consecutive images should be considered together. Unfortunately, such threshold dependencies limit their generality to different types of scenes. In this paper, we present a placeless place recognition algorithm using a novel match-density estimation technique that avoids heuristically discretizing the space. Instead, our approach considers place recognition as a problem of continuous matching between image streams, automatically discovering regions of high match density that represent overlapping trajectory segments. The algorithm uses well-studied statistical tests to identify the relevant matching regions which are subsequently passed to an absolute pose algorithm to recover the geometric alignment. We demonstrate the efficiency and accuracy of our methodology on three outdoor sequences, including a comprehensive evaluation against ground-truth from publicly available datasets that shows our approach outperforms several state-of-the-art algorithms for place recognition. Furthermore we compare our overall algorithm to the currently best performing system for global localization and show how we outperform the approach on challenging indoor and outdoor datasets.     

Progressive Refinement Imaging

This paper presents a novel technique for progressive online integration of uncalibrated image sequences with substantial geometric and/or photometric discrepancies into a single, geometrically and photometrically consistent image. Our approach can handle large sets of images, acquired from a nearly planar or infinitely distant scene at different resolutions in object domain and under variable local or global illumination conditions. It allows for efficient user guidance as its progressive nature provides a valid and consistent reconstruction at any moment during the online refinement process. Our approach avoids global optimization techniques, as commonly used in the field of image refinement, and progressively incorporates new imagery into a dynamically extendable and memory-efficient Laplacian pyramid. Our image registration process includes a coarse homography and a local refinement stage using optical flow. Photometric consistency is achieved by retaining the photometric intensities given in a reference image, while it is being refined. Globally blurred imagery and local geometric inconsistencies due to, e.g. motion are detected and removed prior to image fusion. We demonstrate the quality and robustness of our approach using several image and video sequences, including handheld acquisition with mobile phones and zooming sequences with consumer cameras.     

Remote estimation of chlorophyll a in optically complex waters based on optical classification

Accurate assessment of phytoplankton chlorophyll a (Chla) concentration in turbid waters by means of remote sensing is challenging due to optically complexity and significant variability of case 2 waters, especially in inland waters with multiple optical types. In this study, a water optical classification algorithm is developed, and two semi-analytical algorithms (three- and four-band algorithm) for estimating Chla are calibrated and validated using four independent datasets collected from Taihu Lake, Chaohu Lake, and Three Gorges Reservoir. The optical classification algorithm is developed using the dataset collected in Taihu Lake from 2006 to 2009. This dataset is also used to calibrate the three- and four-band Chla estimation algorithms. The optical classification technique uses remote sensing reflectance at three bands: Rrs(G), Rrs(650), and Rrs(NIR), where G indicates the location of reflectance peak in the green region (around 560 nm), and NIR is the location of reflectance peak in the near-infrared region (around 700 nm). Optimal reference wavelengths of the three- and four-band algorithm are located through model tuning and accuracy optimization. The three- and four-band algorithm accuracy is further evaluated using other three independent datasets. The improvement of optical classification in Chla estimation is revealed by comparing the performance of the two algorithms for non-classified and classified waters.Using the slopes of the three reflectance bands, the 138 reflectance spectra samples in the calibration dataset are classified into three classes, each with a specific spectral shape character. The three- and four-band algorithm performs well for both non-classified and classified waters in estimating Chla. For non-classified waters, strong relationships are yielded between measured and predicted Chla, but the performance of the two algorithms is not satisfactory in low Chla conditions, especially for samples with Chla below 30 mg m^− 3. For classified waters, the class-specific algorithms perform better than for non-classified waters. Class-specific algorithms reduce considerable mean relative error from algorithms for non-classified waters in Chla predicting. Optical classification makes that there is no need to adjust the optimal position to estimate Chla for other waters using the class-specific algorithms. The findings in this study demonstrate that optical classification can greatly improve the accuracy of Chla estimation in optically complex waters.