光场成像技术及其在计算机视觉中的应用

目的 光场成像技术刚刚在计算机视觉研究中展开初步应用,其相关研究比较零散,缺乏系统性。本文旨在系统介绍光场成像技术发展以及其应用在计算机视觉研究中有代表性的工作。方法 从解决计算机视觉问题的角度出发,4个层面讨论光场成像技术最近十年的研究工作,包括:1)主流的光场成像设备及其作为计算机视觉传感器的优点与不足;2)光场相机作为视觉传感器的标定、解码以及预处理方法;3)基于4维光场的图像渲染与重建技术,以及其如何促进计算机视觉研究;4)以4维光场数据为基础的特征表达方法。结果 逐层梳理出光场成像在求解视觉问题中的优势和局限,分析其中根本性的原理与掣肘,力图总结出亟待解决的关键问题以及未来的发展趋势。结论 作为一种颇具前景的新型计算机视觉传感器技术,光场成像技术的研究必将更为广泛和深入。研究应用于计算机视觉的光场成像技术将有力的引导和促进计算机视觉和光场成像技术协同发展。     

海面目标运动速度的视觉估计方法

为方便快速地实现对平面直角导航坐标系下匀速直航海面目标的速度测量,提出一种立体视觉估计方法。该方法首先基于计算机视觉理论标定得到摄像机参数(包括内参数及外参数),建立立体视觉测量数学模型,然后利用光电成像设备连续获取目标图像序列,结合图像目标特征角点信息测量被测目标相对距离,同时计算得到不同时刻目标方位。在此基础上,根据目标运动几何关系,采用拟线性方法处理目标运动方程,最后实现对目标速度的最小二乘估计。通过大量仿真实验,研究了不同初始条件对目标速度估计性能的影响。结果表明,对于3000m以内航速在10Kn~30Kn的海面目标,估计精度优于0.2m/s。该方法要求简单,便于实现,所得结论可为岸基监控海面目标或水下隐蔽观测站仅利用图像信息快速判断被测目标运动态势及作战决策提供参考。     

深度学习及其在计算机视觉领域中的应用

首先回顾了计算视觉发展的历史,介绍了神经元、多层感知机和反向传播等人工神经网络的基本知识以及卷积神经网络的发展史及其卷积、池化等基本操作;讨论了AlexNet、VGGNet、GoogLeNet和ResNet等经典卷积神经网络结构,并重点介绍了CapsNet;总结了卷积神经网络在图像分类、语义分割、目标检测以及图像生成等领域的研究进展;最后提出了卷积神经网络研究所面临的挑战以及对CapsNet未来研究的展望。     

Combining vision and computer graphics for video motion capture

We describe innovative methods for extracting three-dimensional motion of humans and animals from unrestricted monocular video, using a combination of new and established computer vision and computer graphics techniques. We identity features using image processing and active contours. Active contours become anchored to model segments in specified image frames and automatically pull the segments to feature positions in other frames. Adjustments are subject to joint limits and may use inverse kinematics. Occluded contour points are detected using object geometry and do not participate in feature tracking. Interactive adjustments are possible at any time in the process, allowing extraction of complicated movements regardless of background, camera movement, or feature clarity.     

Deformation identification and estimation of one-dimensional objects by vision sensors

Using vision sensors to study object deformation is introduced in this article. The vision sensor is used to identify the deformation characteristic and estimate the deformation behavior of flexible beams. First, a nonlinear equation describing the deformation of the beam is given. Then, cubic spline functions are used to approximate the contour of the deformed beam after the beam is detected by the vision sensor. Using the cubic spline approximation and Maclaurin's series expansion, a procedure to determine the deformation characteristic is formulated. To estimate the deformation behavior, numerical differential method to solve the nonlinear equation is used. The error of this numerical method can be controlled with the aid of the vision sensor. Finally, experimental results are given to verify that the proposed methods are effective.     

Robust adaptive-scale parametric model estimation for computer vision

Robust model fitting essentially requires the application of two estimators. The first is an estimator for the values of the model parameters. The second is an estimator for the scale of the noise in the (inlier) data. Indeed, we propose two novel robust techniques: the two-step scale estimator (TSSE) and the adaptive scale sample consensus (ASSC) estimator. TSSE applies nonparametric density estimation and density gradient estimation techniques, to robustly estimate the scale of the inliers. The ASSC estimator combines random sample consensus (RANSAC) and TSSE, using a modified objective function that depends upon both the number of inliers and the corresponding scale. ASSC is very robust to discontinuous signals and data with multiple structures, being able to tolerate more than 80 percent outliers. The main advantage of ASSC over RANSAC is that prior knowledge about the scale of inliers is not needed. ASSC can simultaneously estimate the parameters of a model and the scale of the inliers belonging to that model. Experiments on synthetic data show that ASSC has better robustness to heavily corrupted data than least median squares (LMedS), residual consensus (RESC), and adaptive least Kth order squares (ALKS). We also apply ASSC to two fundamental computer vision tasks: range image segmentation and robust fundamental matrix estimation. Experiments show very promising results.     

Real-time marker prediction and CoR estimation in optical motion capture

Optical motion capture systems suffer from marker occlusions resulting in loss of useful information. This paper addresses the problem of real-time joint localisation of legged skeletons in the presence of such missing data. The data is assumed to be labelled 3d marker positions from a motion capture system. An integrated framework is presented which predicts the occluded marker positions using a Variable Turn Model within an Unscented Kalman filter. Inferred information from neighbouring markers is used as observation states; these constraints are efficient, simple, and real-time implementable. This work also takes advantage of the common case that missing markers are still visible to a single camera, by combining predictions with under-determined positions, resulting in more accurate predictions. An Inverse Kinematics technique is then applied ensuring that the bone lengths remain constant over time; the system can thereby maintain a continuous data-flow. The marker and Centre of Rotation (CoR) positions can be calculated with high accuracy even in cases where markers are occluded for a long period of time. Our methodology is tested against some of the most popular methods for marker prediction and the results confirm that our approach outperforms these methods in estimating both marker and CoR positions.     

A highly robust estimator through partially likelihood functionmodeling and its application in computer vision

The authors present a highly robust estimator, known as the model fitting (MF) estimator for general regression. They explain that high robustness becomes possible through partially but completely modeling the unknown log likelihood function. The partial modeling takes place by taking the Bayesian statistical decision rule and a number of important heuristics into consideration while maximizing the log likelihood function. Applications include the automatic selection of multiple thresholds, single rigid motion estimation or multiple rigid motion segmentation, and estimation from two perspective views. It is believed that the proposed MF estimator will aid in solving many robust estimation problems that demand an estimator that is either highly robust or capable of handling contaminated Gaussian mixture models     

Real-time motion estimation for image and video processing applications

This contribution focuses on different topics that are covered by the special issue titled “Real-Time Motion Estimation for image and video processing applications” and which incorporate GPUS, FPGAs, VLSI systems, DSPs, and Multicores, among other platforms. The guest editors have solicited original contributions, which address a wide range of theoretical and practical issues related to high-performance motion estimation image processing including, but not limited to: real-time matching motion estimation systems, real-time energy-based motion estimation systems, gradient-based motion estimation systems, optical flow estimation systems, color motion estimation systems, multi-scale motion estimation systems, optical flow and motion estimation systems, analysis or comparison of specialized architectures for motion estimation systems and real-world applications.     

Real-time 3D motion capture by monocular vision and virtual rendering

Networked 3D virtual environments allow multiple users to interact over the Internet by means of avatars and to get some feeling of a virtual telepresence. However, avatar control may be tedious. 3D sensors for motion capture systems based on 3D sensors have reached the consumer market, but webcams remain more widespread and cheaper. This work aims at animating a user’s avatar by real-time motion capture using a personal computer and a plain webcam. In a classical model-based approach, we register a 3D articulated upper-body model onto video sequences and propose a number of heuristics to accelerate particle filtering while robustly tracking user motion. Describing the body pose using wrists 3D positions rather than joint angles allows efficient handling of depth ambiguities for probabilistic tracking. We demonstrate experimentally the robustness of our 3D body tracking by real-time monocular vision, even in the case of partial occlusions and motion in the depth direction.     

Real-time Unsupervised Segmentation of human whole-body motion and its application to humanoid robot acquisition of motion symbols

An interactive loop between motion recognition and motion generation is a fundamental mechanism for humans and humanoid robots. We have been developing an intelligent framework for motion recognition and generation based on symbolizing motion primitives. The motion primitives are encoded into Hidden Markov Models (HMMs), which we call “motion symbols”. However, to determine the motion primitives to use as training data for the HMMs, this framework requires a manual segmentation of human motions. Essentially, a humanoid robot is expected to participate in daily life and must learn many motion symbols to adapt to various situations. For this use, manual segmentation is cumbersome and impractical for humanoid robots. In this study, we propose a novel approach to segmentation, the Real-time Unsupervised Segmentation (RUS) method, which comprises three phases. In the first phase, short human movements are encoded into feature HMMs. Seamless human motion can be converted to a sequence of these feature HMMs. In the second phase, the causality between the feature HMMs is extracted. The causality data make it possible to predict movement from observation. In the third phase, movements having a large prediction uncertainty are designated as the boundaries of motion primitives. In this way, human whole-body motion can be segmented into a sequence of motion primitives. This paper also describes an application of RUS to AUtonomous Symbolization of motion primitives (AUS). Each derived motion primitive is classified into an HMM for a motion symbol, and parameters of the HMMs are optimized by using the motion primitives as training data in competitive learning. The HMMs are gradually optimized in such a way that the HMMs can abstract similar motion primitives. We tested the RUS and AUS frameworks on captured human whole-body motions and demonstrated the validity of the proposed framework.     

Enhanced motion parameters estimation for an active vision system

In this paper, we present an enhanced approach for estimating 3D motion parameters from 2D motion vector fields. The proposed method achieves valuable reduction in computational time and shows high robustness against noise in the input data. The output of the algorithm is part in a multiobject segmentation approach implemented in an active vision system. Hence, the improvement in the motion parameters estimation process leads to speed-up in the overall segmentation process. The text was submitted by the authors in English. Mohamed Shafik obtained his B.Sc. in mechanical engineering at the University of Banha. In 2004 he earned an Information Technology Diploma in Mechatronics from the Information Technology Institute (ITI). In 2006 he obtained his M.Eng. in applied mechatronics at the University of Paderborn. Since 2006, he is a PhD student and a scientific assistant in the GET Lab. His research interests focus on robotic vision, neural networks, and mechatronic systems. Baerbel Mertsching studied electrical engineering and obtained her PhD at the University of Paderborn. Between 1994 and 2003, she was professor of computer science at the University of Hamburg. In 2003 she returned to the University of Paderborn where she is now professor of electrical engineering and director of the GET Lab. Her research interests focus on cognitive systems engineering, especially active vision systems, and microelectronics for image and speech processing. She has been a member of a variety of scientific councils and editorial boards and is author of more than 120 scientific publications.     

Using computer vision to simulate the motion of virtual agents

In this paper, we propose a new model to simulate the movement of virtual humans based on trajectories captured automatically from filmed video sequences. These trajectories are grouped into similar classes using an unsupervised clustering algorithm, and an extrapolated velocity field is generated for each class. A physically‐based simulator is then used to animate virtual humans, aiming to reproduce the trajectories fed to the algorithm and at the same time avoiding collisions with other agents. The proposed approach provides an automatic way to reproduce the motion of real people in a virtual environment, allowing the user to change the number of simulated agents while keeping the same goals observed in the filmed video. Copyright © 2007 John Wiley & Sons, Ltd.     

视觉匀度分析的改进梯度指数法及其应用

以纸张为研究对像，利用计算机视觉方法，在分析了标准匀度检测方法的基础上，采用类金字塔结构方差给出了改进的梯度指数纹理分析算法，它既体现了微观匀度特性，又体现了宏观匀度特性．实验结果表明，该方法对于分析纸张匀度是可行的，并具有较高的鲁棒性能．     

Quadtree-based genetic algorithm and its applications to computer vision

Many computer vision problems can be formulated as optimization problems. Presented in this paper is a new framework based on the quadtree-based genetic algorithm that can be applied to solve many of these problems. The proposed algorithm incorporates the quadtree structure into the conventional genetic algorithm. The solutions of image-related problems are encoded through encoding the corresponding quadtrees, and therefore, the 2D locality within a solution can be preserved. Examples addressed using the proposed framework include image segmentation, stereo vision, and motion estimation. In all cases, encouraging results are obtained.     

Algorithmic differentiation: application to variational problems in computer vision

Many vision problems can be formulated as minimization of appropriate energy functionals. These energy functionals are usually minimized, based on the calculus of variations (Euler-Lagrange equation). Once the Euler-Lagrange equation has been determined, it needs to be discretized in order to implement it on a digital computer. This is not a trivial task and, is moreover, error- prone. In this paper, we propose a flexible alternative. We discretize the energy functional and, subsequently, apply the mathematical concept of algorithmic differentiation to directly derive algorithms that implement the energy functional's derivatives. This approach has several advantages: First, the computed derivatives are exact with respect to the implementation of the energy functional. Second, it is basically straightforward to compute second-order derivatives and, thus, the Hessian matrix of the energy functional. Third, algorithmic differentiation is a process which can be automated. We demonstrate this novel approach on three representative vision problems (namely, denoising, segmentation, and stereo) and show that state-of-the-art results are obtained with little effort.     

Residual motion estimation with point targets and its application to airborne repeat-pass SAR interferometry

Due to the lack of accuracy in the navigation system, airborne repeat-pass synthetic aperture radar (SAR) interferometry (InSAR) suffers residual motion errors (RMEs). Previously, we proposed the multisquint technique with point targets (MTPT) to estimate the errors, but its implementation needs improvement and its accuracy has not been assessed by real airborne repeat-pass InSAR data. In this article, the modified MTPT is introduced first. The modification is mainly embodied in two aspects: automatic target selection and noise removing. Because the multisquint (MS) technique is also capable of estimating the RMEs in SAR interferogram and its accuracy has been verified to be high, here it is used as a comparison. In addition, from the viewpoint of MTPT, MS can be understood from another perspective. Using real X-band airborne repeat-pass SAR data, the performance of MTPT is evaluated. The experiment shows that MTPT is able to achieve high accuracy when a large number of point targets are distributed in the observed scene. The limitations of MTPT are also discussed at the end of this article.     

Construction of confidence regions for motion trajectories of objects in computer vision problems

The problem of finding an object lost for several number of frames is considered within the framework of motion analysis problem. A linear model of objects motion is designed on a finite time interval. The hypotheses on independence and normality of observation errors are tested. The Gauss-Markov estimate is found for motion parameters. Based on available measurements, hyperbolic confidence tubes are constructed for each of the coordinates of the center of mass of the moving object. The technique for derivation of the equation of an ellipse of dispersion for the two-dimensional Gauss vector of a general form is demonstrated. The form of the ellipse of dispersion for the addressed problem is found. The results of testing the algorithm for open bases of video sequences PETS and ETISEO are presented.