Monocular scene flow estimation via variational method

Scene flow provides the 3D motion field of point clouds, which correspond to image pixels. Current algorithms usually need complex stereo calibration before estimating flow, which has strong restrictions on the position of the camera. This paper proposes a monocular camera scene flow estimation algorithm. Firstly, an energy functional is constructed, where three important assumptions are turned into data terms derivation: a brightness constancy assumption, a gradient constancy assumption, and a short time object velocity constancy assumption. Two smooth operators are used as regularization terms. Then, an occluded map computation algorithm is used to ensure estimating scene flow only on un-occluded points. After that, the energy functional is solved with a coarse-to-fine variational equation on Gaussian pyramid, which can prevent the iteration from converging to a local minimum value. The experiment results show that the algorithm can use three sequential frames at least to get scene flow in world coordinate, without optical flow or disparity inputting.     

Three-dimensional flow characterization using vector pattern matching

This paper describes a novel method for regional characterization of three-dimensional vector fields using a pattern matching approach. Given a three-dimensional vector field, the goal is to automatically locate, identify, and visualize a selected set of classes of structures or features. Rather than analytically defining the properties that must be fulfilled in a region in order to be classified as a specific structure, a set of idealized patterns for each structure type is constructed. Similarity to these patterns is then defined and calculated. Examples of structures of interest include vortices, swirling flow, diverging or converging flow, and parallel flow. Both medical and aerodynamic applications are presented in this paper.     

Three-dimensional laminar incompressible flow in straight polar ducts

The flow in the entrance region of long ducts of rectangular and polar cross sections is studied using the three-dimensional parabolized Navier-Stokes equations, together with the energy equation, for an incompressible viscous fluid. Numerical solutions are obtained using an alternating-direction implicit technique for the parabolized equations which are elliptic in the cross-plane of the duct, and the calculations march forward in the axial direction. Computations are also made for polar ducts with a rotating outer-radial wall so as to provide meaningful results for a model problem comprising the simplest form of a turbomachinery cascade. The study of this model problem serves to analyze some of the important features of three-dimensional internal viscous flows. Also of interest is the rapidly converging solution of the Neumann problem for the cross-plane variation of pressure for polar ducts.     

On 3-D scene flow and structure recovery from multiview image sequences

Two novel systems computing dense three-dimensional (3-D) scene flow and structure from multiview image sequences are described in this paper. We do not assume rigidity of the scene motion, thus allowing for nonrigid motion in the scene. The first system, integrated model-based system (IMS), assumes that each small local image region is undergoing 3-D affine motion. Non-linear motion model fitting based on both optical flow constraints and stereo constraints is then carried out on each local region in order to simultaneously estimate 3-D motion correspondences and structure. The second system is based on extended gradient-based system (EGS), a natural extension of two-dimensional (2-D) optical flow computation. In this method, a new hierarchical rule-based stereo matching algorithm is first developed to estimate the initial disparity map. Different available constraints under a multiview camera setup are further investigated and utilized in the proposed motion estimation. We use image segmentation information to adopt and maintain the motion and depth discontinuities. Within the framework for EGS, we present two different formulations for 3-D scene flow and structure computation. One formulation assumes that initial disparity map is accurate, while the other does not. Experimental results on both synthetic and real imagery demonstrate the effectiveness of our 3-D motion and structure recovery schemes. Empirical comparison between IMS and EGS is also reported.     

Three-dimensional compressible-incompressible turbulent flow simulation using a pressure-based algorithm

In this work, we extend a finite-volume pressure-based incompressible algorithm to solve three-dimensional compressible and incompressible turbulent flow regimes. To achieve a hybrid algorithm capable of solving either compressible or incompressible flows, the mass flux components instead of the primitive velocity components are chosen as the primary dependent variables in a SIMPLE-based algorithm. This choice warrants to reduce the nonlinearities arose in treating the system of conservative equations. The use of a new Favre-averaging like technique plays a key role to render this benefit. The developed formulations indicate that there is less demand to interpolate the fluxes at the cell faces, which is definitely a merit. To impose the hyperbolic behavior in compressible flow regimes, we introduce an artificial hyperbolicity in pressure correction equation. We choose k-ω turbulence model and incorporate the compressibility effect as a correction. It is shown that the above considerations grant to achieve a robust algorithm with great capabilities in solving both flow regimes with a reasonable range of Mach number applications. To evaluate the ability of the new pressure-based algorithm, three test cases are targeted. They are incompressible backward-facing step problem, compressible flow over a wide range of open to closed cavities, and compressible turbulent flow in a square duct. The current results indicate that there are reliable agreements with those of experiments and other numerical solutions in the entire range of investigation.     

Revised definition of optical flow: integration of radiometric andgeometric cues for dynamic scene analysis

Optical flow has been commonly defined as the apparent motion of image brightness patterns in an image sequence. In this paper, we propose a revised definition to overcome shortcomings in interpreting optical flow merely as a geometric transformation field. The new definition is a complete representation of geometric and radiometric variations in dynamic imagery. We argue that this is more consistent with the common interpretation of optical flow induced by various scene events. This leads to a general framework for the investigation of problems in dynamic scene analysis, based on the integration and unified treatment of both geometric and radiometric cues in time-varying imagery. We discuss selected models, including the generalized dynamic image model, for the estimation of optical flow. We show how various 3D scene information are encoded in, and thus may be extracted from, the geometric and radiometric components of optical flow. We provide selected examples based on experiments with real images     

Three-dimensional computer analysis of flow and combustion in automotive internal combustion engines

The paper presents a method for calculating the three-dimensional flow-fields in reciprocating internal combustion engines, as a function of space and time, throughout the complete four-stroke cycle. The method is based on a computational procedure which solves the governing elliptic partial-differential equations on a finite-difference grid which expands and contracts with the motion of the piston, using a fully-implicit, iterative, finite-difference scheme. Results are presented for typical engines, under engine-motoring and spark-ignited conditions. It is concluded that careful physical experiments should now be conducted in parallel with the computer experiments to validate the predictions, before the model can be used directly in assisting engine design; but also that it is now practical, for the first time, to perform fully three-dimensional calculations of the flow within the engine.     

Immersive audio-visual scene reproduction using semantic scene reconstruction from 360 cameras

Virtual Reality - As personalised immersive display systems have been intensely explored in virtual reality (VR), plausible 3D audio corresponding to the visual content is required to provide more...     

基于语义的风景图像检索

基于语义的图像检索就是要对图像正确分类,提供对图像内容的理解.基于粗糙集方法对基于语义的风景图像检索进行研究,突破了传统语义模板方法缺乏变化的不足,对图像外观变化具有一定的抗噪能力.仿真实验结果表明,对日出日落、瀑布、山脉、花草4类风景图像的分类准确率达到85%以上.     

Summarizing high-level scene behavior

We present several novel techniques to summarize the high-level behavior in surveillance video. Our proposed methods can employ either optical flow or trajectories as input, and incorporate spatial and temporal information together, which improve upon existing approaches for summarization. To begin, we extract common pathway regions by performing graph-based clustering on similarity matrices describing the relationships between location/orientation states. We then employ the activities along the pathway regions to extract the aggregate behavioral patterns throughout scenes. We show how our summarization methods can be applied to detect anomalies, retrieve video clips of interest, and generate adaptive-speed summary videos. We examine our approaches on multiple complex urban scenes and present experimental results.     

A change of scene

Films and television share a unique language for visual storytelling. One feature of this language lets us move through space and time in ways that are impossible in real life. For example, in a stage play, there needs to be some continuity of time and place: if a character is in the kitchen at one moment, he or she can't be behind the wheel of a speeding car in the next moment. But this kind of abrupt transition happens all the time in films. The simplest and most familiar way to get from one scene to another is via the cut. To create a cut, you can literally take two pieces of film, cut them at frame boundaries, and tape them together. The cut belongs to a general class of transitions: visual effects that form the boundary between different scenes. Let's say that we're watching scene A and we want to go to scene B. If the transition takes many frames, then at each step along the transition we conceptually take the appropriate frames from A and B and combine them to create a new image. The cut is the simplest transition. It takes zero frames to execute and simply stops showing A and starts showing B. Perhaps the next most familiar transition is the dissolve (also called the cross-dissolve), where we smoothly blend from A to B. A popular variation on the dissolve is the fade to black, which is just a dissolve where the B scene is a black frame. Many video-editing programs ship with a collection of ready-to-use transitions. The author discusses the field and then presents some transitions which he has developed.     

Dynamic scene occlusion culling

Large, complex 3D scenes are best rendered in an output-sensitive way, i.e., in time largely independent of the entire scene model's complexity. Occlusion culling is one of the key techniques for output-sensitive rendering. We generalize existing occlusion culling algorithms, intended for static scenes, to handle dynamic scenes having numerous moving objects. The data structure used by an occlusion culling method is updated to reflect the objects' possible positions. To avoid updating the structure for every dynamic object at each frame, a temporal bounding volume (TBV) is created for each occluded dynamic object, using some known constraints on the object's motion. The TBV is inserted into the structure instead of the object. Subsequently, the object is ignored as long as the TBV is occluded and guaranteed to contain the object. The generalized algorithms' rendering time is linearly affected only by the scene's visible parts, not by hidden parts or by occluded dynamic objects. Our techniques also save communications in distributed graphic systems, e.g., multiuser virtual environments, by eliminating update messages for hidden dynamic objects. We demonstrate the adaptation of two occlusion culling algorithms to dynamic scenes: hierarchical Z-buffering and BSP tree projection     

Learning multi-label scene classification

In classic pattern recognition problems, classes are mutually exclusive by definition. Classification errors occur when the classes overlap in the feature space. We examine a different situation, occurring when the classes are, by definition, not mutually exclusive. Such problems arise in semantic scene and document classification and in medical diagnosis. We present a framework to handle such problems and apply it to the problem of semantic scene classification, where a natural scene may contain multiple objects such that the scene can be described by multiple class labels (e.g., a field scene with a mountain in the background). Such a problem poses challenges to the classic pattern recognition paradigm and demands a different treatment. We discuss approaches for training and testing in this scenario and introduce new metrics for evaluating individual examples, class recall and precision, and overall accuracy. Experiments show that our methods are suitable for scene classification; furthermore, our work appears to generalize to other classification problems of the same nature.