Providing multiple views for objects

The need for multiple views of an object is often encountered in software practice. This paper presents our experience in addressing this need under a software architecture known as the law‐governed architecture. We introduce the notion of a surrogate object which allows an object to appear different and behave differently when used from different parts of a system. This concept requires some minor modifications to the classical inheritance‐based object‐oriented systems, mostly involving a judicious use of delegation. A concrete implementation of surrogates under the law‐governed architecture is described and some applications of surrogates are briefly discussed. Copyright © 2000 John Wiley & Sons, Ltd.     

无重叠视域的多摄像机之间的目标匹配

在无重叠视域的多摄像机监控中,由于不同摄像机的视域差别和视域分离,同一运动目标在不同的视域中的成像可能会非常不同,因此在这种情况下对运动目标进行匹配是一项具有挑战性的工作。提出了一种可以容忍光照的不同,在无重叠视域的多摄像机下进行目标匹配的方法。该方法经过初始聚类和K-means聚类对目标进行主颜色谱的提取,利用EMKM算法改善K-means对初始中心点的依赖性,把提取出来的主颜色谱直方图作为目标的特征,然后利用特征相似度测量来判定任意两个物体之间是否匹配;当无法对某些物体进行准确匹配时,再利用SIFT特征进行下一步匹配。该方法也可以用于有重叠视域的多摄像机目标匹配中,通过与其他匹配方法相结合,提高匹配的准确度。实验结果证实了该方法具有较高的准确度。     

Towards automated visual surveillance using gait for identity recognition and tracking across multiple non-intersecting cameras

Despite the fact that personal privacy has become a major concern, surveillance technology is now becoming ubiquitous in modern society. This is mainly due to the increasing number of crimes as well as the essential necessity to provide secure and safer environment. Recent research studies have confirmed now the possibility of recognizing people by the way they walk i.e. gait. The aim of this research study is to investigate the use of gait for people detection as well as identification across different cameras. We present a new approach for people tracking and identification between different non-intersecting un-calibrated stationary cameras based on gait analysis. A vision-based markerless extraction method is being deployed for the derivation of gait kinematics as well as anthropometric measurements in order to produce a gait signature. The novelty of our approach is motivated by the recent research in biometrics and forensic analysis using gait. The experimental results affirmed the robustness of our approach to successfully detect walking people as well as its potency to extract gait features for different camera viewpoints achieving an identity recognition rate of 73.6 % processed for 2270 video sequences. Furthermore, experimental results confirmed the potential of the proposed method for identity tracking in real surveillance systems to recognize walking individuals across different views with an average recognition rate of 92.5 % for cross-camera matching for two different non-overlapping views.     

Structure recovery with multiple cameras from scaled orthographicand perspective views

This paper presents a novel framework for Euclidean structure recovery utilizing a scaled orthographic view and perspective views simultaneously. A scaled orthographic view is introduced in order to automatically obtain camera parameters such as camera positions, orientation, and focal length. Scaled orthographic properties enable all camera parameters to be calculated implicitly and perspective properties enable a Euclidean structure to be recovered. The method can recover a Euclidean structure with at least seven point correspondences across a scaled orthographic view and perspective views. Experimental results for both computed and natural images verify that the method recovers structure with sufficient accuracy to demonstrate potential utility. The proposed method can be applied to an interface for 3D modeling, recognition and tracking     

合成数据驱动目标姿态追踪的快速收敛网络

目的 受遮挡与累积误差因素的影响,现有目标6维（6 dimensions, 6D）姿态实时追踪方法在复杂场景中表现不佳。为此,提出了一种高鲁棒性的刚体目标6D姿态实时追踪网络。方法 在网络的整体设计上,将当前帧彩色图像和深度图像（red green blue-depth map,RGB-D）与前一帧姿态估计结果经升维残差采样滤波和特征编码处理获得姿态差异,与前一帧姿态估计结果共同计算目标当前的6D姿态;在残差采样滤波模块的设计中,采用自门控swish(searching for activation functions)激活函数保留目标细节特征,提高目标姿态追踪的准确性;在特征聚合模块的设计中,将提取的特征分解为水平与垂直两个方向分量,分别从时间和空间上捕获长程依赖并保留位置信息,生成一组具有位置与时间感知的互补特征图,加强目标特征提取能力,从而加速网络收敛。结果 实验选用YCBVideo(Yale-CMU-Berkeley-video)和YCBInEoAT(Yale-CMU-Berkeley in end-of-arm-tooling)数据集。实验结果表明,本文方法追踪速度达到90....     

Photometric visual servoing for omnidirectional cameras

2D visual servoing consists in using data provided by a vision sensor for controlling the motions of a dynamic system. Most of visual servoing approaches has relied on the geometric features that have to be tracked and matched in the image acquired by the camera. Recent works have highlighted the interest of taking into account the photometric information of the entire image. This approach was tackled with images of perspective cameras. We propose, in this paper, to extend this technique to central cameras. This generalization allows to apply this kind of method to catadioptric cameras and wide field of view cameras. Several experiments have been successfully done with a fisheye camera in order to control a 6 degrees of freedom robot and with a catadioptric camera for a mobile robot navigation task.     

Tracking the soccer ball using multiple fixed cameras

This paper demonstrates innovative techniques for estimating the trajectory of a soccer ball from multiple fixed cameras. Since the ball is nearly always moving and frequently occluded, its size and shape appearance varies over time and between cameras. Knowledge about the soccer domain is utilized and expressed in terms of field, object and motion models to distinguish the ball from other movements in the tracking and matching processes. Using ground plane velocity, longevity, normalized size and color features, each of the tracks obtained from a Kalman filter is assigned with a likelihood measure that represents the ball. This measure is further refined by reasoning through occlusions and back-tracking in the track history. This can be demonstrated to improve the accuracy and continuity of the results. Finally, a simple 3D trajectory model is presented, and the estimated 3D ball positions are fed back to constrain the 2D processing for more efficient and robust detection and tracking. Experimental results with quantitative evaluations from several long sequences are reported.     

Real-time panorama video system using networked multiple cameras

Although panorama creation has been studied extensively, previous works still have limitations because they depend on customized hardware or post processing, resulting in high production cost and complexity. We propose and implement real-time panorama video system by using off-the-shelf embedded boards and camera modules. For real-time, we developed the performance improving methods such as reducing the homography computation load and employing multi thread structure. We conducted a series of experiments to investigate how different blending combinations and the number of threads affect the performance and image quality. We also analyzed the time consumption of threads and stitching steps.     

Behavior monitoring for assistive environments using multiple views

This work presents an approach to behavior understanding using multiple cameras. This approach is appropriate for monitoring people in an assistive environment for the purpose of issuing alerts in cases of abnormal behavior. The output of multiple classifiers is used to model and extract abnormal behavior from both the target trajectory and the target short-term activity (i.e., walking, running, abrupt motion, etc.). Spatial information is obtained after an offline camera registration using homography information. The proposed approach is verified experimentally in an indoor environment. The experiments are performed with a single moving target; however, the method can be generalized to multiple moving targets, which may occlude each other, due to the use of multiple cameras.     

Multiple cameras audio visual speech recognition using active appearance model visual features in car environment

Consideration of visual speech features along with traditional acoustic features have shown decent performance in uncontrolled auditory environment. However, most of the existing audio-visual speech recognition (AVSR) systems have been developed in the laboratory conditions and rarely addressed the visual domain problems. This paper presents an active appearance model (AAM) based multiple-camera AVSR experiment. The shape and appearance information are extracted from jaw and lip region to enhance the performance in vehicle environments. At first, a series of visual speech recognition (VSR) experiments are carried out to study the impact of each camera on multi-stream VSR. Four cameras in car audio-visual corpus is used to perform the experiments. The individual camera stream is fused to have four-stream synchronous hidden Markov model visual speech recognizer. Finally, optimum four-stream VSR is combined with single stream acoustic HMM to build five-stream AVSR. The dual modality AVSR system shows more robustness compared to acoustic speech recognizer across all driving conditions.     

EGT for multiple view geometry and visual servoing: robotics vision with pinhole and panoramic cameras

The Epipolar Geometry Toolbox (EGT) for MATLAB is a software package targeted to research and education in computer vision and robotics visual servoing. It provides the user with a wide set of functions for designing multicamera systems for both pinhole and panoramic cameras. Functions include camera placement and visualization, computation, and estimation of epipolar geometry entities. The compatibility of EGT with the Robotics Toolbox enables users to address general vision-based control issues. Two applications of EGT to visual servoing tasks are examined in this article. Several epipolar geometry estimation algorithms have been implemented.     

Projected predictive Energy-Bounding Approach for multiple degree-of-freedom haptic teleoperation

This paper presents a geometrically and dynamically transparent projected predictive Energy-Bounding Approach (EBA) for multiple degree-of-freedom (m-DOF) haptic teleoperation with time delays. The straightforward extension of a single-DOF teleoperation to m-DOF teleoperation suffers from a bad force distortion problem that becomes worse with the increase in communication time delays between master and slave sites. Due to this, geometry perception of remote m-DOF objects at master site is severely deteriorated. To solve this problem, a projected predictive m-DOF EBA is proposed by combining the m-DOF Predictive EBA and the projection method that can compensate the force distortion problem. The proposed approach also includes steps to get stably the information about the contact locations (geometry) of m-DOF remote objects without geometry detection sensors. In order to validate the proposed approach, some experiments performing surface contour-following are conducted using two Phantoms: (i) 2-DOF object (bobbin), and (ii) 3-DOF semi-spherical object in the presence of large time delays.     

Flexible three-dimensional modeling of plants using low- resolution cameras and visual odometry

The three-dimensional reconstruction of plants using computer vision methods is a promising alternative to non-destructive metrology in plant phenotyping. However, diversity in plants form and size, different surrounding environments (laboratory, greenhouse or field), and occlusions impose challenging issues. We propose the use of state-of-the-art methods for visual odometry to accurately recover camera pose and preliminary three-dimensional models on image acquisition time. Specimens of maize and sunflower were imaged using a single free-moving camera and a software tool with visual odometry capabilities. Multiple-view stereo was employed to produce dense point clouds sampling the plant surfaces. The produced three-dimensional models are accurate snapshots of the shoot state and plant measurements can be recovered in a non-invasive way. The results show a free-moving low-resolution camera is able to handle occlusions and variations in plant size and form, allowing the reconstruction of different species, and specimens in different stages of development. It is also a cheap and flexible method, suitable for different phenotyping needs. Plant traits were computed from the point clouds and compared to manually measured reference, showing millimeter accuracy. All data, including images, camera calibration, pose, and three-dimensional models are publicly available.     

A clustering method for concurrent photos obtained from multiple cameras using max-flow network model

With the popularization of digital cameras, the use of several cameras by group photographers at the same event is becoming common. Photographers can share their contents and even take pictures of each other. So it is becoming important to manage concurrent photos from multiple cameras in order to classify many accumulated photos into proper clusters. In this paper, we propose a novel photo clustering method based on the max-flow network algorithm, and we visualize a network graph for cluster verification. To apply our algorithm, input concurrent photos are used to create an edge-weighted graph structure. In order to transform the photo clustering problem into a graph partition one, first we need to construct an Augmented Concurrent photo Graph (ACG) and then rewrite our original problem in terms of the graph partition one using the min-cut max-flow network model. The previous methods dealt with photo clustering as a 1-D problem using a linear partition. But we consider clustering for concurrent group photos as a 2-D partition based on other users’ photo contents. Each photo is used to create a node and similarities between photos are used to create the edge weights (capacities) of the network. We partition the network into two subgraphs according to the min-cut, which represents the weakest edge connections between the photos. Using repeated graph partitions for each subgraph (sub-network), we can obtain suitable subgraphs corresponding to photo clusters. The graph construction or partition can be adjusted according to user preferences in order to obtain the intended results.     

多摄像机图像间的稳定切换控制

为获得连续动态的图像雅可比矩阵,分析了融合方式和传统图像直接切换方法的缺陷,提出了一种基于融合的多图像稳定切换算法,算法采用动态加权融合策略。在移动机器人位置未知和无标定多摄像机的情况下,仿真和实验结果表明:该算法比传统方法具有更高的适应能力,图像切换过程的稳定性得到了很大提高。     

PCA-based 3D shape reconstruction of human foot using multiple viewpoint cameras

This paper describes a multiple camera-based method to reconstruct the 3D shape of a human foot. From a foot database, an initial 3D model of the foot represented by a cloud of points is built. The shape parameters, which can characterize more than 92% of a foot, are defined by using the principal component analysis method. Then, using ＂active shape models＂, the initial 3D model is adapted to the real foot captured in multiple images by applying some constraints （edge points＇ distance and color variance）. We insist here on the experiment part where we demonstrate the efficiency of the proposed method on a plastic foot model, and also on real human feet with various shapes. We propose and compare different ways of texturing the foot which is needed for reconstruction. We present an experiment performed on the plastic foot model and on human feet and propose two different ways to improve the final 3D shapers accuracy according to the previous experiments＇ results. The first improvement proposed is the densification of the cloud of points used to represent the initial model and the foot database. The second improvement concerns the projected patterns used to texture the foot. We conclude by showing the obtained results for a human foot with the average computed shape error being only 1.06 mm.     

Spatio‐temporal geometry fusion for multiple hybrid cameras using moving least squares surfaces

Multi‐view reconstruction aims at computing the geometry of a scene observed by a set of cameras. Accurate 3D reconstruction of dynamic scenes is a key component for a large variety of applications, ranging from special effects to telepresence and medical imaging. In this paper we propose a method based on Moving Least Squares surfaces which robustly and efficiently reconstructs dynamic scenes captured by a calibrated set of hybrid color+depth cameras. Our reconstruction provides spatio‐temporal consistency and seamlessly fuses color and geometric information. We illustrate our approach on a variety of real sequences and demonstrate that it favorably compares to state‐of‐the‐art methods.