Segmenting and tracking fluorescent cells in dynamic 3-D microscopy with coupled active surfaces.

Cell migrations and deformations play essential roles in biological processes, such as parasite invasion, immune response, embryonic development, and cancer. We describe a fully automatic segmentation and tracking method designed to enable quantitative analyses of cellular shape and motion from dynamic three-dimensional microscopy data. The method uses multiple active surfaces with or without edges, coupled by a penalty for overlaps, and a volume conservation constraint that improves outlining of cell/cell boundaries. Its main advantages are robustness to low signal-to-noise ratios and the ability to handle multiple cells that may touch, divide, enter, or leave the observation volume. We give quantitative validation results based on synthetic images and show two examples of applications to real biological data.     

Tracking of migrating cells under phase-contrast video microscopy with combined mean-shift processes

In this paper, we propose a combination of mean-shift-based tracking processes to establish migrating cell trajectories through in vitro phase-contrast video microscopy. After a recapitulation on how the mean-shift algorithm permits efficient object tracking we describe the proposed extension and apply it to the in vitro cell tracking problem. In this application, the cells are unmarked (i.e., no fluorescent probe is used) and are observed under classical phase-contrast microscopy. By introducing an adaptive combination of several kernels, we address several problems such as variations in size and shape of the tracked objects (e.g., those occurring in the case of cell membrane extensions), the presence of incomplete (or noncontrasted) object boundaries, partially overlapping objects and object splitting (in the case of cell divisions or mitoses). Comparing the tracking results automatically obtained to those generated manually by a human expert, we tested the stability of the different algorithm parameters and their effects on the tracking results. We also show how the method is resistant to a decrease in image resolution and accidental defocusing (which may occur during long experiments, e.g., dozens of hours). Finally, we applied our methodology on cancer cell tracking and showed that cytochalasin-D significantly inhibits cell motility.     

3-D model-based vehicle tracking.

This paper aims at tracking vehicles from monocular intensity image sequences and presents an efficient and robust approach to three-dimensional (3-D) model-based vehicle tracking. Under the weak perspective assumption and the ground-plane constraint, the movements of model projection in the two-dimensional image plane can be decomposed into two motions: translation and rotation. They are the results of the corresponding movements of 3-D translation on the ground plane (GP) and rotation around the normal of the GP, which can be determined separately. A new metric based on point-to-line segment distance is proposed to evaluate the similarity between an image region and an instantiation of a 3-D vehicle model under a given pose. Based on this, we provide an efficient pose refinement method to refine the vehicle's pose parameters. An improved EKF is also proposed to track and to predict vehicle motion with a precise kinematics model. Experimental results with both indoor and outdoor data show that the algorithm obtains desirable performance even under severe occlusion and clutter.     

Three-dimensional structural changes in living hippocampal neurons imaged using magnetic AC mode atomic force microscopy

We developed the magnetic AC (MAC) mode atomic force microscopy (AFM) to image the 3D ultrastructure of living hippocampal neurons under physiological conditions. Initially, the soma, the dendrites and the growth cones of hippocampal neurons were imaged. The imaging force was adjusted to a small value for the long-term observation. The neural spines were damaged when the tip produced a large force; the spines regenerated after the force was reduced. Subsequently, we explored the relationship between structural changes in hippocampal neurons and Alzheimer's disease by employing the new imaging technique. Time-lapse image acquisition (10 min intervals) showed that the growth cone collapsed after the addition amyloid peptide fragment beta(25-35), which is thought to initiate Alzheimer's disease. In addition, we found substantial changes in mechanical properties and in the volume of individual growth cone. This study suggested that MAC mode AFM may be a powerful tool for observing long-term structural changes in living neural cells under physiological conditions.     

Multiple particle tracking in 3-D+t microscopy: method and application to the tracking of endocytosed quantum dots.

We propose a method to detect and track multiple moving biological spot-like particles showing different kinds of dynamics in image sequences acquired through multidimensional fluorescence microscopy. It enables the extraction and analysis of information such as number, position, speed, movement, and diffusion phases of, e.g., endosomal particles. The method consists of several stages. After a detection stage performed by a three-dimensional (3-D) undecimated wavelet transform, we compute, for each detected spot, several predictions of its future state in the next frame. This is accomplished thanks to an interacting multiple model (IMM) algorithm which includes several models corresponding to different biologically realistic movement types. Tracks are constructed, thereafter, by a data association algorithm based on the maximization of the likelihood of each IMM. The last stage consists of updating the IMM filters in order to compute final estimations for the present image and to improve predictions for the next image. The performances of the method are validated on synthetic image data and used to characterize the 3-D movement of endocytic vesicles containing quantum dots.     

Traffic accident prediction using 3-D model-based vehicle tracking

Intelligent visual surveillance for road vehicles is the key to developing autonomous intelligent traffic systems. Recently, traffic incident detection employing computer vision and image processing has attracted much attention. In this paper, a probabilistic model for predicting traffic accidents using three-dimensional (3-D) model-based vehicle tracking is proposed. Sample data including motion trajectories are first obtained by 3-D model-based vehicle tracking. A fuzzy self-organizing neural network algorithm is then applied to learn activity patterns from the sample trajectories. Finally, vehicle activity is predicted by locating and matching each partial trajectory with the learned activity patterns, and the occurrence probability of a traffic accident is determined. Experiments show the effectiveness of the proposed algorithms.     

Online relational tracking with camera motion suppression

To overcome challenges in multiple-object tracking (MOT) tasks, recent algorithms use interaction cues alongside motion and appearance features. These algorithms use graph neural networks or transformers to extract interaction features that lead to high computation costs. In this paper, a novel interaction cue based on geometric features is presented aiming to detect occlusion and reidentify lost targets with low computational costs. Moreover, in the majority of algorithms, camera motion is considered negligible, which is a strong assumption that is not always true and can lead to identity (ID) switching or mismatching of targets. In this paper, a method for measuring camera motion is presented that efficiently reduces its effect on tracking. The proposed algorithm is evaluated on MOT17 and MOT20 datasets and achieves state-of-the-art performance on MOT17 with comparable results on MOT20. The code is also publicly available.¹     

3-D model-based multiple-object video tracking for treatment room supervision

We present a method to monitor a patient and the equipment in a radiotherapy treatment room, by exploiting the information in the treatment plan, enriched with other elements such as visual, geometric, and "semantic" information. Using all these information items, and a generic model, a virtual environment of the scene is created, with maximum precision. The images resulting from video sequences with several cameras are also used to confront the filmed information on the scene and its numerical representation. The method is based on the features of the scene elements, and on a fuzzy formalism. The feasibility of the method is being quantitatively evaluated in the absence of treatment, to be further exploited in a module for external control by video in real conditions.     

3-D tracking and motion estimation using hierarchical Kalman filter

The authors present a novel approach to the problem of tracking and reconstructing articulated objects in 3-D space. The newly conceived computational process and its supporting data structure, the hierarchical Kalman filter (HKF) and the adaptive hierarchical structure (AHS). Allow the problem to be treated in a singlet unified framework. There are three novelties in the authors' formulation: reducing the 3-D tracking problem to 2-D tracking; incorporating the kinematic and the dynamic properties of object; and tracking nonrigid objects. To demonstrate the appropriateness of the proposed method, the authors present some of the experimental results on both synthetic and real images     

A robust 3-D IVUS transducer tracking using single-plane cineangiography

During an intravascular ultrasound (IVUS) intervention, a catheter with an ultrasound transducer is introduced in the body through a blood vessel, and then, pulled back to image a sequence of vessel cross sections. Unfortunately, there is no 3-D information about the position and orientation of these cross-section planes, which makes them less informative. To position the IVUS images in space, some researchers have proposed complex stereoscopic procedures relying on biplane angiography to get two X-ray image sequences of the IVUS transducer trajectory along the catheter. To simplify this procedure, we and others have elaborated algorithms to recover the transducer 3-D trajectory with only a single view X-ray image sequence. In this paper, we present an improved method that provides both automated 2-D and 3-D transducer tracking based on pullback speed as a priori information. The proposed algorithm is robust to erratic pullback speed and is more accurate than the previous single-plane 3-D tracking methods.     

Online Boosting tracking algorithm combined with occlusion sensing

Online Boosting tracking algorithm combined with occlusion sensing was presented.In this method,occlusion sensor was introduced to check the tracking results,and classifier updating strategy was adjusted depending on the occlusion checking results.By this way,the feature pool of the classifier can be kept pure,which will improve the tracking robustness under occlusion.Experimental results show that compared with traditional Boosting tracking algorithm,improved algorithm can solve the problem of occlusion very well.     

Multiview deblurring for 3-D images from light-sheet-based fluorescence microscopy

We propose an algorithm for 3-D multiview deblurring using spatially variant point spread functions (PSFs). The algorithm is applied to multiview reconstruction of volumetric microscopy images. It includes registration and estimation of the PSFs using irregularly placed point markers (beads). We formulate multiview deblurring as an energy minimization problem subject to L1-regularization. Optimization is based on the regularized Lucy-Richardson algorithm, which we extend to deal with our more general model. The model parameters are chosen in a profound way by optimizing them on a realistic training set. We quantitatively and qualitatively compare with existing methods and show that our method provides better signal-to-noise ratio and increases the resolution of the reconstructed images.     

Dead-reckoning sensor system and tracking algorithm for 3-D pipeline mapping

A dead-reckoning sensor system and a tracking algorithm for 3-D pipeline mapping are proposed for a tap water pipeline for which the diameter is small and the inner surface is rough due to pipe scales. The goals of this study are to overcome the performance limitations of small and low-grade sensors by combining various sensors with complementary functions and achieve robustness against a severe environment. A dead-reckoning sensor system consists of a small, low-cost micro electromechanical system inertial measurement unit (MEMS IMU) and an optical navigation sensor (used in laser mice). A tracking algorithm consists of a multi-rate extended Kalman filter (EKF) to fuse redundant and complementary data from the MEMS IMU and the optical navigation sensor and a geometry compensation method to reduce position estimation error using the end point of the pipeline. Two sets of experimental data have been obtained by driving a radio-controlled car equipped with the sensor system in a 3-D pipeline and on asphalt pavement. Our study can be used to estimate the path of a 3-D pipeline or mobile robots.     

Pattern tracking and 3-D motion reconstruction of a rigid body from a 2-D image sequence

This paper presents an integrated method to identify an object pattern from an image, and track its movement over a sequence of images. The sequence of images comes from a single perspective video source, which is capturing data from a precalibrated scene. This information is used to reconstruct the scene in three-dimension (3-D) within a virtual environment where a user can interact and manipulate the system. The steps that are performed include the following: i) Identify an object pattern from a two-dimensional perspective video source. The user outlines the region of interest (ROI) in the initial frame; the procedure builds a refined mask of the dominant object within the ROI using the morphological watershed algorithm. ii) The object pattern is tracked between frames using object matching within the mask provided by the previous and next frame, computing the motion parameters. iii) The identified object pattern is matched with a library of shapes to identify a corresponding 3-D object. iv) A virtual environment is created to reconstruct the scene in 3-D using the 3-D object and the motion parameters. This method can be applied to real-life application problems, such as traffic management and material flow congestion analysis.     

Automated mitosis detection of stem cell populations in phase-contrast microscopy images

Due to the enormous potential and impact that stem cells may have on regenerative medicine, there has been a rapidly growing interest for tools to analyze and characterize the behaviors of these cells in vitro in an automated and high throughput fashion. Among these behaviors, mitosis, or cell division, is important since stem cells proliferate and renew themselves through mitosis. However, current automated systems for measuring cell proliferation often require destructive or sacrificial methods of cell manipulation such as cell lysis or in vitro staining. In this paper, we propose an effective approach for automated mitosis detection using phase-contrast time-lapse microscopy, which is a nondestructive imaging modality, thereby allowing continuous monitoring of cells in culture. In our approach, we present a probabilistic model for event detection, which can simultaneously 1) identify spatio-temporal patch sequences that contain a mitotic event and 2) localize a birth event, defined as the time and location at which cell division is completed and two daughter cells are born. Our approach significantly outperforms previous approaches in terms of both detection accuracy and computational efficiency, when applied to multipotent C3H10T1/2 mesenchymal and C2C12 myoblastic stem cell populations.     

数字共焦显微镜实验3维点扩散函数的构建

为了获取作者所在实验室数字共焦显微镜光学系统实验3维点扩散函数,采用荧光微珠模拟点光源,利用该显微镜对荧光微珠不同散焦量的切片图像进行采集,采用多图像平均法对切片图像进行降噪处理,以此构建显微镜光学系统的实验3维点扩散函数。以该实验点扩散函数进行3维显微图像复原,并与理论点扩散函数的复原效果进行了比较。结果表明,两种点扩散函数对图像复原时均获得良好的复原效果,但是以正确实验方法获取的实验3维点扩散函数,更符合该光学系统的光学传递特性,复原效果更准确。所构建的实验3维点扩散函数,为下一步的生物显微图像复原处理提供了一种较好的选择,为数字共焦显微镜实验3维点扩散函数的构建提供了一种有效的方法。     

MONO－PULSE RADAR 3－D IMAGING TECHNIQUES FOR TARGET IN STEPPED TRACKING MODE

A method for mono-pulse radar 3-D imaging in stepped tracking mode is presented and the amplitude linear modulation of error signals in stepped tracking mode is analyzed with its compensation method followes,so the problem of precisely tracking of target is solved.Finally the validity of these methods is proven by the simulation results.     

基于条纹跟踪的动态过程三维轮廓术

针对在快变化动态过程三维轮廓术中,成像设备拍摄频率不满足抽样定理,对物体表面动态条纹时间采样不足以及物体表面发生破裂的动态过程,提出一种采用标记条纹进行跟踪的动态过程三维轮廓术,能有效地解决因成像设备时间采样不足和条纹断裂对三维相位展开所带来的影响问题,获得物体正确的动态面形分布,计算机模拟证实了此种方法的正确性。