Effects of Environmental Conditions on the Performance of Thermal Imagers

An experimental setup for characterization of thermal imagers has been designed and constructed. The setup comprises a climatic chamber and a temperature-controlled flat blackbody target that allow testing of thermal imagers in the ambient temperature range from ?13 °C to + 23  °C and at target temperatures from ?15 °C to + 120 °C. The setup has been tested with a high quality thermal imager for scientific use and a mid-range thermal imager for industrial applications. The test results indicate that the setup allows monitoring the changes in indication of thermal imagers within the standard uncertainty limits of 0.5 °C.     

Pose estimation for autonomous grasping with a robotic arm system

A robust and accurate method for estimating the 3-D pose of a planar rigid object is presented. This article demonstrates that 3-D pose estimation becomes feasible by using the 2-D tracking points on an object of scale-invariant feature transform (SIFT) and 3-D point cloud detected by stereo vision on an object, assuming that a 3-D geometric model of an object is known a priori. The roll and pitch angles of an object are estimated by the normal vector of approximate plane of 3-D point cloud on an object and the yaw angle is estimated by 2-D tracking point on an object of SIFT. Accurate object detection and localization in the camera coordinate system is crucial for grasping. In the motion planning, the bidirectional rapidly exploring random tree algorithm is used to search for a valid path for efficient grasping. Our robot arm can robustly and autonomously grasp a randomly rotative rigid object detected by SIFT in 3-D space. We have realized a grasping scenario with a dexterous arm (ADAM) such that an object in front of ADAM can be grasped. This demonstration shows how the proposed components build a dexterous and robust system integrating object detection, pose estimation, and motion planning.     

Analysis of Thermal Imagers

A thermal imager with a microbolometer focal plane array (FPA) detector which does not require cooling is a relatively new type of instrument. With increasing use of thermal imagers for absolute temperature measurements, there is also an increasing need for their calibration. Five thermal imagers from three manufacturers were evaluated to identify parameters which affect their performance for accurate temperature measurements. Evaluation methods and measurements of parameters such as accuracy, temperature resolution, drift between internal calibration, non-uniformity, size-of-source effect, and distance effect are discussed. Based on the results, it is indicated which parameters should be evaluated in the calibration procedure as well as the uncertainty budget. The suitability of a blackbody for calibration was also addressed, especially in terms of dimensions related to the field of view of a thermal imager at the minimum focal distance.     

Calibration of Thermal Imagers by Evaluation of the Entire Field-of-View

Calibration laboratories and national metrology institutes are commonly calibrating thermal imagers with equipment and procedures that were primarily developed for the calibration of optical radiation thermometers. This approach provides direct calibration results only for a small percentage of the detector elements, which are in view of a blackbody aperture. For the remaining detector elements of the thermal imager under calibration, it is assumed that they have the same characteristics. This inference may not be always true, so a new calibration procedure is presented, which evaluates not only the correction and uncertainty of the selected detector elements, but also the uniformity of all detector elements of the thermal imager. Uniformity evaluation is based on two different approaches for the evaluation of the entire field-of-view (FOV) of the thermal imager. The first approach consists of dividing the imager FOV into partial FOVs according to the available size of the blackbody aperture. Each of the partial FOVs is successively positioned in front of a blackbody aperture, and it is automatically assembled into a single result image. The idea of the second approach is to position all detector elements (entire FOV) at the same time in front of a large blackbody aperture. Experimental work and analysis of both approaches for the calibration procedure of thermal imagers are presented.     

基于贝叶斯推理时空交互式多视角目标跟踪

本文针对多摄像机视频跟踪的应用场景,在贝叶斯推理的框架下,提出了一种具有分布式时空交互计算特点的目标跟踪算法.本文首先利用贝叶斯网络对拓扑确定的多摄像机系统进行建模,并对待估状态(目标位置)高阶联合后验概率密度函数进行时空的递推,最后借助序列蒙特卡洛(粒子滤波)逼近后验概率密度函数,并采用高效的数据传送机制高效求解出跟...     

面向非稳态传热的热像仪相对测温性能评估

提出了一种热像仪相对测温的性能评价方法。将薄片热电偶的热节点固定在温度场均匀的加热单元表面,作为标准辐射温度发生单元,被待评价的热像仪观测和测温,对比热像仪观测的热电偶热节点表面温度场数据和热电偶自身测得的温度数据的相似度,对热像仪相对测温性能进行评价。基于国内外4个品牌的7款不同规格热像仪进行测试,结果表明热像仪的相对测温误差和绝对测温误差相差较大,相对测温误差通常介于噪声等效温差和绝对测温误差之间,同时发现6号热像仪性能较差,温漂严重,通过装置中的A、B两靶标同时测试热像仪性能,评价结果的标准差均在8%以内,这表明该方法具有较高的可靠性,对热像仪相对测温性能评价研究有一定的参考意义。     

Thermal Image Degradation Through a Heated Sapphire Window

Optical windows are indispensable for monitoring industrial processes under vacuum or high pressure by using thermal imagers and radiation thermometers. When a thermal imager observes a sample through an infrared window at elevated temperatures, the window emits additional thermal radiation and increases the background signal of the thermal images, which results in image degradation. Standard four-bar images with various radiance temperature differences were measured using a thermal imager with a spectral band from 3 ??m to 5 ??m through a UV-grade sapphire window. The four-bar images are given by a blackbody collimator with various image patterns. The window was indirectly heated in a furnace and then rapidly placed on the optical path between the collimator and the thermal imager. The four-bar image degradation was measured as a function of the window temperature and the radiance temperature difference of the four-bar pattern. A simple equation which describes the contrast of the four-bar image by using the transmittance and reflectance of the sapphire window was proposed. It was confirmed that the model can properly predict the window temperature when the appearance of the four-bar pattern cannot be determined.     

基于红外热成像技术的复合式温场测温方法

针对测量目标发射率和吸收率估计不准确将造成红外热像仪测量温场误差较大的问题,提出了用热电偶作为标准建立红外热像仪修正模型的方法.建立热电偶所在位置的修正模型后,用其修正热像仪在其它位置的测量结果,从而提高了热像仪测量温场的准确度.30～200℃的实验证明其修正值具有较好的重复性和复现性.     

Estimation of contour motion and deformation for nonrigid object tracking

We present an algorithm for nonrigid contour tracking in heavily cluttered background scenes. Based on the properties of nonrigid contour movements, a sequential framework for estimating contour motion and deformation is proposed. We solve the nonrigid contour tracking problem by decomposing it into three subproblems: motion estimation, deformation estimation, and shape regulation. First, we employ a particle filter to estimate the global motion parameters of the affine transform between successive frames. Then we generate a probabilistic deformation map to deform the contour. To improve robustness, multiple cues are used for deformation probability estimation. Finally, we use a shape prior model to constrain the deformed contour. This enables us to retrieve the occluded parts of the contours and accurately track them while allowing shape changes specific to the given object types. Our experiments show that the proposed algorithm significantly improves the tracker performance.     

Pseudorandom phase masks for superresolution imaging from subpixel shifting

We present a method for overcoming the pixel-limited resolution of digital imagers. Our method combines optical point-spread function engineering with subpixel image shifting. We place an optimized pseudorandom phase mask in the aperture stop of a conventional imager and demonstrate the improved performance that can be achieved by combining multiple subpixel shifted images. Simulation results show that the pseudorandom phase-enhanced lens (PRPEL) imager achieves as much as 50% resolution improvement over a conventional multiframe imager. The PRPEL imager also enhances reconstruction root-mean-squared error by as much as 20%. We present experimental results that validate the predicted PRPEL imager performance.     

Spatial Scatter Effects in the Calibration of IR Pyrometers and Imagers

Differences in the calibration conditions and the real-life applications of infrared pyrometers, radiometers, or imagers can contribute to significant measurement errors due to the presence of scattered light from the areas surrounding the reference source during the calibration process or the measured object in the field measurements. This out-of-field scatter (also known as size-of-source effect, SSE) has to be analyzed separately for each artifact to ensure applicability of the calibration results to the scene of actual measurement. This article discusses SSE characterization methods and specific requirements for calibrating single-element radiometers in the near- and mid-IR parts of the optical radiation spectrum. Two new SSE tools developed at National Institute of Standards and Technology to support routine calibration of IR pyrometers, radiometers, and imagers at the recently developed Advanced Infrared Radiometry and Imaging (AIRI) facility are described. The results of characterization of different commonly used radiometers, including an industrial-grade pyrometer, a high-accuracy pyrometer, two different infrared spectrometers, and an infrared imager, are presented.     

Equation-based triangle orientation discrimination sensor performance model based on sampling effects

An equation-based triangle orientation discrimination (TOD) performance model was first developed to focus on staring thermal imagers. Specifically, the spatial spectra distribution of a standard triangle pattern is determined. The modulation effects of the overall components of the system on a nonperiodic standard triangle pattern are analyzed and modeled. The matched filter idea is used to characterize quantitatively the spatiotemporal integration of the eye and brain to signal, aliasing, and various noise components over a triangle pattern area, and the perceived signal-to-noise ratio of the staring thermal imager is derived. Further, the TOD performance theoretical model is established. Comparison with the experimental results shows that this theoretical model gives a reasonable prediction of the TOD performance curve for staring thermal imagers. Although more tests and modifications are required, these preliminary results suggest that this model can be developed into a model that predicts the TOD for a wide range of sensors.     

Efficient object detection and tracking in video sequences

One of the most important problems in computer vision is the computation of the two-dimensional projective transformation (homography) that maps features of planar objects in different images and videos. This computation is required by many applications such as image mosaicking, image registration, and augmented reality. The real-time performance imposes constraints on the methods used. In this paper, we address the real-time detection and tracking of planar objects in a video sequence where the object of interest is given by a reference image template. Most existing approaches for homography estimation are based on two steps: feature extraction (first step) followed by a combinatorial optimization method (second step) to match features between the reference template and the scene frame. This paper has two main contributions. First, we detect both planar and nonplanar objects via efficient object feature classification in the input images, which is applied prior to performing the matching step. Second, for the tracking part (planar objects), we propose a fast method for the computation of the homography that is based on the transferred object features and their associated local raw brightness. The advantage of the proposed schemes is a fast matching as well as fast and robust object registration that is given by either a homography or three-dimensional pose.     

Visible-regime polarimetric imager: a fully polarimetric,real-time imaging system

A fully polarimetric optical camera system has been constructed to obtain polarimetric information simultaneously from four synchronized charge-coupled device imagers at video frame rates of 60 Hz and a resolution of 640 x 480 pixels. The imagers view the same scene along the same optical axis by means of a four-way beam-splitting prism similar to ones used for multiple-imager, common-aperture color TV cameras. Appropriate polarizing filters in front of each imager provide the polarimetric information. Mueller matrix analysis of the polarimetric response of the prism, analyzing filters, and imagers is applied to the detected intensities in each imager as a function of the applied state of polarization over a wide range of linear and circular polarization combinations to obtain an average polarimetric calibration consistent to approximately 2%. Higher accuracies can be obtained by improvement of the polarimetric modeling of the splitting prism and by implementation of a pixel-by-pixel calibration.     

Optimal design of CMOS pseudoactive pixel sensor (PAPS) structure for low-dark-current and large-array-size imager applications

In this paper, a pixel structure called the optimal pseudoactive pixel sensor (OPAPS) is proposed and analyzed for the applications of CMOS imagers. The shared zero-biased-buffer in the pixel is used to suppress both dark current of photodiode and leakage current of pixel switches by keeping both biases of photodiode and parasitic pn junctions in the pixel bus at zero voltage or near zero voltage. The factor of photocurrent-to-dark-current ratio per pixel area (PDRPA) is defined to characterize the performance of the OPAPS structure. It is found that a zero-biased-buffer shared by four pixels can achieve the highest PDRPA. In addition, the column sampling circuits and output correlated double sampling circuits are also used to suppress fixed-pattern noise, clock feedthrough noise, and channel charge injection. An experimental chip of the proposed OPAPS CMOS imager with the format of 352/spl times/288 (CIF) has been designed and fabricated by using 0.25-/spl mu/m single-poly-five-level-metal (1P5M) N-well CMOS process. In the fabricated CMOS imager, one shared zero-biased-buffer is used for four pixels where the PDRPA is equal to 47.29 /spl mu/m/sup -2/. The fabricated OPAPS CMOS imager has a pixel size of 8.2/spl times/.2 /spl mu/m, fill factor of 42%, and chip size of 3630/spl times/3390 /spl mu/m. Moreover, the measured maximum frame rate is 30 frames/s and the dark current is 82 pA/cm/sup 2/. Additionally, the measured optical dynamic range is 65 dB. It is found that the proposed OPAPS structure has lower dark current and higher optical dynamic range as compared with the active pixel sensor (APS) and the conventional passive pixel sensor (PPS). Thus, the proposed OPAPS structure has high potential for the applications of high-quality and large-array-size CMOS imagers.     

A multimodal approach for individual tracking of people and their belongings

Abstract

In this study, a fully automatic surveillance system for indoor environments which is capable of tracking multiple objects using both visible and thermal band images is proposed. These two modalities are fused to track people and the objects they carry separately using their heat signatures and the owners of the belongings are determined. Fusion of complementary information from different modalities (for example, thermal images are not affected by shadows and there is no thermal reflection or halo effect in visible images) is shown to result in better object detection performance. We use adaptive background modeling and local intensity operation for object detection and the mean-shift tracking algorithm for fully automatic tracking. Trackers are refreshed to resolve potential problems which may occur due to the changes in object’s size, shape and to handle occlusion-split and to detect newly emerging objects as well as objects that leave the scene. The proposed scheme is applied to the abandoned object detection problem and the results are compared with the state of art methods. The results show that the proposed method facilitate individual tracking of objects for various applications, and provide lower false alarm rates compared to the state of art methods when applied to the abandoned object detection problem.     

ACTIVE-EYES: an adaptive pixel-by-pixel image-segmentation sensor architecture for high-dynamic-range hyperspectral imaging

The ACTIVE-EYES (adaptive control for thermal imagers via electro-optic elements to yield an enhanced sensor) architecture, an adaptive image-segmentation and processing architecture, based on digital micromirror (DMD) array technology, is described. The concept provides efficient front-end processing of multispectral image data by adaptively segmenting and routing portions of the scene data concurrently to an imager and a spectrometer. The goal is to provide a large reduction in the amount of data required to be sensed in a multispectral imager by means of preprocessing the data to extract the most useful spatial and spectral information during detection. The DMD array provides the flexibility to perform a wide range of spatial and spectral analyses on the scene data. The spatial and spectral processing for different portions of the input scene can be tailored in real time to achieve a variety of preprocessing functions. Since the detected intensity of individual pixels may be controlled, the spatial image can be analyzed with gain varied on a pixel-by-pixel basis to enhance dynamic range. Coarse or fine spectral resolution can be achieved in the spectrometer by use of dynamically controllable or addressable dispersion elements. An experimental prototype, which demonstrated the segmentation between an imager and a grating spectrometer, was demonstrated and shown to achieve programmable pixelated intensity control. An information theoretic analysis of the dynamic-range control aspect was conducted to predict the performance enhancements that might be achieved with this architecture. The results indicate that, with a properly configured algorithm, the concept achieves the greatest relative information recovery from a detected image when the scene is made up of a relatively large area of moderate-dynamic-range pixels and a relatively smaller area of strong pixels that would tend to saturate a conventional sensor.     

Probabilistic 3D object recognition and pose estimation using multiple interpretations generation

This paper presents a probabilistic object recognition and pose estimation method using multiple interpretation generation in cluttered indoor environments. How to handle pose ambiguity and uncertainty is the main challenge in most recognition systems. In order to solve this problem, we approach it in a probabilistic manner. First, given a three-dimensional (3D) polyhedral object model, the parallel and perpendicular line pairs, which are detected from stereo images and 3D point clouds, generate pose hypotheses as multiple interpretations, with ambiguity from partial occlusion and fragmentation of 3D lines especially taken into account. Different from the previous methods, each pose interpretation is represented as a region instead of a point in pose space reflecting the measurement uncertainty. Then, for each pose interpretation, more features around the estimated pose are further utilized as additional evidence for computing the probability using the Bayesian principle in terms of likelihood and unlikelihood. Finally, fusion strategy is applied to the top ranked interpretations with high probabilities, which are further verified and refined to give a more accurate pose estimation in real time. The experimental results show the performance and potential of the proposed approach in real cluttered domestic environments.     

Ultraviolet and visible imaging and spectrographic imaging instrument

The Ultraviolet and Visible Imaging and Spectrographic Imaging experiment consists of five spectrographic imagers and four imagers. These nine sensors provide spectrographic and imaging capabilities from 110 to 900 nm. The spectrographic imagers share an off-axis design in which selectable slits alternate fields of view (1.00° × 0.10° or 1.00° × 0.05°) and spectral resolutions between 0.5 and 4 nm. Image planes of the spectrographic imager have a programmable spectral dimension with 68, 136, or 272 pixels across each individual spectral band, and a programmable spatial dimension with 5, 10, 20, or 40 pixels across the 1° slit length. A scan mirror sweeps the slit through a second spatial dimension to generate a 1° × 1° spectrographic image once every 5, 10, or 20 s, depending on the scan rate. The four imagers provide narrow-field (1.28° × 1.59°) and wide-field (10.5° × 13.1°) viewing. Each imager has a six-position filter wheel that selects various spectral regimes and neutral densities. The nine sensors ut lize intensified CCD detectors that have an intrascene dynamic range of ~ 10(3) and an interscene dynamic range of ~ 10(5); neutral-density filters provide an additional dynamic range of ~ 10(2-3). The detector uses an automatic gain control that permits the sensors to adjust to scenes of varying intensity. The sensors have common boresights and can operate separately, simultaneously, or synchronously. To be launched aboard the Midcourse Space Experiment spacecraft in the mid-1990's, the ultraviolet and visible imaging and spectrographic imaging instrument will investigate a multitude of celestial, atmospheric, and point sources during its planned 4-yr life.     

Grasp Planning and Visual Servoing for an Outdoors Aerial Dual Manipulator

This paper describes a system for grasping known objects with unmanned aerial vehicles (UAVs) provided with dual manipulators using an RGB-D camera. Aerial manipulation remains a very challenging task. This paper covers three principal aspects for this task: object detection and pose estimation, grasp planning, and in-flight grasp execution. First, an artificial neural network (ANN) is used to obtain clues regarding the object’s position. Next, an alignment algorithm is used to obtain the object’s six-dimensional (6D) pose, which is filtered with an extended Kalman filter. A three-dimensional (3D) model of the object is then used to estimate an arranged list of good grasps for the aerial manipulator. The results from the detection algorithm—that is, the object’s pose—are used to update the trajectories of the arms toward the object. If the target poses are not reachable due to the UAV’s oscillations, the algorithm switches to the next feasible grasp. This paper introduces the overall methodology, and provides the experimental results of both simulation and real experiments for each module, in addition to a video showing the results.