Characteristics of infrared imaging systems that benefit from superresolution reconstruction

There have been numerous applications of superresolution reconstruction algorithms to improve the range performance of infrared imagers. These studies show there can be a dramatic improvement in range performance when superresolution algorithms are applied to undersampled imager outputs. These occur when the imager is moving relative to the target, which creates different spatial samplings of the field of view for each frame. The degree of performance benefit is dependent on the relative sizes of the detector/spacing and the optical blur spot in focal plane space. The minimum blur spot size achievable on the focal plane is dependent on the system F/number. Hence, we provide a range of these sensor characteristics, for which there is a benefit from superresolution reconstruction algorithms. Additionally, we quantify the potential performance improvements associated with these algorithms. We also provide three infrared sensor examples to show the range of improvements associated with provided guidelines.     

Pseudorandom phase masks for superresolution imaging from subpixel shifting

We present a method for overcoming the pixel-limited resolution of digital imagers. Our method combines optical point-spread function engineering with subpixel image shifting. We place an optimized pseudorandom phase mask in the aperture stop of a conventional imager and demonstrate the improved performance that can be achieved by combining multiple subpixel shifted images. Simulation results show that the pseudorandom phase-enhanced lens (PRPEL) imager achieves as much as 50% resolution improvement over a conventional multiframe imager. The PRPEL imager also enhances reconstruction root-mean-squared error by as much as 20%. We present experimental results that validate the predicted PRPEL imager performance.     

Superresolution image reconstruction from a sequence of aliased imagery

We present a superresolution image reconstruction from a sequence of aliased imagery. The subpixel shifts (displacement) among the images are unknown due to the uncontrolled natural jitter of the imager. A correlation method is utilized to estimate subpixel shifts between each low-resolution aliased image with respect to a reference image. An error-energy reduction algorithm is derived to reconstruct the high-resolution alias-free output image. The main feature of this proposed error-energy reduction algorithm is that we treat the spatial samples from low-resolution images that possess unknown and irregular (uncontrolled) subpixel shifts as a set of constraints to populate an oversampled (sampled above the desired output bandwidth) processing array. The estimated subpixel locations of these samples and their values constitute a spatial domain constraint. Furthermore, the bandwidth of the alias-free image (or the sensor imposed bandwidth) is the criterion used as a spatial frequency domain constraint on the oversampled processing array. The results of testing the proposed algorithm on the simulated low- resolution forward-looking infrared (FLIR) images, real-world FLIR images, and visible images are provided. A comparison of the proposed algorithm with a standard interpolation algorithm for processing the simulated low-resolution FLIR images is also provided.     

Prototype development and field-test results of an adaptive multiresolution PANOPTES imaging architecture

The design, development, and field-test results of a visible-band, folded, multiresolution, adaptive computational imaging system based on the Processing Arrays of Nyquist-limited Observations to Produce a Thin Electro-optic Sensor (PANOPTES) concept is presented. The architectural layout that enables this imager to be adaptive is described, and the control system that ensures reliable field-of-view steering for precision and accuracy in subpixel target registration is explained. A digital superresolution algorithm introduced to obtain high-resolution imagery from field tests conducted in both nighttime and daytime imaging conditions is discussed. The digital superresolution capability of this adaptive PANOPTES architecture is demonstrated via results in which resolution enhancement by a factor of 4 over the detector Nyquist limit is achieved.     

面向非稳态传热的热像仪相对测温性能评估

提出了一种热像仪相对测温的性能评价方法。将薄片热电偶的热节点固定在温度场均匀的加热单元表面,作为标准辐射温度发生单元,被待评价的热像仪观测和测温,对比热像仪观测的热电偶热节点表面温度场数据和热电偶自身测得的温度数据的相似度,对热像仪相对测温性能进行评价。基于国内外4个品牌的7款不同规格热像仪进行测试,结果表明热像仪的相对测温误差和绝对测温误差相差较大,相对测温误差通常介于噪声等效温差和绝对测温误差之间,同时发现6号热像仪性能较差,温漂严重,通过装置中的A、B两靶标同时测试热像仪性能,评价结果的标准差均在8%以内,这表明该方法具有较高的可靠性,对热像仪相对测温性能评价研究有一定的参考意义。     

红外热像仪作用距离现场检测装置的研究

针对目前红外热像仪现场性能检测手段的不足,设计一套便携式红外热像仪最小可分辨温差(MRTD)检测装置,首先现场测量出红外热像仪的MRTD,再由大气透过率模型计算得到目标的表观温差,最后通过Johnson准则分析计算得到红外热像仪探测或识别目标的能力.试验结果表明该方法准确可行,可用于各类红外热像仪作用距离的检测.     

Face recognition performance with superresolution

With the prevalence of surveillance systems, face recognition is crucial to aiding the law enforcement community and homeland security in identifying suspects and suspicious individuals on watch lists. However, face recognition performance is severely affected by the low face resolution of individuals in typical surveillance footage, oftentimes due to the distance of individuals from the cameras as well as the small pixel count of low-cost surveillance systems. Superresolution image reconstruction has the potential to improve face recognition performance by using a sequence of low-resolution images of an individual's face in the same pose to reconstruct a more detailed high-resolution facial image. This work conducts an extensive performance evaluation of superresolution for a face recognition algorithm using a methodology and experimental setup consistent with real world settings at multiple subject-to-camera distances. Results show that superresolution image reconstruction improves face recognition performance considerably at the examined midrange and close range.     

Image preprocessing for improving computational efficiency in implementation of restoration and superresolution algorithms

Computational complexity is a major impediment to the real-time implementation of image restoration and superresolution algorithms in many applications. Although powerful restoration algorithms have been developed within the past few years utilizing sophisticated mathematical machinery (based on statistical optimization and convex set theory), these algorithms are typically iterative in nature and require a sufficient number of iterations to be executed to achieve the desired resolution improvement that may be needed to meaningfully perform postprocessing image exploitation tasks in practice. Additionally, recent technological breakthroughs have facilitated novel sensor designs (focal plane arrays, for instance) that make it possible to capture megapixel imagery data at video frame rates. A major challenge in the processing of these large-format images is to complete the execution of the image processing steps within the frame capture times and to keep up with the output rate of the sensor so that all data captured by the sensor can be efficiently utilized. Consequently, development of novel methods that facilitate real-time implementation of image restoration and superresolution algorithms is of significant practical interest and is the primary focus of this study. The key to designing computationally efficient processing schemes lies in strategically introducing appropriate preprocessing steps together with the superresolution iterations to tailor optimized overall processing sequences for imagery data of specific formats. For substantiating this assertion, three distinct methods for tailoring a preprocessing filter and integrating it with the superresolution processing steps are outlined. These methods consist of a region-of-interest extraction scheme, a background-detail separation procedure, and a scene-derived information extraction step for implementing a set-theoretic restoration of the image that is less demanding in computation compared with the superresolution iterations. A quantitative evaluation of the performance of these algorithms for restoring and superresolving various imagery data captured by diffraction-limited sensing operations are also presented.     

Impact of measurement precision and noise on superresolution image reconstruction

The performance of uniform and nonuniform detector arrays for application to the PANOPTES (processing arrays of Nyquist-limited observations to produce a thin electro-optic sensor) flat camera design is analyzed for measurement noise environments including quantization noise and Gaussian and Poisson processes. Image data acquired from a commercial camera with 8 bit and 14 bit output options are analyzed, and estimated noise levels are computed. Noise variances estimated from the measurement values are used in the optimal linear estimators for superresolution image reconstruction.     

Design Optimization of a Digital Sun Sensor for Use With Parametric Processing

In this paper, we use genetic algorithms (GAs) as a heuristic for optimizing the illumination pattern for a single-axis digital sun sensor. Previous work has demonstrated that parametric algorithms can be used to provide better estimates of sun position than conventional centroiding techniques. The performance of these algorithms depends, in part, on the illumination pattern on the detector. Using a linear-phase superresolution technique that is combined with GA, we alter the number, shape, and placement of illuminating features. The GA estimator discovered high-fitness solutions that offer threefold to fivefold improvements over the baseline sensor design. We contend that these multiple peak patterns can greatly improve the performance of the sun sensor when they are coupled with parametric methods for sun position estimation. The optimal illumination pattern can be implemented, at minimal cost, by fabricating a replacement aperture mask.      

基于红外热成像技术的复合式温场测温方法

针对测量目标发射率和吸收率估计不准确将造成红外热像仪测量温场误差较大的问题,提出了用热电偶作为标准建立红外热像仪修正模型的方法.建立热电偶所在位置的修正模型后,用其修正热像仪在其它位置的测量结果,从而提高了热像仪测量温场的准确度.30～200℃的实验证明其修正值具有较好的重复性和复现性.     

ACTIVE-EYES: an adaptive pixel-by-pixel image-segmentation sensor architecture for high-dynamic-range hyperspectral imaging

The ACTIVE-EYES (adaptive control for thermal imagers via electro-optic elements to yield an enhanced sensor) architecture, an adaptive image-segmentation and processing architecture, based on digital micromirror (DMD) array technology, is described. The concept provides efficient front-end processing of multispectral image data by adaptively segmenting and routing portions of the scene data concurrently to an imager and a spectrometer. The goal is to provide a large reduction in the amount of data required to be sensed in a multispectral imager by means of preprocessing the data to extract the most useful spatial and spectral information during detection. The DMD array provides the flexibility to perform a wide range of spatial and spectral analyses on the scene data. The spatial and spectral processing for different portions of the input scene can be tailored in real time to achieve a variety of preprocessing functions. Since the detected intensity of individual pixels may be controlled, the spatial image can be analyzed with gain varied on a pixel-by-pixel basis to enhance dynamic range. Coarse or fine spectral resolution can be achieved in the spectrometer by use of dynamically controllable or addressable dispersion elements. An experimental prototype, which demonstrated the segmentation between an imager and a grating spectrometer, was demonstrated and shown to achieve programmable pixelated intensity control. An information theoretic analysis of the dynamic-range control aspect was conducted to predict the performance enhancements that might be achieved with this architecture. The results indicate that, with a properly configured algorithm, the concept achieves the greatest relative information recovery from a detected image when the scene is made up of a relatively large area of moderate-dynamic-range pixels and a relatively smaller area of strong pixels that would tend to saturate a conventional sensor.     

A CMOS floating-gate matrix transform imager

A new transform imager technology and architecture is introduced in this paper. This approach allows for retina and higher-level bio-inspired computation in a programmable architecture that still possesses similar high-fill factor pixels of APS imagers. This imager is capable of programmable matrix operations on the image, where the image can be presented as either a full matrix or using block matrix operations. Each pixel is composed of a photodiode sensor element and a multiplier. The core imager performs computation at the pixel plane but still holds to a fill factor greater than 46 %. The resulting data-flow architecture directly allows computation of spatial transforms, motion computations, and stereo computations.     

Continuous radial superresolution filters with large focal depth

A new set of continuous superresolution filters is proposed which exhibits a radial superresolution performance with an extended depth of focus in an optical system by properly choosing the design parameters. Numerical simulation results of the performance parameters of the superresolution gain, the radial central core size, the Strehl ratio, the side-lobe factor and the depth of focus with different design parameters for the optimized patterns are displayed. We also give a design example for this kind of filter characterized by a birefringent element inserted between two parallel polarizers. This kind of filter would be useful in fields such as optical data storage systems.     

Superresolved imaging in digital holography by superposition of tilted wavefronts

A technique based on superresolution by digital holographic microscopic imaging is presented. We used a two dimensional (2-D) vertical-cavity self-emitting laser (VCSEL) array as spherical-wave illumination sources. The method is defined in terms of an incoherent superposition of tilted wavefronts. The tilted spherical wave originating from the 2-D VCSEL elements illuminates the target in transmission mode to obtain a hologram in a Mach-Zehnder interferometer configuration. Superresolved images of the input object above the common lens diffraction limit are generated by sequential recording of the individual holograms and numerical reconstruction of the image with the extended spatial frequency range. We have experimentally tested the approach for a microscope objective with an exact 2-D reconstruction image of the input object. The proposed approach has implementation advantages for applications in biological imaging or the microelectronic industry in which structured targets are being inspected.     

Compressive imaging system design using task-specific information

We present a task-specific information (TSI) based framework for designing compressive imaging (CI) systems. The task of target detection is chosen to demonstrate the performance of the optimized CI system designs relative to a conventional imager. In our optimization framework, we first select a projection basis and then find the associated optimal photon-allocation vector in the presence of a total photon-count constraint. Several projection bases, including principal components (PC), independent components, generalized matched-filter, and generalized Fisher discriminant (GFD) are considered for candidate CI systems, and their respective performance is analyzed for the target-detection task. We find that the TSI-optimized CI system design based on a GFD projection basis outperforms all other candidate CI system designs as well as the conventional imager. The GFD-based compressive imager yields a TSI of 0.9841 bits (out of a maximum possible 1 bit for the detection task), which is nearly ten times the 0.0979 bits achieved by the conventional imager at a signal-to-noise ratio of 5.0. We also discuss the relation between the information-theoretic TSI metric and a conventional statistical metric like probability of error in the context of the target-detection problem. It is shown that the TSI can be used to derive an upper bound on the probability of error that can be attained by any detection algorithm.     

Midwave infrared and visible sensor performance modeling: small craft identification discrimination criteria for maritime security

The new emphasis on Anti-Terrorism and Force Protection (AT/FP), for both shore and sea platform protection, has resulted in a need for infrared imager design and evaluation tools that demonstrate field performance against U.S. Navy AT/FP requirements. In the design of infrared imaging systems for target acquisition, a discrimination criterion is required for successful sensor realization. It characterizes the difficulty of the task being performed by the observer and varies for different target sets. This criterion is used in both assessment of existing infrared sensor and in the design of new conceptual sensors. We collected 12 small craft signatures (military and civilian) in the visible band during the day and the long-wave and midwave infrared spectra in both the day and the night environments. These signatures were processed to determine the targets' characteristic dimension and contrast. They were also processed to band limit the signature's spatial information content (simulating longer range), and a perception experiment was performed to determine the task difficulty (N50 and V50). The results are presented and can be used for Navy and Coast Guard imaging infrared sensor design and evaluation.     

Mathematical extrapolation of image spectrum for constraint-set design and set-theoretic superresolution

Several powerful iterative algorithms are being developed for the restoration and superresolution of diffraction-limited imagery data by use of diverse mathematical techniques. Notwithstanding the mathematical sophistication of the approaches used in their development and the potential for resolution enhancement possible with their implementation, the spectrum extrapolation that is central to superresolution comes in these algorithms only as a by-product and needs to be checked only after the completion of the processing steps to ensure that an expansion of the image bandwidth has indeed occurred. To overcome this limitation, a new approach of mathematically extrapolating the image spectrum and employing it to design constraint sets for implementing set-theoretic estimation procedures is described. Performance evaluation of a specific projection-onto-convex-sets algorithm by using this approach for the restoration and superresolution of degraded images is outlined. The primary goal of the method presented is to expand the power spectrum of the input image beyond the range of the sensor that captured the image.     

Effects of Environmental Conditions on the Performance of Thermal Imagers

An experimental setup for characterization of thermal imagers has been designed and constructed. The setup comprises a climatic chamber and a temperature-controlled flat blackbody target that allow testing of thermal imagers in the ambient temperature range from ?13 °C to + 23  °C and at target temperatures from ?15 °C to + 120 °C. The setup has been tested with a high quality thermal imager for scientific use and a mid-range thermal imager for industrial applications. The test results indicate that the setup allows monitoring the changes in indication of thermal imagers within the standard uncertainty limits of 0.5 °C.     

Analysis of Thermal Imagers

A thermal imager with a microbolometer focal plane array (FPA) detector which does not require cooling is a relatively new type of instrument. With increasing use of thermal imagers for absolute temperature measurements, there is also an increasing need for their calibration. Five thermal imagers from three manufacturers were evaluated to identify parameters which affect their performance for accurate temperature measurements. Evaluation methods and measurements of parameters such as accuracy, temperature resolution, drift between internal calibration, non-uniformity, size-of-source effect, and distance effect are discussed. Based on the results, it is indicated which parameters should be evaluated in the calibration procedure as well as the uncertainty budget. The suitability of a blackbody for calibration was also addressed, especially in terms of dimensions related to the field of view of a thermal imager at the minimum focal distance.