Gender recognition from visible and thermal infrared facial images

Most present research of gender recognition focuses on visible facial images, which are sensitive to illumination changes. In this paper, we proposed hybrid methods for gender recognition by fusing visible and thermal infrared images. First, the active appearance model is used to extract features from visible images, as well as local binary pattern features and several statistical temperature features are extracted from thermal infrared images. Then, feature selection is performed by using the F-test statistic. Third, we propose using Bayesian Networks to perform explicit and implicit fusion of visible and thermal infrared image features. For explicit fusion, we propose two Bayesian Networks to perform decision-level and feature-level fusion. For implicit fusion, we propose using features from one modality as privileged information to improve gender recognition by another modality. Finally, we evaluate the proposed methods on the Natural Visible and Infrared facial Expression spontaneous database and the Equinox face database. Experimental results show that both feature-level and decision-level fusion improve the gender recognition performance, compared to that achieved from one modality. The proposed implicit fusion methods successfully capture the role of privileged information of one modality, thus enhance the gender recognition from another modality.     

A new thermal infrared and visible spectrum images-based pedestrian detection system

In this paper, we propose a hybrid system for pedestrian detection, in which both thermal and visible images of the same scene are used. The proposed method is achieved in two basic steps: (1) Hypotheses generation (HG) where the locations of possible pedestrians in an image are determined and (2) hypotheses verification (HV), where tests are done to check the presence of pedestrians in the generated hypotheses. HG step segments the thermal image using a modified version of OTSU thresholding technique. The segmentation results are mapped into the corresponding visible image to obtain the regions of interests (possible pedestrians). A post-processing is done on the resulting regions of interests to keep only significant ones. HV is performed using random forest as classifier and a color-based histogram of oriented gradients (HOG) together with the histograms of oriented optical flow (HOOF) as features. The proposed approach has been tested on OSU Color-Thermal, INO Video Analytics and LITIV data sets and the results justify its effectiveness.

     

Fusion of thermal infrared and visible spectra for robust moving object detection

The detection of moving objects is a crucial step for many video surveillance applications whether using a visible camera (VIS) or an infrared (IR) one. In order to profit from both types, several fusion methods were proposed in the literature: low-level fusion, medium-level fusion and high-level fusion. The first one is the most used for moving objects’ detection in IR and VIS spectra. In this paper, we present an overview of the different moving object detection methods in IR and VIS spectra and a state of the art of the low-level fusion techniques. Moreover, we propose a new method for moving object detection using low-level fusion of IR and VIS spectra. In order to evaluate quantitatively and qualitatively our proposed method, three series of experiments were carried out using two well-known datasets namely “OSU Color-Thermal Database” and “INO-Database”; the results of these evaluations show promising results and demonstrate the effectiveness of the proposed method.     

Cloud classification from visible and infrared SMS-1 data

The success of a statistical classification-design sample model in discriminating cloud-type samples of visible and infrared meteorological satellite data depends on the selection of the design parameters for the system and the ability of the labelled design samples to characterize and discriminate class patterns within the given geographical region. In a companion study by Parikh (1977), pattern recognition design parameters were examined for a four-class problem and a three-class problem for NOAA-1 cloud data. The purpose of this study was to evaluate pattern recognition systems designed in the previous study on SMS-1 design and test sets. Experiments were conducted for both a four-class problem (separation of “low”, “mix”, “cirrus”, and “cumulonimbus” samples) and a three-class problem (separation of “low”, “cirrus”, and “cumulonimbus” samples). For the four-class problem, decreases in classification accuracy ranging from 4% to 11% occurred when the pattern recognition systems were designed and tested on two different data sets selected from the same satellite orbit. A similar decrease was not observed for the well-defined three-class problem.     

Fusion of infrared and visible images using neutrosophic fuzzy sets

Fusion of infrared and visible image is a technology which combines information from two different sensors for the same scene. It also gives extremely effective information complementation, which is widely used for the monitoring systems and military fields. Due to limited field depth in an imaging device, visible images can’t identify some targets that may not be apparent due to poor lighting conditions or because that the background color is similar to the target. To deal with this problem, a simple and efficient image fusion approach of infrared and visible images is proposed to extract target’s details from infrared images and enhance the vision in order to improve the performance of monitoring systems. This method depends on maximum and minimum operations in neutrosophic fuzzy sets. Firstly, the image is transformed from its spatial domain to the neutrosophic domain which is described by three membership sets: truth membership, indeterminacy membership, and falsity membership. The indeterminacy in the input data is handled to provide a comprehensive fusion result. Finally, deneutrosophicised process is made which means that the membership values are retransformed into a normal image space. At the end of the study, experimental results are applied to evaluate the performance of this approach and compare it to the recent image fusion methods using several objective evaluation criteria. These experiments demonstrate that the proposed method achieves outstanding visual performance and excellent objective indicators.

     

Background-subtraction using contour-based fusion of thermal and visible imagery

We present a new background-subtraction technique fusing contours from thermal and visible imagery for persistent object detection in urban settings. Statistical background-subtraction in the thermal domain is used to identify the initial regions-of-interest. Color and intensity information are used within these areas to obtain the corresponding regions-of-interest in the visible domain. Within each region, input and background gradient information are combined to form a Contour Saliency Map. The binary contour fragments, obtained from corresponding Contour Saliency Maps, are then fused into a single image. An A^* path-constrained search along watershed boundaries of the regions-of-interest is used to complete and close any broken segments in the fused contour image. Lastly, the contour image is flood-filled to produce silhouettes. Results of our approach are evaluated quantitatively and compared with other low- and high-level fusion techniques using manually segmented data.     

Object matching between visible and infrared images using a Siamese network

Applied Intelligence - In this study, we propose a method for object matching between visible and infrared images. We consider object matching between visible and infrared images as a computational...     

Contactless blood pressure sensing using facial visible and thermal images

Hypertension is one of the leading risk factors for several diseases. Measurement and monitoring of blood pressure anytime and anywhere are important to lower blood pressure and prevent pathogenesis of diseases. Non-contact blood pressure measurement is desired to monitor blood pressure anytime and anywhere. The aim of this study was to develop a non-contact blood pressure sensing system. A previous study reported that amplitude and time differences of facial photoplethysmogram (PPG) components extracted using brightness variation of facial skin color in facial visible images could be useful indices for estimating blood pressure. The maximum error between measured and estimated blood pressure using facial PPG components was 12 mmHg. An additional signal processing algorithm is desired to increase the accuracy for estimating blood pressure using facial PPG components. By contrast, facial skin temperature also reflects changes in the facial blood circulation. High-accuracy estimation of blood pressure could be expected using both facial PPG components and facial skin temperature. In this study, improvement of accuracy for estimating blood pressure using facial PPG components by attempting to apply additional signal processing to facial skin color variation. Furthermore, a correlation analysis between facial skin temperature and measured blood pressure was performed, and individual models for blood pressure estimation were created.     

Determination and correction of the relative shift between the visible and thermal infrared GOES sensor images

A simple method for the correction of the relative shift between the visible and thermal infrared GOES sensor images is introduced. It makes use of the variance operator and the cross-correlation between two patterns. Results indicate that the proposed method is very promising.     

Fully convolutional network-based infrared and visible image fusion

Multimedia Tools and Applications - This study proposes a novel fusion framework for infrared and visual images based on a full convolutional network (FCN) in the local non-subsampled shearlet...     

Multispectral particle filter tracking using adaptive decision-based fusion of visible and thermal sequences

Multimedia Tools and Applications - One of the main challenges of detection and tracking of objects in video monitoring is the lighting conditions of the scene under surveillance and its...     

Intelligent fault diagnosis of rotating machinery using infrared thermal image

This study presents a new intelligent diagnosis system for classification of different machine conditions using data obtained from infrared thermography. In the first stage of this proposed system, two-dimensional discrete wavelet transform is used to decompose the thermal image. However, the data attained from this stage are ordinarily high dimensionality which leads to the reduction of performance. To surmount this problem, feature selection tool based on Mahalanobis distance and relief algorithm is employed in the second stage to select the salient features which can characterize the machine conditions for enhancing the classification accuracy. The data received from the second stage are subsequently utilized to intelligent diagnosis system in which support vector machines and linear discriminant analysis methods are used as classifiers. The results of the proposed system are able to assist in diagnosing of different machine conditions.     

Integrating visible, near-infrared and short-wave infrared hyperspectral and multispectral thermal imagery for geological mapping at Cuprite, Nevada

This study investigated the potential value of integrating hyperspectral visible, near-infrared, and short-wave infrared imagery with multispectral thermal data for geological mapping. Two coregistered aerial data sets of Cuprite, Nevada were used: Airborne Visible/Infrared Imaging Spectrometer (AVIRIS) hyperspectral data, and MODIS/ASTER Airborne Simulator (MASTER) multispectral thermal data. Four classification methods were each applied to AVIRIS, MASTER, and a combined set. Confusion matrices were used to assess the classification accuracy. The assessment showed, in terms of kappa coefficient, that most classification methods applied to the combined data achieved a marked improvement compared to the results using either AVIRIS or MASTER thermal infrared (TIR) data alone. Spectral angle mapper (SAM) showed the best overall classification performance. Minimum distance classification had the second best accuracy, followed by spectral feature fitting (SFF) and maximum likelihood classification. The results of the study showed that SFF applied to the combination of AVIRIS with MASTER TIR data are especially valuable for identification of silicified alteration and quartzite, both of which exhibit distinctive features in the TIR region. SAM showed some advantages over SFF in dealing with multispectral TIR data, obtaining higher accuracy in discriminating low albedo volcanic rocks and limestone which do not have unique, distinguishing features in the TIR region.     

Detection of blue-absorbing aerosols using near infrared and visible (ocean color) remote sensing observations

An algorithm is presented, which is designed to identify blue-absorbing aerosols from near infrared and visible remote-sensing observations, as they are in particular collected by satellite ocean color sensors. The technique basically consists in determining an error budget at one wavelength around 510 nm, based on a first-guess estimation of the atmospheric path reflectance as if the atmosphere was of a maritime type, and on a reasonable hypothesis about the marine signal at this wavelength. The budget also includes the typical calibration uncertainty and the natural variability in the ocean optical properties. Identification of blue-absorbing aerosols is then achieved when the error budget demonstrates a significant over-correction of the atmospheric signal when using non-absorbing maritime aerosols. Implementation of the algorithm is presented, and its application to real observations by the MERIS and SeaWiFS ocean color sensors is discussed. The results demonstrate the skill of the algorithm in various regions of the ocean where absorbing aerosols are present, and for two different sensors. A validation of the results is also performed against in situ data from the AERONET, and further illustrates the skill of the algorithm and its general applicability.     

Soil moisture retrieval using thermal inertia, determined with visible and thermal spaceborne data, validated for European forests

Variations in soil moisture strongly affect surface energy balances, regional runoff, land erosion and vegetation productivity (potential crop yield). Hence, the detection of soil moisture content (SMC) is very valuable in the social, economic, humanitarian (food security) and environmental segments of society. A method to estimate SMC from optical and thermal spectral information of METEOSAT imagery based on thermal inertia (TI) is presented. Minimum and maximum TI values from time series are combined in the Soil Moisture Saturation Index (SMSI). To convert surface to soil profile values, a Markov type filter is used, based on a simple two layer water balance equation (the surface layer and the reservoir below) and an autocorrelation function. Ten-daily SMC values are compared with up-scaled (using AVHRR/NDVI) observations on 10 EUROFLUX sites in Europe for the 1997 growing season (March-October). Moreover, the thermal inertia approach is compared for 1997, with ERS Scatterometer data for eight EUROFLUX sites. METEOSAT pixels are up-scaled to accommodate the ERS Scatterometer spatial resolution. The regression coefficients (slope, intercept and R²) of the thermal inertia approach versus the up-scaled soil moisture observations from EUROFLUX sites vary between 0.811-1.148, − 0.0029-0.66 and 0.544-0.877, respectively, with a RRMSE range of 3.9% to 35.7%. The regression coefficients of the comparison of ERS Scatterometer derived Soil Water Index (SWI) versus the up-scaled Soil Moisture Saturation Index for the pooled case (binning the eight EUROFLUX sites) are 0.587, 0.105 and 0.441, respectively, with a RRMSE of 38%. A simple error propagation model applied for the thermal inertia approach reveals that the absolute and relative errors of the obtained soil moisture content is at least 0.010 m³ m^− 3 or 2.0% with a SMC of 0.203 m³ m^− 3. Recommendations are made to test and implement the TI methodology using NOAA/AVHRR imagery.     

Effect of surface wind speed and sensor view zenith angle dependence of emissivity on SST retrieval from thermal infrared data: ATSR

The Along Track Scanning Radiometer (ATSR) data are currently being processed at various places within the European community including the Rutherford Appleton Laboratory (RAL) in the U.K. In generating an atmospherically corrected sea-surface temperature (SST) field, the emissivity of the sea surface is assumed to be independent of the sensor view zenith angle, sea state and wavelength (Ian Barton, RAL, personal communication). The sensor view zenith angle dependence of the emissivity is generally not known because of the complications introduced by the surface wind speed. This paper attempts to evaluate the uncertainties introduced in the SST due to the variation of emissivity with the sensor view zenith angle and surface roughness generated by the wind speed.

Using the Cox and Munk formulation, Takashima and Takayama have simulated the sea water emissivities as a function of wind speed of up to 15ms^-1 and a range of the sensor view zenith angles. Their emissivity data for 11 and 12μm channels corresponding to the viewing geometry of the ATSR have been used in this work. It is shown that, depending on the value of the SST, there can be significant errors due to the sensor view zenith angle and sea surface roughness dependence of the emissivity. For example, if the SST is 10°C and the wind speed is 0ms^-1, then the errors due to the sensor view zenith angle dependence of the emissivity are shown to be 0·77°C and 0·55°C in 11 and 12μm channels, respectively, and at 15 ms^-l the respective errors reach about 1·ldeg K and 0·86 deg K. The errors due to the deviations of the emissivities from unity for nadir view in calm conditions are about 2·1°C and 3·5°C, respectively, in the 11 and 12μm channels. All of these errors are additive, indicating the importance of the calibration and validation.     

Robust Satellite Techniques for oil spill detection and monitoring using AVHRR thermal infrared bands

In this article, a new satellite technique for oil spill detection and monitoring is fully presented and discussed. It is based on the general RST (Robust Satellite Techniques) approach applied to Advanced Very High Resolution Radiometer (AVHRR) observations in the thermal infrared region of the electromagnetic spectrum. The proposed approach, which exploits the analysis of multi-temporal satellite records, seems to be able to detect the anomalous signals on the sea due to the oil polluted areas with excellent reliability (0% of false alarms) and good sensitivity in different observational conditions. Its performances have also been evaluated in comparison with another well-known AVHRR approach, analysing the spill event which happened during the Gulf War off the Kuwait and Saudi Arabia coasts in January 1991. The results confirm the reliability of the proposed approach which promises to offer new economically sustainable opportunities for building a near-real-time monitoring system for oil spills on a global scale. Moreover, in order to further assess the exportability of the proposed approach in different observational and environmental conditions, outcomes obtained by applying it to the Seki–Baynunah event affecting the Gulf of Oman in March 1994 are also shown.     

Nondestructive determination of nutritional information in oilseed rape leaves using visible/near infrared spectroscopy and multivariate calibrations

Nitrogen, phosphorus and potassium content are the three most important nutritional parameters for growing oilseed rape. We investigated visible and near infrared (Vis/NIR) spectroscopy combined with chemometrics for the fast and nondestructive determination of nutritional information in oilseed rape leaves. A total of 154 leaf samples were collected, with 104 randomly selected as the calibration set, and the remaining 50 samples used as the validation set. The performance of eight different preprocessing m...     

An infrared and visible image fusion algorithm based on ResNet-152

Multimedia Tools and Applications - The fusion of infrared and visible images can obtain a combined image with hidden objective and rich visible details. To improve the details of the fusion image...     

Estimation of nuclear power plants influence on the Baltic Sea thermal state by using infrared thermal satellite data

The influence of the activity of nuclear power plants on the state of the environment attracts constant attention from environmental and scientific organizations. The great amount of heated water thrown off permanently into a water basin by nuclear power plants is one of the negative factors, one which seriously disturbs the thermal balance of water basins and leads to irreversible environmental changes. The degree of algae growth in a basin depends on the amount of inflowing biogenic substances, as well as the water temperature of a reservoir. The main aim of this research was to develop the remote method of monitoring the influence of nuclear power plants on the thermal state of water basins and to compare it with different natural inputs of heat. The objects of this research were the operating nuclear power plants of the Baltic Sea rim.