Novel, fourier filtering method that reuses interference-patterned spectra to extend the calibration set for thickness determination

Determining the thickness of plastic sheets on the basis of near-infrared spectra by building a multivariate calibration model requires a relatively large sample set. In the thickness region, where just a few non-interference-patterned samples are available, it is a waste of information if interference-patterned spectra are excluded. After eliminating the interference pattern from the spectra (filtering), the calibration set can be extended with these filtered spectra. Fourier transformation of an interference-patterned spectrum versus wavenumber leads to a Fourier spectrum as a function of the optical path length containing an easily recognizable interference peak. Unfortunately, this peak coincides with components of the spectral information of absorbance, on which multivariate calibration is based. Hence, replacing the interference peak is a cardinal step of the filtering process. Since the Fourier spectrum versus optical path length function is not known, it has been shown that interpolated data over the remaining Fourier components can be substituted for the missing part of the spectrum. In this paper, a novel method is proposed that uses a linear approximation between the Fourier spectra and the thickness values so that the regression coefficients are calculated on components of all but the interference-patterned Fourier spectra and the corresponding thicknesses, and then the deleted components in the filtered spectrum are replaced. This latter method yields more detailed Fourier spectra. Reducing the disturbing effect of scattering is also discussed. The effectiveness of the filtering was tested on low-density polyethylene sheets. The performance of different calibration models with or without filtering was compared by significance tests on standard error of prediction values. Application of the new Fourier type filtering technique led to significant improvements in the calibration performance.     

New indicator for optimal preprocessing and wavelength selection of near-infrared spectra

Preprocessing of near-infrared spectra to remove unwanted, i.e., non-related spectral variation and selection of informative wavelengths is considered to be a crucial step prior to the construction of a quantitative calibration model. The standard methodology when comparing various preprocessing techniques and selecting different wavelengths is to compare prediction statistics computed with an independent set of data not used to make the actual calibration model. When the errors of reference value are large, no such values are available at all, or only a limited number of samples are available, other methods exist to evaluate the preprocessing method and wavelength selection. In this work we present a new indicator (SE) that only requires blank sample spectra, i.e., spectra of samples that are mixtures of the interfering constituents (everything except the analyte), a pure analyte spectrum, or alternatively, a sample spectrum where the analyte is present. The indicator is based on computing the net analyte signal of the analyte and the total error, i.e., instrumental noise and bias. By comparing the indicator values when different preprocessing techniques and wavelength selections are applied to the spectra, the optimal preprocessing technique and the optimal wavelength selection can be determined without knowledge of reference values, i.e., it minimizes the non-related spectral variation. The SE indicator is compared to two other indicators that also use net analyte signal computations. To demonstrate the feasibility of the SE indicator, two near-infrared spectral data sets from the pharmaceutical industry were used, i.e., diffuse reflectance spectra of powder samples and transmission spectra of tablets. Especially in pharmaceutical spectroscopic applications, it is expected beforehand that the non-related spectral variation is rather large and it is important to remove it. The indicator gave excellent results with respect to wavelength selection and optimal preprocessing. The SE indicator performs better than the two other indicators, and it is also applicable to other situations where the Beer-Lambert law is valid.     

Identification of chemical substances by their near-infrared spectra

Salamin, P.A., Cornelis, Y. and Bartels, H., 1988. Identification of chemical substances by their near-infrared spectra. Chemometrics and Intelligent Laboratory Systems, 3: 329–333.Near-infrared spectra show a large variability due to physical parameters such as particle size. This makes the identification of chemical substances by spectral comparison difficult. This article reviews an earlier method of identification of chemical substances by near-infrared spectra based on the Mahalanobis distance and introduces a new method based on the multiplicative scatter correction. This new method can to a great extent eliminate the spectral variation due to physical parameters and allows a plain comparison of two complete spectra.     

Quantitative analysis of cotton-polyester textile blends from near-infrared spectra

Quantitative analysis of textile blends and textile fabrics is currently of particular interest in the industrial context. In this frame, this work investigates whether the use of Fourier transform (FT) near-infrared (NIR) spectroscopy and chemometrics is powerful for rapid and accurate quantitative analysis of cotton-polyester content in blend products. As samples of the same composition have many sources of variability that affect NIR spectra, indirect prediction is particularly challenging and a large sample population is required to design robust calibration models. Thus, a total of more than three-hundred cotton-polyester samples were selected covering the range from the 0% to 100% cotton and the corresponding NIR reflectance spectra were measured on raw fabrics. The data set obtained was used to develop multivariate models for quantitative prediction from reference measurements. A successful approach was found to rely on partial least squares (PLS) regression combined with genetic algorithms (GAs) for wavelength selection. It involved evaluating a set of calibration models considering different spectral regions. The results obtained considering 27.5% of the original variables yielded a prediction error (RMSEP) of 2.3 in percent cotton content. It demonstrates that FT-NIR spectroscopy has the potential to be used in the textile industry for the prediction of the composition of cotton-polyester blends. As a further consequence, it was observed that the spectral preprocessing and the complexity of the model are simplified compared to the full-spectrum approach. Also, the relevancy of the spectral intervals retained after variable selection can be discussed.     

一种基于局部回归算法的多组分定量分析方法

文章针对复杂样本吸收光谱重叠或组分之间相互作用导致偏离朗伯一比尔定律的问题提出了一种以灰色综合关联度作为样本相似性判据的多组分定量分析局部回归建模方法,主要内容是对校正集样本的光谱曲线与待测样本曲线进行灰色综合关联度分析,然后以最小预测均方根误差原则选择与待测样本属性相近的样本组成校正子集,最后建立基于校正子集的偏最小二乘回归模型。相比马氏距离方法,灰色综合关联度结合了绝对位置差和变化率两方面因素,能够更为全面的反映样本之间的相似程度。建立实验系统将本方法应用于食用色素苋菜红、胭脂红、柠檬黄和日落黄混合溶液的定量分析中,实验结果表明,该方法优于全局建模方法,尤其在光谱响应与浓度之间的非线性响应段预测精度得到了明显的提升。     

A variable selection method based on uninformative variable elimination for multivariate calibration of near-infrared spectra

Variable (or wavelength) selection plays an important role in the quantitative analysis of near-infrared (NIR) spectra. A modified method of uninformative variable elimination (UVE) was proposed for variable selection in NIR spectral modeling based on the principle of Monte Carlo (MC) and UVE. The method builds a large number of models with randomly selected calibration samples at first, and then each variable is evaluated with a stability of the corresponding coefficients in these models. Variables with poor stability are known as uninformative variable and eliminated. The performance of the proposed method is compared with UVE-PLS and conventional PLS for modeling the NIR data sets of tobacco samples. Results show that the proposed method is able to select important wavelengths from the NIR spectra, and makes the prediction more robust and accurate in quantitative analysis. Furthermore, if wavelet compression is combined with the method, more parsimonious and efficient model can be obtained.     

Wavelength interval selection in multicomponent spectral analysis by moving window partial least-squares regression with applications to mid-infrared and near-infrared spectroscopic data

A new wavelength interval selection procedure, moving window partial least-squares regression (MWPLSR), is proposed for multicomponent spectral analysis. This procedure builds a series of PLS models in a window that moves over the whole spectral region and then locates useful spectral intervals in terms of the least complexity of PLS models reaching a desired error level. Based on a proposed theory demonstrating the necessity of wavelength selection, it is shown that MWPLSR provides a viable approach to eliminate the extra variability generated by non-composition-related factors such as the perturbations in experimental conditions and physical properties of samples. A salient advantage of MWPLSR is that the calibration model is very stable against the interference from non-composition-related factors. Moreover, the selection of spectral intervals in terms of the least model complexity enables the reduction of the size of a calibration sample set in calibration modeling. Two strategies are suggested for coupling the MWPLSR procedure with PLS for multicomponent spectral analysis: One is the inclusion of all selected intervals to develop a PLS calibration model, and the other is the combination of the PLS models built separately in each interval. The combination of multiple PLS models offers a novel potential tool for improving the performance of individual models. The proposed procedures are evaluated using two open-path Fourier transform infrared data sets and one near-infrared data set, each having different noise characteristics. The results reveal that the proposed procedures are very promising for vibrational spectroscopy-based multicomponent analyses and give much better prediction than the full-spectrum PLS modeling.     

Fourier transform infrared-attenuated total reflection nitrate determination of soil pastes using principal component regression, partial least squares, and cross-correlation

This paper investigates the use of Fourier transform infrared (FTIR) attenuated total reflectance (ATR) spectroscopy as a fast and simple way for direct determination of nitrate concentration in soil pastes, which would assist precision fertilizer placement and reduce nitrate pollution. Eight types of soils are investigated, with nitrate concentrations ranging from 0 to 1000 ppm-N. The spectral region around the nitrate band (1300-1550 cm(-1)) is analyzed by (1) principal component regression (PCR), (2) partial least squares (PLS), and (3) cross-correlation with reference libraries that include spectra of pure ions and/or soils. The main obstacle to accurate nitrate measurement appears to be an interfering band present in calcareous soils. This band, which may be due to carbonate, is located around 1450 cm(-1) and overlaps with the nitrate band centered around 1370 cm(-1). For non-calcareous soils, and in particular for light sandy agricultural soils, PLS and cross-correlation with a reference library containing only spectra of ions in water give similar results (about 8 ppm-N on dry soil basis), while PCR leads to slightly poorer results. When calcareous soils are included in the analysis, the prediction errors are about twice as large. In this case, the best results are obtained using PLS, followed by PCR, while cross-correlation with reference libraries leads to poorer results.     

Theoretical Justification of Wavelength Selection in PLS Calibration: Development of a New Algorithm

The mathematical basis of improved calibration through selection of informative variables for partial least-squares calibration has been identified. A theoretical investigation of calibration slopes indicates that including uninformative wavelengths negatively affect calibrations by producing both large relative bias toward zero and small additive bias away from the origin. These theoretical results are found regardless of the noise distribution in the data. Studies are performed to confirm this result using a previously used selection method compared to a new method, which is designed to perform more appropriately when dealing with data having large outlying points by including estimates of spectral residuals. Three different data sets are tested with varying noise distributions. In the first data set, Gaussian and log-normal noise was added to simulated data which included a single peak. Second, near-infrared spectra of glucose in cell culture media taken with an FT-IR spectrometer were analyzed. Finally, dispersive Raman Stokes spectra of glucose dissolved in water were assessed. In every case considered here, improved prediction is produced through selection, but data with different noise characteristics showed varying degrees of improvement depending on the selection method used. The practical results showed that, indeed, including residuals into ranking criteria improves selection for data with noise distributions resulting in large outliers. It was concluded that careful design of a selection algorithm should include consideration of spectral noise distributions in the input data to increase the likelihood of successful and appropriate selection.     

The effect of flow rate, accelerometer location and temperature in acoustic chemometrics on liquid flow: Spectral changes and robustness of the prediction models

Prediction of chemical composition of flowing liquids using passive acoustic measurements and multivariate regression (acoustic chemometrics) has been reported as a promising in-line measurement method. However, the passive acoustic measurement results are also affected directly or indirectly by other factors than composition of the liquid, i.e. physical conditions of the flow and equipment/pipe properties. The present study focuses on the effects of flow rate, accelerometer location and temperature on the acoustic spectra and prediction of composition of liquids. The studied liquids were two-component mixtures of sucrose and water, and three-component mixtures of ethanol, sucrose and water. Multivariate models were estimated using both local and global calibration on full spectra, and augmented frequency and amplitude matrices derived from full spectra. Flow rate and accelerometer location had the most pronounced effect on acoustic spectra and prediction results from recalibrated local models. Temperature had a minor effect on the acoustic spectra and prediction results. The prediction error for determination of ethanol, sucrose and water increased with increasing flow rate. Changes in flow rate resulted in considerable spectral variations, causing the resultant local calibration model to perform poorly predicting the new samples taken at other flow conditions. Global models performed well on prediction of liquid composition at all studied flow and temperature levels. The global models, however, needed higher number of PLS factors and led to higher prediction errors compared to local models. Using the augmented frequency and amplitude matrices in PLS/PPLS global regression models led to higher prediction errors compared to full spectra models. However, the augmented frequency and amplitude models were more parsimonious (4–6 PLS factors) compared to the full spectra models (10–12 PLS factors).     

Multivariate analysis and classification of the chemical quality of 7-aminocephalosporanic acid using near-infrared reflectance spectroscopy

The capability of near-infrared (NIR) spectroscopy in comparison to conventional chemical testing to control the chemical quality of a pharmaceutical intermediate has been investigated. Multivariate projection methods including principal component analysis, partial least-squares discriminant analysis and soft independent modeling of class analogy have been evaluated. 7-Aminocephalosporanic acid has been chosen as an example providing a large variation of quality due to its relative chemical instability. Three sets of production lots have been selected to study the extent of quality information extractable from NIR spectra. The first set of 91 lots covers a very broad range of chemical quality assessed by 8 parameters with a partially extended characterization by physical properties. The general congruence of spectral, chemical, and physical information has been investigated. The second set of 110 lots covers a very narrow range of chemical quality assessed by 11 parameters. With extended quality information, the intrinsic selectivity within the spectral data structure has been studied. The third set of 228 lots characterized by 8 parameters is a selection out of more than 1000 lots over a production period of two years. The ruggedness of the multivariate approach has been confirmed by a cross validation of the classification test.     

High-resolution absorption coefficient and refractive index spectra of carbon monoxide gas at millimeter and submillimeter wavelengths

The use of dispersive Fourier transform techniques in interferometry allows the measurement of absorption coefficient and refractive index spectra with great precision. This paper presents the absorption coefficient and refractive index spectra of carbon monoxide (CO) gas at millimeter and submillimeter wavelengths. To assess rotational lines of CO, the gas was measured at four different pressures, giving insight into the behavior of the spectral lines with varying parameters. The measurements in this study demonstrate that varying the pressure of the gas affects only the amplitude of the absorption lines and not their exact position. This is critical in air pollution studies when trying to single out a specific gas from a field sample with unknown constituents.     

In situ determination of growing stages and harvest time of tomato (Lycopersicon esculentum ) fruits using fiber-optic visible-near-infrared (Vis-NIR) spectroscopy

Nondestructive in situ measurement of tomato fruits is essential to determine growing stages and to assist in automatic picking of fruits. This study evaluates the applicability of visible and near-infrared (Vis-NIR) spectroscopy for in situ determination of growing stages and harvest time of three cultivars of tomato fruits. A mobile fiber-type AgroSpec Vis-NIR spectrophotometer (Tec5 Co., Germany) with a spectral range of 350-2200 nm was used to measure tomato spectra in reflection mode. A new growing stage (GS) index defined as the ratio of the current growing age in days to the on-vine duration before harvest in days was proposed. After dividing spectra into a calibration set (70%) and an independent prediction set (30%), spectra in the calibration set were subjected to a partial least squares regression (PLSR) with leave-one-out cross-validation to establish calibration models relating GS to the spectra of tomato fruits. Separate models were developed for each tomato cultivar and compared with a general model that used combined spectra of all three cultivars. The results show that PLSR based on the new GS is successful and robust in predicting the growing stages and harvest time of tomato fruits. Validation of calibration models on the independent prediction set indicates that successful prediction of GS can be achieved using the three models developed separately for each cultivar with a coefficient of determination (R(2)) of 0.91-0.92, root mean square error of prediction (RMSEP) of 0.081-0.097, and residual prediction deviation (RPD) of 3.29-3.70. General calibration using the combined spectra produces good prediction performance, although less accurate than that for the three individual cultivar models. The analysis of regression coefficient plots resulting from PLSR analysis indicates consistent assignment of important wavelengths for individual cultivar spectra and combined spectra. It is concluded that the Vis-NIR PLSR based on GS index can be adopted successfully for in situ determination of growing stages and harvest time of on-vine tomato fruits, which allows for automatic picking of fruits by a horticultural robot.     

Spectral simulation methodology for calibration transfer of near-infrared spectra

A spectrum simulation method is described for use in the development and transfer of multivariate calibration models from near-infrared spectra. By use of previously measured molar absorptivities and solvent displacement factors, synthetic calibration spectra are computed using only background spectra collected with the spectrometer for which a calibration model is desired. The resulting synthetic calibration set is used with partial least squares regression to form the calibration model. This methodology is demonstrated for use in the analysis of physiological levels of glucose (1-30 mM) in an aqueous matrix containing variable levels of alanine, ascorbate, lactate, urea, and triacetin. Experimentally measured data from two different Fourier transform spectrometers with different noise levels and stabilities are used to evaluate the simulation method. With the more stable instrument (A), well-performing calibration models are obtained, producing a standard error of prediction (SEP) of 0.70 mM. With the less stable instrument (B), the calibration based solely on synthetic spectra is less successful, producing an SEP value of 1.58 mM. For cases in which the synthetic spectra do not describe enough spectral variance, an augmentation protocol is evaluated in which the synthetic calibration spectra are augmented with the spectra of a small number of experimentally measured calibration samples. For instruments A and B, respectively, augmentation with measured spectra of nine samples lowers the SEP values to 0.64 and 0.85 mM.     

Attenuated total reflection infrared difference spectroscopy (ATR-IRDS) for quantitative reaction monitoring

Monitoring of chemical reactors is key to optimizing yield and efficiency of chemical transformation processes. Aside from tracking pressure and temperature, the measurement of the chemical composition is essential in this context. We present an infrared difference spectroscopy approach for determining the reactant (cyclooctene) and product (cyclooctane) concentrations during a catalytic hydrogenation reaction in the solvent cyclohexane, which is present in large excess. Subtracting the spectrum of the pure solvent from the reactor mixture spectra yields infrared (IR) spectra, which can ultimately be evaluated using a curve-fitting procedure based on spectral soft modeling. An important feature of our evaluation approach is that the calibration only requires recording the pure component spectra of the reactants, products, and solvent. Hence, no time-consuming preparation of mixtures for calibration is necessary. The IR concentration results are in good agreement with gas chromatography measurements.     

Enhancement of infrared spectral images for maximizing chemical information by minimizing baseline interferences

The popularity of spectral images in many areas of analysis has greatly increased during the last decade due to the development of charge-coupled device (CCD) and infrared sensitive cameras. Large amounts of spatial information can be obtained in short periods of time. The general goal in analytical chemistry is to convert spectral images into chemical images, which show the spatial locations of various chemical components. Self-modeling multivariate curve resolution methods can be used to extract pure component spectra from the mixture spectra in images and produce chemical images. However, there is a difficulty in processing infrared spectral images due to large pixel-to-pixel baseline variations. Herein, a method for minimizing baseline interferences using fast Fourier transform (FFT) filtering in both the spectral and spatial domains is discussed. The methodology is demonstrated on a microscopic sample of butter contaminated with non-pathogenic E. coli and on a cross-sectional sample of rabbit aorta containing plaque. The processing to reduce baseline effects improved the spatial resolution without compromising the spectral resolution.     

Evaluation techniques for two-way data from in situ fourier transform mid-infrared reaction monitoring in aqueous solution

A model chemical reaction was monitored with in situ Fourier transform mid-infrared spectroscopy using an attenuated total reflectance probe. The evaluation of the IR spectra is complicated by the fact that the reaction runs in nonisothermal aqueous solution with large variations in pH. Despite this, it was possible to extract large amounts of useful information on the reaction after suitable pretreatment of the spectra. Alternating least-squares (ALS) multivariate curve resolution is shown to be a useful technique for obtaining pure component spectra and concentrations if suitable spectral regions are analyzed. Rank mapping methods are used as the basis for this sectioning into smaller regions. Techniques for finding and analyzing selective spectral regions are also shown to be applicable to this type of data. Partial least-squares (PLS) regression models based on spectral data were used to verify the results where possible. The correlation between the concentrations predicted from PLS and ALS is excellent.     

Correcting attenuated total reflection-Fourier transform infrared spectra for water vapor and carbon dioxide

Fourier transform infrared (FT-IR) spectroscopy is a valuable technique for characterization of biological samples, providing a detailed fingerprint of the major chemical constituents. However, water vapor and CO(2) in the beam path often cause interferences in the spectra, which can hamper the data analysis and interpretation of results. In this paper we present a new method for removal of the spectral contributions due to atmospheric water and CO(2) from attenuated total reflection (ATR)-FT-IR spectra. In the IR spectrum, four separate wavenumber regions were defined, each containing an absorption band from either water vapor or CO(2). From two calibration data sets, gas model spectra were estimated in each of the four spectral regions, and these model spectra were applied for correction of gas absorptions in two independent test sets (spectra of aqueous solutions and a yeast biofilm (C. albicans) growing on an ATR crystal, respectively). The amounts of the atmospheric gases as expressed by the model spectra were estimated by regression, using second-derivative transformed spectra, and the estimated gas spectra could subsequently be subtracted from the sample spectra. For spectra of the growing yeast biofilm, the gas correction revealed otherwise hidden variations of relevance for modeling the growth dynamics. As the presented method improved the interpretation of the principle component analysis (PCA) models, it has proven to be a valuable tool for filtering atmospheric variation in ATR-FT-IR spectra.     

Application of pattern recognition to mass spectral data of toxic organic compounds in ambient air

The low resolution mass spectra of a set of 78 toxic, volatile organic compounds, the presence of which is routinely monitored in ambient air samples, were examined for information concerning chemical classes and compound identification. The mass spectra were converted to their autocorrelation spectra, and the transformed spectra were studied using pattern recognition techniques. The inherent structure of the data showed three major classes: nonhaloaromatics, chlorocarbons, and bromo- and bromochlorocarbons. Principal components models, with dimensionality obtained with a cross-validation technique, contained two or three dimensions. Classification accuracy for the training set compounds was high. A hierarchical classification scheme, using statistical pattern recognition and k-nearest neighbor methods, was developed for chemical class assignment and for compound identification from mass spectral data files. The method was applied to gas chromatography—mass spectral data of known compounds of calibration sets obtained on the air quality monitoring mass spectrometer. Classification accuracy as to chemical class was 87% and compound identification accuracy was 84% for the calibration data. A classification scheme, which could be the basis for a microcomputer based expert system for mass spectral interpretation, is described.     

Spectral variable selection for partial least squares calibration applied to authentication and quantification of extra virgin olive oils using Fourier transform Raman spectroscopy

The limits of quantitative multivariate assays for the analysis of extra virgin olive oil samples from various Greek sites adulterated by sunflower oil have been evaluated based on their Fourier transform (FT) Raman spectra. Different strategies for wavelength selection were tested for calculating optimal partial least squares (PLS) models. Compared to the full spectrum methods previously applied, the optimum standard error of prediction (SEP) for the sunflower oil concentrations in spiked olive oil samples could be significantly reduced. One efficient approach (PMMS, pair-wise minima and maxima selection) used a special variable selection strategy based on a pair-wise consideration of significant respective minima and maxima of PLS regression vectors, calculated for broad spectral intervals and a low number of PLS factors. PMMS provided robust calibration models with a small number of variables. On the other hand, the Tabu search strategy recently published (search process guided by restrictions leading to Tabu list) achieved lower SEP values but at the cost of extensive computing time when searching for a global minimum and less robust calibration models. Robustness was tested by using packages of ten and twenty randomly selected samples within cross-validation for calculating independent prediction values. The best SEP values for a one year's harvest with a total number of 66 Cretian samples were obtained by such spectral variable optimized PLS calibration models using leave-20-out cross-validation (values between 0.5 and 0.7% by weight). For the more complex population of olive oil samples from all over Greece (total number of 92 samples), results were between 0.7 and 0.9% by weight with a cross-validation sample package size of 20. Notably, the calibration method with Tabu variable selection has been shown to be a valid chemometric approach by which a single model can be applied with a low SEP of 1.4% for olive oil samples across three different harvest years.