Forecasting of traffic origin NO and NO2 concentrations by Support Vector Machines and neural networks using Principal Component Analysis

The main aim of this paper is to predict NO and NO₂ concentrations four days in advance comparing two artificial intelligence learning methods, namely, Multi-Layer Perceptron and Support Vector Machines on two kinds of spatial embedding of the temporal time series. Hourly values of NO and NO₂ concentrations, as well as meteorological variables were recorded in a cross-road monitoring station with heavy traffic in Szeged in order to build a model for predicting NO and NO₂ concentrations several hours in advance. The prediction of NO and NO₂ concentrations was performed partly on the basis of their past values, and partly on the basis of temperature, humidity and wind speed data. Since NO can be predicted more accurately, its values were considered primarily when forecasting NO₂. Time series prediction can be interpreted in a way that is suitable for artificial intelligence learning. Two effective learning methods, namely, Multi-Layer Perceptron and Support Vector Regression are used to provide efficient non-linear models for NO and NO₂ times series predictions. Multi-Layer Perceptron is widely used to predict these time series, but Support Vector Regression has not yet been applied for predicting NO and NO₂ concentrations. Grid search is applied to select the best parameters for the learners. To get rid of the curse of dimensionality of the spatial embedding of the time series Principal Component Analysis is taken to reduce the dimension of the embedded data. Three commonly used linear algorithms were considered as references: one-day persistence, average of several-day persistence and linear regression. Based on the good results of the average of several-day persistence, a prediction scheme was introduced, which forms weighted averages instead of simple ones. The optimization of these weights was performed with linear regression in linear case and with the learning methods mentioned in non-linear case. Concerning the NO predictions, the non-linear learning methods give significantly better predictions than the reference linear methods. In the case of NO₂ the improvement of the prediction is considerable; however, it is less notable than for NO.     

Predicting bank financial failures using neural networks,support vector machines and multivariate statistical methods: A comparative analysis in the sample of savings deposit insurance fund (SDIF) transferred banks in Turkey

Bank failures threaten the economic system as a whole. Therefore, predicting bank financial failures is crucial to prevent and/or lessen the incoming negative effects on the economic system. This is originally a classification problem to categorize banks as healthy or non-healthy ones. This study aims to apply various neural network techniques, support vector machines and multivariate statistical methods to the bank failure prediction problem in a Turkish case, and to present a comprehensive computational comparison of the classification performances of the techniques tested. Twenty financial ratios with six feature groups including capital adequacy, asset quality, management quality, earnings, liquidity and sensitivity to market risk (CAMELS) are selected as predictor variables in the study. Four different data sets with different characteristics are developed using official financial data to improve the prediction performance. Each data set is also divided into training and validation sets. In the category of neural networks, four different architectures namely multi-layer perceptron, competitive learning, self-organizing map and learning vector quantization are employed. The multivariate statistical methods; multivariate discriminant analysis, k-means cluster analysis and logistic regression analysis are tested. Experimental results are evaluated with respect to the correct accuracy performance of techniques. Results show that multi-layer perceptron and learning vector quantization can be considered as the most successful models in predicting the financial failure of banks.     

一种基于感知器的样本空间划分方法

二类分类问题是机器学习中的最基本的一类重要问题.目前广泛使用的,也是最为有效的学习算法是支持向量机 (SVM).然而对于某些非线性分类问题,SVM 还不能给出令人满意的解,因此希望能找到一种方法对 SVM 解决非线性分类问题的能力加以改进.对二类分类问题,提出一种基于感知器的样本空间划分方法.该方法首先用感知器提取样本的分布信息,将整体问题划分为局部空间中的分类问题,而后使用 SVM 求出各个局部问题的最优分界面,并用最小最大模块化网络对局部分界面进行综合,得到问题的全局解.仿真实验表明,新方法能够有效地分析样本空间,提取样本分布信息,在测试数据上得到了比原有方法更好的准确率.新方法实现了预期的目标,提高了分类器处理非线性分类问题的能力.     

基于模糊时序和支持向量机的高速公路SO2浓度预测算法

针对现有SO₂浓度预测方法中存在的污染物来源和影响因素认识不统一、小样本数据敏感、易于陷入局部最优等问题,文中提出了基于模糊时序和支持向量机的高速公路SO₂浓度预测算法,为搭建高速公路环境健康监测系统提供了可靠的理论支持.该方法依据SO₂浓度的季节变动规律,以季节作为时间序列,以24h为粒化窗宽,通过高斯核函数提取原始样本数据的特征值,输入支持向量机训练模型,并利用k重交叉验证法结合网格划分优化模型参数.文中应用该方法建立了SO₂浓度预测模型,并以2014年4月至2015年3月山西省太旧高速公路某监测点SO₂小时浓度监测值为样本数据,在MATLAB平台下应用LIBSVM工具实现了计算过程.结果表明,基于模糊时序和支持向量机的高速公路SO₂浓度预测算法不受机理性理论研究的限制,支持小样本学习,非线性拟合效果好,泛化能力强.     

Air pollution forecasting based on attention-based LSTM neural network and ensemble learning

With air pollution having become a global concern, scientists are committed to working on its amelioration. In the field of air pollution prediction, there have been good results in experimental research so far, but few studies have integrated weather forecast information and the properties of air pollution drift. In this work, we propose a novel wind-sensitive attention mechanism with a long short-term memory (LSTM) neural network model to predict the air pollution - PM2.5 concentrations by considering the influence of wind direction and speed on the changes of spatial–temporal PM2.5 concentrations in neighbouring areas. Preliminary predictions for PM2.5 are then made by an LSTM neural network regarding neighbouring pollution; these predictions are “paid attention to” and we finally apply an ensemble learning method based on e X treme G radient B oosting (XGBoost) to combine the preliminary predictions with weather forecasting to make second phase predictions of PM2.5. The experiment is conducted using PM2.5 data and weather forecast data. Our results illustrate that the proposed method is superior to other methods in predicting PM2.5 concentrations, including multi-layer perceptron, support vector regression, LSTM neural network, and extreme gradient boosting algorithm.     

Real-time prediction of order flowtimes using support vector regression

In a make-to-order production system, a due date must be assigned to new orders that arrive dynamically, which requires predicting the order flowtime in real-time. This study develops a support vector regression model for real-time flowtime prediction in multi-resource, multi-product systems. Several combinations of kernel and loss functions are examined, and results indicate that the linear kernel and the ε$\epsilon$-insensitive loss function yield the best generalization performance. The prediction error of the support vector regression model for three different multi-resource systems of varying complexity is compared to that of classic time series models (exponential smoothing and moving average) and to a feedforward artificial neural network. Results show that the support vector regression model has lower flowtime prediction error and is more robust. More accurately predicting flowtime using support vector regression will improve due-date performance and reduce expenses in make-to-order production environments.     

Principal component and multiple regression analysis in modelling of ground-level ozone and factors affecting its concentrations

Data on the concentrations of seven environmental pollutants (CH₄, NMHC, CO, CO₂, NO, NO₂ and SO₂) and meteorological variables (wind speed and direction, air temperature, relative humidity and solar radiation) were employed to predict the concentration of ozone in the atmosphere using both multiple linear and principal component regression methods. Separate analyses were carried out for day light and night time periods. For both periods the pollutants were highly correlated, but were all negatively correlated with ozone. Multiple regression analysis was used to fit the ozone data using the pollutant and meteorological variables as predictors. A variable selection method based on high loadings of varimax rotated principal components was used to obtain subsets of the predictor variables to be included in the regression model of the logarithm of the ozone data. It was found that while high temperature and high solar energy tended to increase the day time ozone concentrations, the pollutants NO and SO₂ being emitted to the atmosphere were being depleted. Night time ozone concentrations were influenced predominantly by the nitrogen oxides (NO+NO₂), with the meteorological variables playing no significant role. However, the model did not predict the night time ozone concentrations as accurately as it did for the day time. This could be due to other factors that were not explicitly considered in this study.     

Impacts of urban form on air quality in metropolitan areas in the United States

Among 221 metropolitan areas (MAs) in the United States (US), this study explored the impact of urban form, either urban compactness or urban sprawl, on two types of air quality in 2014: NO_x emissions from road traffic and annual average NO2 concentrations. Urban form was quantified using Smart Growth America (SGA) sprawl indexes with density, land use mixing, centeredness, and street connectivity. NO_x emissions from road traffic were derived from the National Emissions Inventory (NEI). Through modeling NO2 concentrations using land use regression (LUR), with satellite-based estimates and kriging, this study measured NO₂ concentrations within MAs in the US The study results showed that higher levels of urban form scores (i.e., higher compactness) and land use mixing were associated with lower per-person NO_x emissions from road traffic. In addition, higher levels of centeredness were associated with lower NO₂ concentrations, but the effect was moderate. On the other hand, regional rainfall and solar insolation had more significant associations with NO₂ concentrations than metropolitan urban form. Meanwhile, localized emissions sources had significant associations with local-level NO₂ concentrations. This study provides additional evidence on the relationship between urban form and air quality in the US MAs. The study suggests that high compactness-oriented development and the reduction of localized emission sources may be effective in reducing NO_x emissions from road traffic and local NO₂ concentrations, respectively. However, future studies need to explore the impact of urban form at both the MA and local levels on NO₂ concentrations and develop a more accurate national NO₂ concentration prediction model.     

稀疏贝叶斯模型与相关向量机学习研究

虽然支持向量机在模式识别的相关领域得到了广泛应用,但它自身固有许多不足之处.相关向量机是在稀疏贝叶斯框架下提出的稀疏模型,模型没有规则化系数,核函数不要求满足Mercer条件.相关向量机不仅具备良好的泛化能力,而且还能够得到具有统计意义的预测结果.首先介绍了稀疏贝叶斯回归和分类模型,通过参数推断过程,将相关向量机学习转化为最大化边缘似然函数估计,并分析了3种估计方法,给出了快速序列稀疏贝叶斯学习算法流程.     

Multivariate adaptive regression spline (MARS) and least squares support vector machine (LSSVM) for OCR prediction

This article investigates the feasibility of multivariate adaptive regression spline (MARS) and least squares support vector machine (LSSVM) for the prediction of over consolidation ratio (OCR) of clay deposits based on Piezocone Penetration Tests (PCPT) data. MARS uses piece-wise linear segments to describe the non-linear relationships between input and output variables. LSSVM is firmly based on the theory of statistical learning, and uses regression technique. The input parameters of the models are corrected cone resistance (q _t ), vertical total stress (σ_v), hydrostatic pore pressure (u ₀), pore pressure at the cone tip (u ₁), and the pore pressure just above the cone base (u ₂). The developed LSSVM model gives error bar of predicted OCR. Equations have also been developed for prediction of OCR. The performance of MARS and LSSVM models has been compared with the traditional methods for OCR prediction. As the results reveal, the proposed MARS and LSSVM models are robust models for determination of OCR.     

支持向量机回归在线建模及应用

支持向量机（SVM）回归理论与神经网络等非线性回归理论相比具有许多独特的优点，讨论了建模中SVM核函数，损失函数的选取和容量控制等问题，并用实验加以验证，将SVM回归动态建模理论应用于非线性，时变，大时延温室环境温度变化的建模和预测，模型简单，预测效果好。     

Retrieval of oceanic chlorophyll concentration with relevance vector machines

In this communication, we evaluate the performance of the relevance vector machine (RVM) for the estimation of biophysical parameters from remote sensing data. For illustration purposes, we focus on the estimation of chlorophyll-a concentrations from remote sensing reflectance just above the ocean surface. A variety of bio-optical algorithms have been developed to relate measurements of ocean radiance to in situ concentrations of phytoplankton pigments, and ultimately most of these algorithms demonstrate the potential of quantifying chlorophyll-a concentrations accurately from multispectral satellite ocean color data. Both satellite-derived data and in situ measurements are subject to high levels of uncertainty. In this context, robust and stable non-linear regression methods that provide inverse models are desirable.Lately, the use of the support vector regression (SVR) has produced good results in inversion problems, improving state-of-the-art neural networks. However, the SVR has some deficiencies, which could be theoretically alleviated by the RVM. In this paper, performance of the RVM is evaluated in terms of accuracy and bias of the estimations, sparseness of the solutions, robustness to low number of training samples, and computational burden. In addition, some theoretical issues are discussed, such as the sensitivity to training parameters setting, kernel selection, and confidence intervals on the predictions.Results suggest that RVMs offer an excellent trade-off between accuracy and sparsity of the solution, and become less sensitive to the selection of the free parameters. A novel formulation of the RVM that incorporates prior knowledge of the problem is presented and successfully tested, providing better results than standard RVM formulations, SVRs, neural networks, and classical bio-optical models for SeaWIFS, such as Morel, CalCOFI and OC2/OC4 models.     

基于最小二乘支持向量回归的水质预测

水质系统是一个开放的、复杂的、非线性动力学系统,具有时变复杂性,针对水质预测方法的研究虽然已经取得了一些成果,但也存在预测精度与计算复杂度等难题。为此,本文提出一种基于最小二乘支持向量回归的水质预测算法。支持向量机是机器学习中一种常用的分类模型,通过核函数将非线性数据从低维映射到高维空间,在高维空间实现线性分类和回归,最小二乘支持向量回归（LS-SVR）利用所有的样本参与回归拟合,使得回归的损失函数不再只与小部分支持向量样本有关,而是由所有样本参与学习修正误差,提高预测精度;同时该算法将标准SVR求解问题由不等式的约束条件及凸二次规划问题转化成线性方程组来求解,提高了运算速度,解决了非线性复杂特性的水质预测问题。     

Parameter-insensitive kernel in extreme learning for non-linear support vector regression

Support vector regression (SVR) is a state-of-the-art method for regression which uses the ε‐sensitive loss and produces sparse models. However, non-linear SVRs are difficult to tune because of the additional kernel parameter. In this paper, a new parameter-insensitive kernel inspired from extreme learning is used for non-linear SVR. Hence, the practitioner has only two meta-parameters to optimise. The proposed approach reduces significantly the computational complexity yet experiments show that it yields performances that are very close from the state-of-the-art. Unlike previous works which rely on Monte-Carlo approximation to estimate the kernel, this work also shows that the proposed kernel has an analytic form which is computationally easier to evaluate.     

Determination of the composition of NO2 and NO mixture by thin film sensor and back-propagation network

In this work, vacuum deposited thin films of lead phthalocyanine (PbPc) were employed as gas sensor to detect mixtures of NO₂ and NO. Data collected from sensor responses were used to train a two-stage back-propagation network (BPN) for the determination of the gas mixture composition. The success rate for the two-stage BPN in identifying categories of gas mixture approaches 100%. The maximum error for the two-stage BPN in predicting NO concentration in a gas mixture with fixed NO₂ concentration is 14.7%.     

Development of a land use regression model for daily NO2 and NOx concentrations in the Brisbane metropolitan area,Australia

Land use regression models are an established method for estimating spatial variability in gaseous pollutant levels across urban areas. Existing LUR models have been developed to predict annual average concentrations of airborne pollutants. None of those models have been developed to predict daily average concentrations, which are useful in health studies focused on the acute impacts of air pollution. In this study, we developed LUR models to predict daily NO₂ and NO_x concentrations during 2009–2012 in the Brisbane Metropolitan Area (BMA), Australia's third-largest city. The final models explained 64% and 70% of spatial variability in NO₂ and NO_x, respectively, with leave-one-out-cross-validation R² of 3–49% and 2–51%. Distance to major road and industrial area were the common predictor variables for both NO₂ and NO_x, suggesting an important role for road traffic and industrial emissions. The novel modeling approach adopted here can be applied in other urban locations in epidemiological studies.     

Extracting the main patterns of natural time series for long-term neurofuzzy prediction

A combination of singular spectrum analysis and locally linear neurofuzzy modeling technique is proposed to make accurate long-term prediction of natural phenomena. The principal components (PCs) obtained from spectral analysis have narrow band frequency spectra and definite linear or nonlinear trends and periodic patterns; hence they are predictable in large prediction horizon. The incremental learning algorithm initiates a model for each of the components as an optimal linear least squares estimation, and adds the nonlinear neurons if they help to reduce error indices over training and validation sets. Therefore, the algorithm automatically constructs the best linear or nonlinear model for each of the PCs to achieve maximum generalization, and the long-term prediction of the original time series is obtained by recombining the predicted components. The proposed method has been primarily tested in long-term prediction of some well-known nonlinear time series obtained from Mackey–Glass, Lorenz, and Ikeda map chaotic systems, and the results have been compared to the predictions made by multi-layered perceptron (MLP) and radial basis functions (RBF) networks. As a real world case study, the method has been applied to the long-term prediction of solar activity where the results have been compared to the long-term predictions of physical precursor and solar dynamo methods.     

Optimal feature selection for support vector machines

Selecting relevant features for support vector machine (SVM) classifiers is important for a variety of reasons such as generalization performance, computational efficiency, and feature interpretability. Traditional SVM approaches to feature selection typically extract features and learn SVM parameters independently. Independently performing these two steps might result in a loss of information related to the classification process. This paper proposes a convex energy-based framework to jointly perform feature selection and SVM parameter learning for linear and non-linear kernels. Experiments on various databases show significant reduction of features used while maintaining classification performance.     

Prediction of a Lorenz chaotic attractor using two-layer perceptron neural network

This paper investigates the prediction of a Lorenz chaotic attractor having relatively high values of Lypunov's exponents. The characteristic of this time series is its rich chaotic behavior. For such dynamic reconstruction problem, regularized radial basis function (RBF) neural network (NN) models have been widely employed in the literature. However, author recommends using a two-layer multi-layer perceptron (MLP) NN-based recurrent model. When none of the available linear models have been able to learn the dynamics of this attractor, it is shown that the proposed NN-based auto regressive (AR) and auto regressive moving average (ARMA) models with regularization have not only learned the true trajectory of this attractor, but also performed much better in multi-step-ahead predictions. However, equivalent linear models seem to fail miserably in learning the dynamics of the time series, despite the low values of Akaike's final prediction error (FPE) estimate. Author proposes to employ the recurrent NN-based ARMA model with regularization which clearly outperforms all other models and thus, it is possible to obtain good results for prediction and reconstruction of the dynamics of the chaotic time series with NN-based models.     

Influences of variable scales and activation functions on the performances of multilayer feedforward neural networks

This paper gives insight into the methods about how to improve the learning capabilities of multilayer feedforward neural networks with linear basis functions in the case of limited number of patterns according to the basic principles of support vector machine (SVM), namely, about how to get the optimal separating hyperplanes. And furthermore, this paper analyses the characteristics of sigmoid-type activation functions, and investigates the influences of absolute sizes of variables on the convergence rate, classification ability and non-linear fitting accuracy of multilayer feedforward networks, and presents the way of how to select suitable activation functions. As a result, this proposed method effectively enhances the learning abilities of multilayer feedforward neural networks by introducing the sum-of-squares weight term into the networks’ error functions and appropriately enlarging the variable components with the help of the SVM theory. Finally, the effectiveness of the proposed method is verified through three classification examples as well as a non-linear mapping one.