Forecasting foreign exchange rates using kernel methods

First, the all-important no free lunch theorems are introduced. Next, kernel methods, support vector machines (SVMs), preprocessing, model selection, feature selection, SVM software and the Fisher kernel are introduced and discussed. A hidden Markov model is trained on foreign exchange data to derive a Fisher kernel for an SVM, the DC algorithm and the Bayes point machine (BPM) are also used to learn the kernel on foreign exchange data. Further, the DC algorithm was used to learn the parameters of the hidden Markov model in the Fisher kernel, creating a hybrid algorithm. The mean net returns were positive for BPM; and BPM, the Fisher kernel, the DC algorithm and the hybrid algorithm were all improvements over a standard SVM in terms of both gross returns and net returns, but none achieved net returns as high as the genetic programming approach employed by Neely, Weller, and Dittmar (1997) and published in Neely, Weller, and Ulrich (2009). Two implementations of SVMs for Windows with semi-automated parameter selection are built.     

Bayesian adaptive bandwidth kernel density estimation of irregular multivariate distributions

In this paper, we propose a new methodology for multivariate kernel density estimation in which data are categorized into low- and high-density regions as an underlying mechanism for assigning adaptive bandwidths. We derive the posterior density of the bandwidth parameters via the Kullback-Leibler divergence criterion and use a Markov chain Monte Carlo (MCMC) sampling algorithm to estimate the adaptive bandwidths. The resulting estimator is referred to as the tail-adaptive density estimator. Monte Carlo simulation results show that the tail-adaptive density estimator outperforms the global-bandwidth density estimators implemented using different global bandwidth selection rules. The inferential potential of the tail-adaptive density estimator is demonstrated by employing the estimator to estimate the bivariate density of daily index returns observed from the USA and Australian stock markets.     

Stock price forecast using Bayesian network

Bayesian network is a probabilistic graphical model that represents a set of random variables and their conditional dependencies via a directed acyclic graph. This paper describes the price earnings ratio (P/E ratio) forecast by using Bayesian network. Firstly, the use of clustering algorithm transforms the continuous P/E ratio to the set of digitized values. The Bayesian network for the P/E ratio forecast is determined from the set of the digitized values. NIKKEI stock average (NIKKEI225) and Toyota motor corporation stock price are considered as numerical examples. The results show that the forecast accuracy of the present algorithm is better than that of the traditional time-series forecast algorithms in comparison of their correlation coefficient and the root mean square error.     

Deconvolution with partially known kernel of nonnegative signals

This paper extends the Bayesian direct deconvolution method to the case in which the convolution kernel depends on few unknown nuisance parameters. Moreover, an acceleration procedure is proposed that drastically reduces the computational burden. Finally, the implementation of the method by means of the fast Fourier transform is fully discussed in the multidimensional case.     

Bayesian function learning using MCMC methods

The paper deals with the problem of reconstructing a continuous 1D function from discrete noisy samples. The measurements may also be indirect in the sense that the samples may be the output of a linear operator applied to the function. Bayesian estimation provides a unified treatment of this class of problems. We show that a rigorous Bayesian solution can be efficiently implemented by resorting to a Markov chain Monte Carlo (MCMC) simulation scheme. In particular, we discuss how the structure of the problem can be exploited in order to improve the computational and convergence performances. The effectiveness of the proposed scheme is demonstrated on two classical benchmark problems as well as on the analysis of IVGTT (IntraVenous glucose tolerance test) data, a complex identification-deconvolution problem concerning the estimation of the insulin secretion rate following the administration of an intravenous glucose injection     

Nonlinear nonnegative matrix factorization based on Mercer kernel construction

Generalizations ofnonnegative matrix factorization (NMF) in kernel feature space, such as projected gradient kernel NMF (PGKNMF) and polynomial Kernel NMF (PNMF), have been developed for face and facial expression recognition recently. However, these existing kernel NMF approaches cannot guarantee the nonnegativity of bases in kernel feature space and thus are essentially semi-NMF methods. In this paper, we show that nonlinear semi-NMF cannot extract the localized components which offer important information in object recognition. Therefore, nonlinear NMF rather than semi-NMF is needed to be developed for extracting localized component as well as learning the nonlinear structure. In order to address the nonlinear problem of NMF and the semi-nonnegative problem of the existing kernel NMF methods, we develop the nonlinear NMF based on a self-constructed Mercer kernel which preserves the nonnegative constraints on both bases and coefficients in kernel feature space. Experimental results in face and expressing recognition show that the proposed approach outperforms the existing state-of-the-art kernel methods, such as KPCA, GDA, PNMF and PGKNMF.     

Bayesian predictive kernel discriminant analysis

Discriminant analysis using Kernel Density Estimator (KDE) is a common tool for classification, but depends on the choice of the bandwidth or smoothing parameter of kernel. In this paper, we introduce a Bayesian Predictive Kernel Discriminant Analysis (BPKDA) eliminating this dependence by integrating the KDE with respect to an appropriate prior probability distribution for the bandwidth. Keypoints of the method are: (1) the formulation of the classification rule in terms of mixture predictive densities obtained by integrating kernel; (2) use of Independent Components Analysis (ICA) to choose a transform matrix so that transformed components are as independent as possible; and (3) nonparametric estimation of the predictive density by KDE for each independent component. Results on benchmark data sets and simulations show that the performance of BPKDA is competitive with, and in some cases significantly better than, Linear Discriminant Analysis (LDA), Quadratic Discriminant Analysis (QDA) and Naives Bayes discriminant Analysis with normal distribution (NNBDA).     

Optimized Bayesian adaptive resonance theory mapping model using a rational quadratic kernel and Bayesian quadratic regularization

Applied Intelligence - Bayesian adaptive resonance theory (ART) and ARTMAP-based neural network classifier (known as BAM) are widely used and achieve good classification performance when solving...     

基于核非负矩阵分解的有向图聚类算法

现有的有向图聚类算法大多基于向量空间中节点间的近似线性关系假设,忽略了节点间存在的非线性相关性。针对该问题,提出一种基于核非负矩阵分解（KNMF）的有向图聚类算法。首先,引入核学习方法将有向图的邻接矩阵投影到核空间,并通过特定的正则项约束原空间及核空间中节点间的相似性。其次,提出了图正则化核非对称NMF算法的目标函数,并在非负约束条件下通过梯度下降方法推导出一个聚类算法。该算法在考虑节点连边的方向性的同时利用核学习方法建模节点间的非线性关系,从而准确地揭示有向图中潜在的结构信息。最后,在专利-引文网络（PCN）数据集上的实验结果表明,簇的数目为2时,和对比算法相比,所提算法将DB值和DQF值分别提高了约0.25和8%,取得了更好的聚类质量。     

Automating the construction of CBR systems using kernel methods

Instance-based reasoning systems and, in general, case-based reasoning systems are normally used in problems for which it is difficult to define rules. Although case-based reasoning methods have proved their ability to solve different types of problems, there is still a demand for methods that facilitate their automation during their creation and the retrieval and reuse stages of their reasoning circle. This paper presents one method based on kernels, which can be used to automate some of the reasoning steps of instance-based reasoning systems. Kernels were originally derived in the context of support vector machines, which identify the smallest number of data points necessary to solve a particular problem (e.g., regression or classification). Unsupervised kernel methods have been used successfully to identify the optimal instances to instantiate an instance base. The efficiency of the kernel model is shown on an oceanographic problem. © 2001 John Wiley & Sons, Inc.     

Forecasting volatility of oil price using an artificial neural network-GARCH model

This paper builds on previous research and seeks to determine whether improvements can be achieved in the forecasting of oil price volatility by using a hybrid model and incorporating financial variables. The main conclusion is that the hybrid model increases the volatility forecasting precision by 30% over previous models as measured by a heteroscedasticity-adjusted mean squared error (HMSE) model. Key financial variables included in the model that improved the prediction are the Euro/Dollar and Yen/Dollar exchange rates, and the DJIA and FTSE stock market indexes.     

Online option price forecasting by using unscented Kalman filters and support vector machines

This study develops a hybrid model that combines unscented Kalman filters (UKFs) and support vector machines (SVMs) to implement an online option price predictor. In the hybrid model, the UKF is used to infer latent variables and make a prediction based on the Black–Scholes formula, while the SVM is employed to model the nonlinear residuals between the actual option prices and the UKF predictions. Taking option data traded in Taiwan Futures Exchange, this study examined the forecasting accuracy of the proposed model, and found that the new hybrid model is superior to pure SVM models or hybrid neural network models in terms of three types of options. This model can help investors for reducing their risk in online trading.     

Bayesian analysis of robust Poisson geometric process model using heavy-tailed distributions

We propose a robust Poisson geometric process model with heavy-tailed distributions to cope with the problem of outliers as it may lead to an overestimation of mean and variance resulting in inaccurate interpretations of the situations. Two heavy-tailed distributions namely Student’s t and exponential power distributions with different tailednesses and kurtoses are used and they are represented in scale mixture of normal and scale mixture of uniform respectively. The proposed model is capable of describing the trend and meanwhile the mixing parameters in the scale mixture representations can detect the outlying observations. Simulations and real data analysis are performed to investigate the properties of the models.     

Model-based kernel sum rule: kernel Bayesian inference with probabilistic models

Machine Learning - Kernel Bayesian inference is a principled approach to nonparametric inference in probabilistic graphical models, where probabilistic relationships between variables are learned...     

ECG biometric authentication based on non-fiducial approach using kernel methods

Identity recognition faces several challenges especially in extracting an individual's unique features from biometric modalities and pattern classifications. Electrocardiogram (ECG) waveforms, for instance, have unique identity properties for human recognition, and their signals are not periodic. At present, in order to generate a significant ECG feature set, non-fiducial methodologies based on an autocorrelation (AC) in conjunction with linear dimension reduction methods are used. This paper proposes a new non-fiducial framework for ECG biometric verification using kernel methods to reduce both high autocorrelation vectors' dimensionality and recognition system after denoising signals of 52 subjects with Discrete Wavelet Transform (DWT). The effects of different dimensionality reduction techniques for use in feature extraction were investigated to evaluate verification performance rates of a multi-class Support Vector Machine (SVM) with the One-Against-All (OAA) approach. The experimental results demonstrated higher test recognition rates of Gaussian OAA SVMs on random unknown ECG data sets with the use of the Kernel Principal Component Analysis (KPCA) as compared to the use of the Linear Discriminant Analysis (LDA) and Principal Component Analysis (PCA).     

基于加权核非负矩阵分解的短文本聚类算法

对互联网产生的大量短文本进行聚类分析具有重要的应用价值,但由于短文本存在特征稀疏和特征难以提取的问题,导致传统的文本聚类算法难以有效处理该问题。为了解决该问题,利用非负矩阵分解（NMF）模型提出基于加权核非负矩阵分解（WKNMF）的短文本聚类算法。该算法通过核方法的映射关系将稀疏特征空间映射到高维隐性空间,从而可以充分利用短文本中的隐性语义特征进行聚类;另外,利用核技巧简化高维数据的复杂运算,并通过迭代更新规则不断地动态调整短文本的权重向量,从而可以区分不同短文本对聚类的重要性。在真实的微博数据集上进行了相关实验,结果表明WKNMF算法比K均值、隐含狄利克雷分布（LDA）、NMF和自组织神经网络（SOM）具有更好的聚类质量,准确度和归一化互信息分别达到了66.38%和66.91%。     

Projected gradient methods for nonnegative matrix factorization

Nonnegative matrix factorization (NMF) can be formulated as a minimization problem with bound constraints. Although bound-constrained optimization has been studied extensively in both theory and practice, so far no study has formally applied its techniques to NMF. In this letter, we propose two projected gradient methods for NMF, both of which exhibit strong optimization properties. We discuss efficient implementations and demonstrate that one of the proposed methods converges faster than the popular multiplicative update approach. A simple Matlab code is also provided.