Kernel dictionary learning based discriminant analysis

Sparse representation based classification (SRC) has been successfully applied in many applications. But how to determine appropriate features that can best work with SRC remains an open question. Dictionary learning (DL) has played an import role in the success of sparse representation, while SRC treats the entire training set as a structured dictionary. In addition, as a linear algorithm, SRC cannot handle the data with highly nonlinear distribution. Motivated by these concerns, in this paper, we propose a novel feature learning method (termed kernel dictionary learning based discriminant analysis, KDL-DA). The proposed algorithm aims at learning a projection matrix and a kernel dictionary simultaneously such that in the reduced space the sparse representation of the data can be easily obtained, and the reconstruction residual can be further reduced. Thus, KDL-DA can achieve better performances in the projected space. Extensive experimental results show that our method outperforms many state-of-the-art methods.     

Nonparametric Bayesian dictionary learning algorithm based on structural similarity

Though nonparametric Bayesian methods possesses significant superiority with respect to traditional comprehensive dictionary learning methods,there is room for improvement of this method as it needs more consideration over the structural similarity and variability of images.To solve this problem,a nonparametric Bayesian dictionary learning algorithm based on structural similarity was proposed.The algorithm improved the structural representing ability of dictionaries by clustering images according to their non-local structural similarity and introducing block structure into sparse representing of images.Denoising and compressed sensing experiments showed that the proposed algorithm performs better than several current popular unsupervised dictionary learning algorithms.     

基于贝叶斯网络工具箱的贝叶斯学习和推理

采用MATLAB语言编制的贝叶斯网络工具箱(Bayesian Networks Toolbox,BNT)可实现贝叶斯网络结构学习、参数学习、推理和构建贝叶斯分类器,此工具箱在贝叶斯学习编程方面非常灵活.介绍了用贝叶斯网络工具箱解决贝叶斯学习和推理问题,并给出了两个实例.     

Context-aware joint dictionary learning for color image demosaicking

Most digital cameras are overlaid with color filter arrays (CFA) on their electronic sensors, and thus only one particular color value would be captured at every pixel location. When producing the output image, one needs to recover the full color image from such incomplete color samples, and this process is known as demosaicking. In this paper, we propose a novel context-constrained demosaicking algorithm via sparse-representation based joint dictionary learning. Given a single mosaicked image with incomplete color samples, we perform color and texture constrained image segmentation and learn a dictionary with different context categories. A joint sparse representation is employed on different image components for predicting the missing color information in the resulting high-resolution image. During the dictionary learning and sparse coding processes, we advocate a locality constraint in our algorithm, which allows us to locate most relevant image data and thus achieve improved demosaicking performance. Experimental results show that the proposed method outperforms several existing or state-of-the-art techniques in terms of both subjective and objective evaluations.     

基于优先级扫描Dyna结构的贝叶斯Q学习方法

贝叶斯Q学习方法使用概率分布来描述Q值的不确定性,并结合Q值分布来选择动作,以达到探索与利用的平衡。然而贝叶斯Q学习存在着收敛速度慢且收敛精度低的问题。针对上述问题,提出一种基于优先级扫描Dyna结构的贝叶斯Q学习方法—Dyna-PS-BayesQL。该方法主要分为2部分：在学习部分,对环境的状态迁移函数及奖赏函数建模,并使用贝叶斯Q学习更新动作值函数的参数;在规划部分,基于建立的模型,使用优先级扫描方法和动态规划方法对动作值函数进行规划更新,以提高对历史经验信息的利用,从而提升方法收敛速度及收敛精度。将Dyna-PS-BayesQL应用于链问题和迷宫导航问题,实验结果表明,该方法能较好地平衡探索与利用,且具有较优的收敛速度及收敛精度。     

Discriminative dictionary learning algorithm based on sample diversity and locality of atoms for face recognition

Dictionary learning is one of the most important algorithms for face recognition. However, many dictionary learning algorithms for face recognition have the problems of small sample and weak discriminability. In this paper, a novel discriminative dictionary learning algorithm based on sample diversity and locality of atoms is proposed to solve the problems. The rational sample diversity is implemented by alternative samples and new error model to alleviate the small sample size problem. Moreover, locality can leads to sparsity and strong discriminability. In this paper, to enhance the dictionary discrimination and to reduce the influence of noise, the graph Laplacian matrix of atoms is used to keep the local information of the data. At the same, the relational theory is presented. A large number of experiments prove that the proposed algorithm can achieve more high performance than some state-of-the-art algorithms.     

基于判别性降维的字典学习在人脸识别中的应用

针对目前大多数人脸识别方法只能单独实施降维或者字典学习而不能完全利用训练样本判别信息的问题,提出了基于判别性降维的字典学习方法,通过联合降维与字典学习使得投影矩阵和字典更好地相互拟合,从而可以获得更高效的人脸分类系统。所提方法的有效性在AR及MPIE两大通用人脸数据库上得到了验证,实验结果表明,相比于几种先进的线性表示方法,所提算法取得了更高的识别率,特别当训练样本数很少的时候,识别效果更佳。     

贝叶斯网络结构学习研究

针对贝叶斯网络结构学习方法难以兼顾高准确率和高效率的问题,提出了一种基于Markov Chain Monte Carlo(MCMC)方法的贝叶斯网络结构学习方法的改进.改进包括:使用依赖关系分析,利用统计学的方法对采样空间进行大幅缩减,能够在精确控制准确度的情况下大幅提高时间效率;结合先验知识,从理论角度将先验知识融入评分中得到完全服从后验分布的结果;搜索最优子结构,对于特定的一些结构搜索最优子结构而不是采用贪心的方法,提高了贝叶斯网络结构学习的准确率.通过理论分析可以证明时间复杂度得到了大幅的降低.并且可以在牺牲可预知的准确率的情况下,将指数时间复杂度降为线性时间.大量的数据实验表明,经改进后的方法在时间和准确性上都具有良好的表现.     

基于优先度排序的3维数据缺失快速插补法

为了快速准确地对3维光学测量的缺失数据进行插值,以利于后期对比研究,提出了基于优先度排序的3维数据缺失快速插补估算方法,并把该算法用于插补模拟实验数据以及20帧振动扬声器测试数据的验证。结果表明,与其它常用缺失数据插补方法相比,该方法运算速度快、插补效果好,有利于处理3维测量结果的多帧和大量的数据。     

基于稀疏贝叶斯学习的MIMO-OFDM电力线通信系统接收机设计

丰富的脉冲噪声干扰对基于MIMO-OFDM技术的电力线通信系统接收机设计带来了巨大挑战。针对这个问题,提出了一种联合估计电力线信道和脉冲噪声的接收机设计方案。该方案主要利用电力信道多径模型参数在频域上的稀疏性和脉冲噪声在时域上的稀疏性特征,将待估计信道模型参数和脉冲噪声联合视作一个稀疏向量,同时利用MIMO系统的空间相关性,构建了一个基于多测量向量的压缩感知模型,并引入多测量向量稀疏贝叶斯学习理论,设计了一种联合估计MIMO信道模型参数和脉冲噪声的方法。仿真结果表明,与传统的MIMO信道估计与脉冲噪声抑制相互分离的接收机方案相比,新方法在估计性能和误比特率性能上有明显提升。     

基于稀疏贝叶斯学习的电力线载波通信接收机设计

针对多径信道和脉冲噪声对电力线载波通信系统性能影响的问题,提出了一种能有效对抗多径信道和脉冲噪声影响的电力线通信系统接收机设计方案.该方案将时域上的电力线信道参数和脉冲噪声联合视作稀疏向量,然后利用稀疏贝叶斯理论联合估计电力线信道和脉冲噪声,从而在接收端得以去除脉冲噪声及补偿信道增益.仿真结果表明,与传统将信道估计与脉冲噪声抑制单独考虑的传统接收机相比,本文提出的接收机方案在误符号率和误比特率等性能指标上有较好的提升.     

The infinite Student's t-mixture for robust modeling

Finite mixture models have been widely used for modeling probability distribution of real data sets due to its benefits from analytical tractability. Among the finite mixtures, the finite Student's t-mixture model (SMM) are tolerant to the untypical data (outliers). However, the SMM could not automatically determine the proper number of components, which is important and may has a significant effect on the learned model. In this paper, we propose an infinite Student's t-mixture model (iSMM) to handle this issue. This model is based on the Dirichlet process mixture which assumes the number of components in a mixture is infinite in advance, and determines the appropriate value of this number according to the observed data. Moreover, we derive an efficient variational Bayesian inference algorithm for the proposed model. Through applications in blind signal detection and image segmentation, it is shown that the iSMM possesses the advantages of both the Student's t-distribution and the Dirichlet process mixture, offering a more powerful and robust performance than competing models.     

Atom specific multiple kernel dictionary based Sparse Representation Classifier for medium scale image classification

Kernel based Sparse Representation Classifier (KSRC) can classify images with acceptable performance. In addition, Multiple Kernel Learning based SRC (MKL-SRC) computes the weighted sum of multiple kernels in order to construct a unified kernel while the weight of each kernel is calculated as a fixed value in the training phase. In this paper, an MKL-SRC with non-fixed kernel weights for dictionary atoms is proposed. Kernel weights are embedded as new variables to the main KSRC goal function and the resulted optimization problem is solved to find the sparse coefficients and kernel weights simultaneously. As a result, an atom specific multiple kernel dictionary is computed in the training phase which is used by SRC to classify test images. Also, it is proved that the resulting optimization problem is convex and is solvable via common algorithms. The experimental results demonstrate the effectiveness of the proposed approach.     

Rakeness with block sparse Bayesian learning for efficient ZigBee‐based EEG telemonitoring

Future healthcare systems are shifted toward long‐term patient monitoring using embedded ultra‐low power devices. In this paper, the strengths of both rakeness‐based compressive sensing (CS) and block sparse Bayesian learning (BSBL) are exploited for efficient electroencephalogram (EEG) transmission/reception over wireless body area networks. A binary sensing matrix based on the rakeness concept is used to find the most energetic signal directions. A balance is achieved between collecting energy and enforcing restricted isometry property to capture the underlying signal structure. Correct presentation of the EEG oscillatory activity, EEG wave shape, and main signal characteristics is provided using the discrete cosine transform based BSBL, which models the intra‐block correlation. The IEEE 802.15.4 wireless communication technology (ZigBee) is employed, since it targets low data rate communications in an energy efficient manner. To alleviate noise and channel multipath effects, a recursive least square based equalizer is used, with an adaptation algorithm that continually updates the filter weights using successive input samples. For the same compression ratio (CR), results indicate that the proposed system permits a higher reconstruction quality compared with the standard CS algorithm. For higher CRs, lower dimensional projections are allowed, meanwhile guaranteeing a correct reconstruction. Thus, low computational high quality data compression/reconstruction are achieved with minimal energy expenditure at the sensors nodes.     

Bayesian network structure learning algorithm based on hybrid binary salp swarm-differential evolution algorithm

Aiming at the disadvantages of Bayesian network structure learned by heuristic algorithms,which were trapping in local minimums and having low search efficiency,a method of learning Bayesian network structure based on hybrid binary slap swarm-differential evolution algorithm was proposed.An adaptive scale factor was used to balance local and global search in the swarm grouping stage.The improved mutation operator and crossover operator were taken into salp search strategy and differential search strategy respectively to renew different subswarms in the update stage.Two-point mutation operator was adopted to improve the swarm’s diversity in the stage of merging of subswarms.The convergence analysis of the proposed algorithm demonstrates that best structure can be found through the iterative search of population.Experimental results show that the convergence accuracy and efficiency of the proposed algorithm are improved compared with other algorithms.     

Hyperparameter Optimization for Machine Learning Models Based on Bayesian Optimization

Hyperparameters are important for machine learning algorithms since they directly control the behaviors of training algorithms and have a significant effect on the performance of machine learning models. Several techniques have been developed and successfully applied for certain application domains. However, this work demands professional knowledge and expert experience. And sometimes it has to resort to the brute-force search. Therefore, if an efficient hyperparameter optimization algorithm can be developed to optimize any given machine learning method, it will greatly improve the efficiency of machine learning. In this paper, we consider building the relationship between the performance of the machine learning models and their hyperparameters by Gaussian processes. In this way, the hyperparameter tuning problem can be abstracted as an optimization problem and Bayesian optimization is used to solve the problem. Bayesian optimization is based on the Bayesian theorem. It sets a prior over the optimization function and gathers the information from the previous sample to update the posterior of the optimization function. A utility function selects the next sample point to maximize the optimization function. Several experiments were conducted on standard test datasets. Experiment results show that the proposed method can find the best hyperparameters for the widely used machine learning models, such as the random forest algorithm and the neural networks, even multi-grained cascade forest under the consideration of time cost.     

Off-grid DOA estimation algorithm based on unitary transform and sparse Bayesian learning

A rapid off-grid DOA estimating method of RV-OGSBL was raised based on unitary transformation,against the problem of traditional sparse Bayesian learning (SBL) algorithm in solving effectiveness of signal’s DOA estimation under condition of lower signal noise ratio (SNR).Actual received signal of uniform linear array was generated through constructing augment matrix as the processing signal used by DOA estimation.Then,estimation model was transformed from complex value to real value by using unitary transformation.In the next step,off-grid model and sparse Bayesian learning algorithm were combined together to process the realization of DOA estimation iteratively.The accuracy of estimation could made relatively high.The simulation result demonstrates that the RV-OGSBL method not only maintains the performance of traditional SBL algorithm,but also reduces the computational complexity significantly.Under the situation of lower signal noise ratio (SNR) and low number of snapshots,the running time of algorithm is reduced about 50%.This shows the RV-OGSBL method is a rapid DOA estimation algorithm.     

A dynamic constraint representation approach based on cross-domain dictionary learning for expression recognition

Facial expression recognition (FER) is an active research area that has attracted much attention from both academics and practitioners of different fields. In this paper, we investigate an interesting and challenging issue in FER, where the training and testing samples are from a cross-domain dictionary. In this context, the data and feature distribution are inconsistent, and thus most of the existing recognition methods may not perform well. Given this, we propose an effective dynamic constraint representation approach based on cross-domain dictionary learning for expression recognition. The proposed approach aims to dynamically represent testing samples from source and target domains, thereby fully considering the feature elasticity in a cross-domain dictionary. We are therefore able to use the proposed approach to predict class information of unlabeled testing samples. Comprehensive experiments carried out using several public datasets confirm that the proposed approach is superior compared to some state-of-the-art methods.     

Optimized sampling distribution based on nonparametric learning for improved compressive sensing performance

In this work, an optimized nonparametric learning approach for obtaining the data-guided sampling distribution is proposed, where a probability density function (pdf) is learned in a nonparametric manner based on past measurements from similar types of signals. This learned sampling distribution is then used to better optimize the sampling process based on the underlying signal characteristics. A realization of this stochastic learning approach for compressive sensing of imaging data is introduced via a stochastic Monte Carlo optimization strategy to learn a nonparametric sampling distribution based on visual saliency. Experiments were performed using different types of signals such as fluorescence microscopy images and laser range measurements. Results show that the proposed optimized sampling method which is based on nonparametric stochastic learning outperforms significantly the previously proposed approach. The proposed method is achieves higher reconstruction signal to noise ratios at the same compression rates across all tested types of signals.