Batch process modelling with mixtures of Gaussian processes

In recent years Gaussian processes have attracted a significant amount of interest with the particular focus being that of process modelling. This has primarily been a consequence of their good predictive performance and inherent analytical properties. Gaussian processes are a member of the family of non-parametric Bayesian regression models and can be derived from the perspective of neural networks. Their behaviour is controlled through the structure of the covariance function. However, when applied to batch processes, whose data exhibits different variance structures throughout the duration of the batch, a single Gaussian process may not be appropriate for the accurate modelling of its behaviour. Furthermore there are issues with respect to the computational costs of Gaussian processes. The implementation of a Gaussian process model requires the repeated computation of a matrix inverse whose order is the cubic of the number of training data points. This renders the algorithm impractical when dealing with large data sets. To address these two issues, a mixture model of Gaussian processes is proposed. The resulting prediction is attained as a weighted sum of the outputs from each Gaussian process component, with the weights determined by a Gaussian kernel gating network. The model is implemented through a Bayesian approach utilising Markov chain Monte Carlo algorithms. The proposed methodology is applied to data from a bench-mark batch simulation polymerization process, methyl methacrylate (MMA), and the results are compared with those from a single Gaussian process to illustrate the advantages of the proposed mixture model approach.     

基于特征选择和数据增强的电池荷电状态预测

现有基于神经网络的电池荷电状态(state of charge, SOC)预测研究大多把重点放在模型结构和相关参数的优化上,却忽略了训练数据的重要作用.针对该问题,文中提出了一种基于特征选择和数据增强的电池SOC预测方法.首先,方法根据原始电池充放电数据进行特征工程,并使用排列重要性(permutation importance, PI)方法选出对模型预测最有帮助的7个特征;其次,通过加入高斯噪声来扩大训练数据样本总量,达到数据增强的目的.实验使用双向长短时记忆网络(bidirectional long short-term memory, Bi-LSTM)作为预测模型,使用Panasonic 18650PF数据集作为训练数据.使用标准Bi-LSTM进行预测时,平均绝对误差(mean absolute error, MAE)和最大误差(max error,MaxE)分别为0.65%和3.92%,而在进行特征选择和数据增强后,模型预测的MAE和MaxE分别为0.47%和2.62%,表明PI特征工程与高斯数据增强方法可以进一步提升电池荷电状态预测模型的精度.     

基于无偏置项LSSVM的稳健在线过程建模方法

针对直接利用最小二乘支持向量机(LSSVM)对动态过程在线建模时预测精度易受过程输出测量值上的粗大误差和噪声影响的问题,在分析样本序列结构特征和噪声作用特征基础上,提出一种基于无偏置项LSSVM的稳健在线过程建模方法。该方法在每一预测周期中根据预测误差与设定阈值之间的关系来识别和恢复异常测量值、识别和修正含噪声测量值,从而降低样本中的噪声,使得出的LSSVM较好地跟踪过程的动态特性。这种在线过程建模方法具有稳健性,能减少输出值上粗大误差和高斯白噪声对LSSVM预测精度的影响,提高预测精度。数字仿真显示该方法的有效性和优越性。     

Rayleigh Mixture Model-Based Hidden Markov Modeling and Estimation of Noise in Noisy Speech Signals

In this paper, we propose a new statistical model for noise periodogram modeling and estimation. The proposed model is a hidden Markov model (HMM) with a Rayleigh mixture model (RMM) in each state. For this new model, we derive an expectation-maximization (EM) training algorithm and a minimum mean-square error (MMSE) noise periodogram estimator. It is shown that when compared to the Gaussian mixture model (GMM)-based HMM, the RMM-based HMM has less computationally complex EM iterations and gives a better fit of the noise periodograms when the mixture models has a low number of components. Furthermore, we propose a specialization of the proposed model, which is shown to provide better MMSE noise periodogram estimates than any other of the tested HMM initializations for cyclo-stationary noise types     

Double linear regression prediction based reversible data hiding in encrypted images

Existing prediction-based works on reversible data hiding in encrypted images usually embed the secret messages by referring to the difference between current pixel and its predicted value. An accurate prediction model may promote an improvement of embedding capacity. Existing schemes, however, may not work well due to involving a bad prediction model so that their embedding capacity cannot be improved further. To address the problem, this paper proposes a new reversible data hiding scheme in encrypted images by designing double linear regression prediction model. Proposed model can significantly improve the prediction accuracy of current pixel based on neighboring pixels, more auxiliary rooms are thus vacated to embed secret data. Furthermore, a prediction error map is constructed to mark the error positions caused by inaccurate prediction, which can be further compressed lossless to lower the capacity of auxiliary data. Reversible recovery for original image can be finally achieved successfully. Experimental results demonstrate that the proposed scheme significantly improves prediction accuracy and data embedding capacity by combining double linear regression prediction model and prediction error map, and then can achieve separable and lossless recovery for the original image. Compared with existing works, the proposed scheme can implement a higher visual quality of decrypted images, while maintaining a larger embedding capacity.

     

基于二次误差的点云配准算法

提出了一种基于二次误差的特征描述子,该特征描述子具有旋转不变性。通过提取点的二次误差和邻域点二次误差得到两种特征描述子。基于高斯混合模型的点云配准算法层出不穷,主要原因是概率模型在噪声和离群值方面具有更好的鲁棒性,然而该类方法对于尺度较大的旋转表现并不好,为此将二次误差特征描述子作为高斯混合模型的局部特征优化了高斯混合模型较大旋转中的配准效果,并提出基于双特征的配准策略优化了单一特征的缺陷。通过实验与鲁棒的ICP(iterative closest point)以及流行的基于特征的配准算法在配准效率和配准精度方面进行对比,效率是鲁棒性ICP的3～4倍。在大尺度的旋转中提出的算法具有良好的鲁棒性并且优于大多数流行的算法。     

Wavelet-based image denoising with the normal inverse Gaussian prior and linear MMSE estimator

A new spatially adaptive wavelet-based method is introduced for reducing noise in images corrupted by additive white Gaussian noise. It is shown that a symmetric normal inverse Gaussian distribution is highly suitable for modelling the wavelet coefficients. In order to estimate the parameters of the distribution, a maximumlikelihood- based technique is proposed, wherein the Gauss?Hermite quadrature approximation is exploited to perform the maximisation in a computationally efficient way. A Bayesian minimum mean-squared error (MMSE) estimator is developed utilising the proposed distribution. The variances corresponding to the noisefree coefficients are obtained from the Bayesian estimates using a local neighbourhood. A modified linear MMSE estimator that incorporates both intra-scale and inter-scale dependencies is proposed. The performance of the proposed method is studied using typical noise-free images corrupted with simulated noise and compared with that of the other state-of-the-art methods. It is shown that the proposed method gives higher values of the peak signal-to-noise ratio compared with most of the other denoising techniques and provides images of good visual quality. Also, the performance of the proposed method is quite close to that of the state-of-the-art Gaussian scale mixture (GSM) method, but with much less complexity.     

基于GMM的间歇过程故障检测

对间歇过程的多操作阶段进行划分时,往往会被离群点和噪声干扰,影响建模的精确性,针对此问题提出一种新的方法:主元分析--多方向高斯混合模型(Principal component analysis-multiple Gaussian mixture model, PCA-MGMM)建模方法.首先用最短长度法对数据进行等长处理,融合不同展开方法相结合的处理方式消除数据预估问题;利用主元分析方法将数据转换到对故障较为敏感的低维子空间中,得到主元的同时消除了离群点和噪声的干扰;通过改进的高斯混合模型(Gaussian mixture model, GMM)算法对各阶段主元进行聚类,减少了运算量的同时自动得到最佳高斯成分和对应的统计分布参数;最后将局部指标融合为全局概率监控指标,实现了连续的在线监控.通过一个实际的半导体制造过程的仿真研究验证了所提方法的有效性.     

一种基于EGMM的高斯过程回归软测量建模

提出了一种基于误差高斯混合模型（EGMM）的高斯过程回归（GPR）软测量方法．首先,选择合适的变量组成误差数据集,利用贝叶斯信息准则优化得到合适的高斯成分的个数;然后用EGMM对误差数据进行拟合计算得到条件误差均值和方差的表达式;最后当新的数据到来时,用建立的GPR模型进行输出预测,并利用EGMM模型得到的条件误差均值对输出进行补偿,从而得到更加精确的建模结果．通过数值仿真及硫回收装置（SRU）的H₂S浓度的软测量,进一步验证所提算法的可行性和有效性．     

Gaussian mixture model learning based image denoising method with adaptive regularization parameters

Gaussian mixture model learning based image denoising as a kind of structured sparse representation method has received much attention in recent years. In this paper, for further enhancing the denoised performance, we attempt to incorporate the gradient fidelity term with the Gaussian mixture model learning based image denoising method to preserve more fine structures of images. Moreover, we construct an adaptive regularization parameter selection scheme by combing the image gradient with the local entropy of the image. Experiment results show that our proposed method performs an improvement both in visual effects and peak signal to noise values.     

Prediction from expert demonstrations for safe tele-surgery

In this paper, we propose a new prediction from expert demonstration （PED） methodology to improve reliability and safety in tele-surgery. Data was collected from expert （clinician） demonstrations for the procedure of trocar insertion. We encoded a set of force, torque and penetration trajectories by using a Gaussian mixture model （GMM）. A generalization of these profiles and associated parameters were retrieved by Gaussian mixture regression （GMR）. We validated the proposed methodology for tele-robotic placement of the trocar in two stages. First, we tested the efficacy of the proposed PED approach for handling transmission error and latency. Our results showed that for the average case （12% packet error and 1070 loss of packet）, a 58.8~0 improvement in performance was obtained in comparison to using an extended Kalman filter. Next, we validated the methodology for surgical assistance on 15 participants. A haptic assistance mode was devised based on the proposed PED model to assist inexperienced operators to perform the procedure. The PED model was tested for instrument deviation, penetration force and penetration depth. Preliminary study results showed that participants with PED assistance performed the task with more consistency and exerted lesser penetration force than subjects without assistance.     

阈值去噪与RBF神经网络在MEMS陀螺仪误差补偿中的应用

针对现有MEMS陀螺仪中随机误差较大,导致器件输出信噪比低进而影响其应用范围的现状,提出一种基于小波阈值去噪与梯度径向基( RBF)神经网络结合的MEMS陀螺漂移非平稳时间序列建模预测方法。首先采用Allan方差法分析了MEMS陀螺仪的主要随机误差,随后利用小波阈值去噪分离出MEMS陀螺误差模型中的白噪声及漂移误差,最后采用RBF神经网络对漂移数据进行建模。通过实验对文中所述的误差补偿方法进行验证,表明了方法的有效性,对于基于MEMS陀螺仪的惯导系统精度提高具有重要意义。     

含局部空间约束的t分布混合模型的点集配准

基于高斯混合模型（Gaussian mixture model,GMM）的点集非刚性配准算法易受重尾点和异常点影响,提出含局部空间约束的t分布混合模型的点集非刚性配准算法. 通过期望最大化（Expectation maximization,EM）框架将高斯混合模型推广为t分布混合模型;把Dirichlet分布作为浮动点的先验权重,并构造含局部空间约束性质的Dirichlet 分布参数. 使用EM算法获得配准参数的闭合解;计算浮动点的自由度,改变其概率密度分布,避免异常点水平估计误差. 实验表明,本文提出的配准算法具有配准误差小、鲁棒性好、抗干扰能力强等优点.     

Optimal instrumental variable estimation and approximate implementations

The accuracy properties of instrumental variables (IV) methods are investigated. Extensions such as prefiltering of data and use of additional instruments are included in the analysis. The parameter estimates are shown to be asymptotically Gaussian distributed. An explicit expression is given for the covariance matrix of their distribution. The covariance matrix is then taken as a (multivariable) measure of accuracy. It is shown how it can be optimized by an appropriate selection of instruments and prefilter. The so obtained optimal instrumental variable estimates cannot be used directly since the true system and the statistical properties of the disturbance must be known in order to compute the optimal instruments and prefilters. A multistep procedure consisting of three or four simple steps is then proposed as a way to overcome this difficulty. This procedure includes modeling of the disturbance as an ARMA process using a statistically efficient method such as a prediction error method. The statistical properties of the estimates obtained with the multistep procedure are also analyzed. Those estimates are shown to be asymptotically Gaussian distributed as well. The covariance matrix of the estimation errors is compared to that corresponding to a prediction error method. For some model structures these two approaches give the same asymptotic accuracy. The conclusion is that the multistep procedure, which is quite easy to implement and also has nice uniqueness properties, can be viewed as an interesting alternative to prediction error methods.     

Bayesian selective combination of multiple neural networks for improving long-range predictions in nonlinear process modelling

A Bayesian selective combination method is proposed for combining multiple neural networks in nonlinear dynamic process modelling. Instead of using fixed combination weights, the probability of a particular network being the true model is used as the combination weight for combining that network. The prior probability is calculated using the sum of squared errors of individual networks on a sliding window covering the most recent sampling times. A nearest neighbour method is used for estimating the network error for a given input data point, which is then used in calculating the combination weights for individual networks. Forward selection and backward elimination are used to select the individual networks to be combined. In forward selection, individual networks are gradually added into the aggregated network until the aggregated network error on the original training and testing data sets cannot be further reduced. In backward elimination, all the individual networks are initially aggregated and some of the individual networks are then gradually eliminated until the aggregated network error on the original training and testing data sets cannot be further reduced. Application results demonstrate that the proposed techniques can significantly improve model generalisation and perform better than aggregating all the individual networks.     

一种BP神经网络机场噪声预测模型

机场噪声预测对机场噪声控制、航班计划制定和机场规划设计具有十分重要的作用。现有的机场噪声预测模型都是以飞机的噪声距离曲线（NPD曲线）为核心,用相应的数学模型将其修正至与具体机场的特定环境条件相关的噪声传播模型,存在预测成本高和误差大的缺点。针对这种情况,提出一种使用BP神经网络利用机场噪声历史监测数据进行NPD曲线修正计算方法,从而建立适用于特定机场环境条件的机场噪声预测模型。实验表明,在特定机场的特定环境条件下,允许误差为0.5 dB时,该模型预测准确率高达91.5%以上,具有预测成本小、准确度高的特点。     

基于隐高斯混合模型的人脑MRI分割方法

针对传统的高斯混合模型的抗噪性能和鲁棒性较差的缺点,提出一种基于隐高斯混合模型的人脑MRI分割方法。传统的高斯混合模型由于忽略了空间信息和未考虑分割结果的分布情况导致模型不完整。针对这些缺点,把分割结果的概率密度函数作为隐含数据引入到高斯混合模型,建立了非线性加权的隐高斯混合模型;同时引入了含空间信息与平滑系数的高斯权重置指数;运用期望最大化算法与牛顿迭代法对类均值,类方差以及平滑系数进行求解,最后根据最大后验概率准则得到人脑MRI的最终分割结果。经实验表明,提出的方法对人脑MRI具有很好的鲁棒性与抗噪性能。     

基于三维空间模型的特高压耐张塔跳线间隙检测方法

提出一种基于三维空间模型的特高压耐张塔跳线间隙检测方法研究,利用三维激光扫描数据建立特高压耐张塔整体空间扫描模型,并描述激光扫描建模的原理。据此确定出了待检测标的物的三维点云数据集合;通过预处理的方式去除点云数据检测系统内部噪声和环境噪声,对点云数据集合进行高斯平滑滤波、坐标转换和点云数据拼接,并输出空间点云数据的文件集合;获取的跳线检测测量结果中还有可能包含粗差、系统误差和偶然性误差,利用检测数据分析前必须消除各种误差因素的影响。实验数据表明提出方法的坐标检测精度值能够控制在+-1,并且总体的检测残差分布也位于合理的区间范围之内,实际检测效果优于传统检测方法。     

一种基于高斯混合模型的轨迹预测算法

在智能交通控制系统、军事数字化战场、辅助驾驶系统中,实时、精确、可靠的移动对象不确定性轨迹预测具有极高的应用价值.智能轨迹预测不仅可以提供精准的基于位置的服务,而且可以提前监测和预判交通状况,进而推荐最佳路线,已经成为移动对象数据库研究的热点,亟需设计准确而高效的位置预测方法.针对现有方法的不足,提出了基于高斯混合模型的轨迹预测方法GMTP,主要步骤包括:(1) 针对复杂运动模式利用高斯混合模型建模;(2) 利用高斯混合模型计算不同运动模式的概率分布,进而将轨迹数据划分为不同分量;(3) 利用高斯过程回归预测移动对象最可能的运动轨迹.GMTP是高斯非线性概率统计模型,其优势在于:计算结果不仅是位置预测值,更是关于移动对象未来所有可能运动轨迹的概率分布,可以利用概率统计分布特性获得某种运动模式(如匀加速运动)下的位置预测.大量真实轨迹数据集上的实验结果表明:与相同参数设置下的高斯回归预测和卡尔曼滤波预测法相比,GMTP的预测准确性平均提高了22.2%和23.8%,预测时间平均缩减了92.7%和95.9%.     

结合聚类参数的圆投影模板匹配改进算法

针对圆投影模板匹配方法特征提取过程中损失大量图像信息的缺点,提出了结合聚类模型参数的线性光照鲁棒圆投影模板匹配方法。所提方法采用线性对比度拉伸来消除光照影响,并将模板图像各圆环内像素点的高斯混合模型聚类参数作为模板特征。匹配时通过一次迭代计算即可得到匹配误差,且该匹配过程可通过查找表来提高匹配速度。在目标搜索时使用了降采样搜索方法,并将降采样搜索匹配后各位置的误差均值作为自适应阈值,对匹配误差小于该阈值的降采样点邻域进行逐点匹配,匹配误差最小的位置作为最终匹配结果。试验及分析说明所提方法的定位误差及可靠度与基于归一化相关及均值的圆投影匹配算法相比有较大提高。