二值probit回归模型的坍缩变分贝叶斯推断算法

给出了二值probit回归模型的坍缩变分贝叶斯推断算法.此算法比变分贝叶斯推断算法能更逼近对数边缘似然,得到更精确的模型参数后验期望值.如果两个算法得到的分类错误一致,则该算法的迭代次数较变分法明显减少.仿真实验结果验证了所提出算法的有效性.     

基于局部变分贝叶斯推断的分布式交互式多模型估计

针对目前部分多模型算法预先设定运动模型转移概率矩阵对状态估计精度的不利影响,本文提出了一种基于局部变分贝叶斯推断的分布式交互式多模型估计算法.不同于传统交互式多模型估计中运动模型转移概率矩阵为先验已知的假设条件,在分布融合估计框架下,首先基于最小化Kullback-Leibler散度准则的递归优化策略实现对运动模型转移概率矩阵的预测与更新;在此基础上,结合变分贝叶斯推断实现对当前时刻目标状态与模型概率的联合估计;最后依据协方差交叉融合策略完成对局部状态估计融合.仿真结果表明:新算法通过对运动模型转移概率矩阵以及模型概率自适应在线估计,有效提升了机动目标的状态估计精度.     

扩散式变阶数最大相关熵准则算法

固定阶数的分布式自适应滤波算法只有在待估计向量的阶数已知且恒定的情况下才能达到相应的估计精度,在阶数未知或时变的情况下算法的收敛性能会受到影响,变阶数的分布式自适应滤波算法是解决上述问题的有效途径。但是目前大多数分布式变阶数自适应滤波算法以最小均方误差(Mean square Error, MSE)准则作为滤波器阶数的代价函数,在脉冲噪声环境下算法的收敛过程会受到较大影响。最大相关熵准则具有对脉冲噪声的强鲁棒性,且计算复杂度低。为提高分布式变阶数自适应滤波算法在脉冲噪声环境下的估计精度,利用最大相关熵准则作为滤波器阶数迭代的代价函数,并将得到的结果代入固定阶数的扩散式最大相关熵准则算法,提出了一种扩散式变阶数最大相关熵准则(Diffusion Variable Tap-length Maximum Correntropy Criterion, DVTMCC)算法。通过与邻域的节点进行通信,所提算法以扩散的方式实现了整个网络的信息融合,具有估计精度高、计算量小等优点。仿真实验对比了在脉冲噪声下DVTMCC算法和其他分布式变阶数自适应滤波算法、固定阶数的扩散式最大相关熵准则算法的收敛性能。...     

基于变分贝叶斯推断的字典学习算法

传统的字典学习算法在对训练图像进行学习时收敛速率慢,当图像受到噪声干扰时学习效果变差.对此,提出一种基于变分推断的字典学习算法.首先设定模型中各参数的共轭稀疏先验分布;然后基于贝叶斯网络求出所有参数的联合概率密度函数;最后利用变分贝叶斯推断原理计算出各参数的最优边缘分布,训练出自适应学习字典.利用该字典进行图像去噪实验以及压缩感知重构实验,仿真结果表明,所提出的算法可显著提高字典学习效率,对测试图像的去噪效果和重构精度有很大改善.     

WSN中一种基于连通支配集的分簇一致性算法

分布式一致性算法可用于解决分布式协作参数估计等许多问题,但在无线传感器网络的应用中还要满足低能耗、高可靠性、实时性的要求.为加快一致性算法的收敛速率,以降低通信能量开销和满足实时性的要求,提出了一类基于连通支配集(CDS)的分簇一致性算法(CBDC),其包括基于CDS的分簇算法和簇上一致性算法两个基本构件.提出了一种基于邻居连通度的连通支配集构造算法(NCCDS)及基于NCCDS的分簇方法.对基于CDS的CBDC算法进行了仿真,结果表明,相对其他经典CDS构造算法,基于NCCDS的CBDC算法对收敛速率的改善更好.     

分簇式无线传感器网络节点故障诊断算法研究

无线传感器网络(WSNs)分布式节点故障诊断算法是一种可用于WSNs节点的故障诊断算法,通过整个网络内邻居节点之间的数据融合诊断出故障节点.但分布式算法的计算量十分巨大,浪费了大量的节点能源,而且分布式算法中使用自定义的全局阈值会降低诊断精度,分簇式的节点故障诊断算法应用LEACH-DFD算法,通过簇头节点完成故障检测...     

分布式采样理论综述

采样是一类基本的计算问题.从一个解空间中依特定概率分布进行随机采样,这一问题在近似计数、概率推断、统计学习等方面都有着诸多重要的应用.在大数据时代,采样问题的分布式算法与分布式计算复杂性受到越来越多的关注.近年来,有一系列的工作对分布式采样理论展开系统性的研究.综述了分布式采样的重要结论,主要包括有严格理论保障的分布式采样算法、采样问题在分布式模型上的计算复杂性以及采样与推断等问题在分布式计算模型中的相互联系.     

基于Unscented粒子滤波的无序观测下水下无线传感器网络目标被动跟踪

为提高水下无线传感器网络(UWSN)中的目标被动跟踪性能,提出了一种新的无序观测量(OOSM)处理算法.利用节点动态分簇建立分布式跟踪结构,簇头节点收集子节点的观测量形成本地估计.基于这种分布式结构,利用Unscented粒子滤波(UPF)结合新观测量,产生粒子滤波的建议密度分布,处理OOSM问题.详细推导了基于UPF的OOSM处理算法(OOSM-UPF)的具体实现步骤.利用转弯率建立机动目标跟踪模型,构建虚拟三维WSN仿真环境,比较了几种OOSM算法的性能.仿真结果表明,与其它算法相比,分布式OOSM-UPF算法的跟踪性能有了明显的提高.     

无线传感器网络中基于压缩感知的分簇数据收集算法

针对无线传感器网络(WSNs)能量有限、通信链路不可靠的特点,提出一种基于稀疏分块对角矩阵进行压缩感知的分簇(SBDMC)数据收集算法.该算法以稀疏分块对角矩阵作为观测矩阵以减少参与收集节点数目;采用分布式分簇路由实现数据的分布式收集;通过分析能耗模型得到最优簇头数目以减少网络能耗.在此基础上,给出一种有效的分簇路由数据收集算法.仿真分析表明:提出的算法较之已有算法可以减少通信能耗、延长网络寿命,同时均衡能耗负载.     

折棍变分贝叶斯图像分割算法

为提高图像分割的抗噪鲁棒性并解决分割数目的自适应确定问题,通过在聚类标签先验概率的折棍构造过程中建立Markov随机场,将空间相关性约束引入Dirichlet过程混合模型的概率建模,使聚类的空间平滑性得以增强,并采用变分推断方法获得聚类标签的收敛解析解,提出一种基于折棍变分贝叶斯推断的图像分割算法,实现了对像素聚类标签和分割数目的同步自适应学习,避免了传统方法中因引入空间相关性约束而出现的计算复杂问题.基于Berkeley BSD500图像测试数据集的数值实验结果表明,该算法具有比现有的混合模型聚类图像分割算法更高的PRI值,且在低于0.1的噪声方差条件下表现出了更优的抗噪鲁棒性.     

Variational Inference Based Kernel Dynamic Bayesian Networks for Construction of Prediction Intervals for Industrial Time Series With Incomplete Input

Prediction intervals (PIs) for industrial time series can provide useful guidance for workers. Given that the failure of industrial sensors may cause the missing point in inputs, the existing kernel dynamic Bayesian networks (KDBN), serving as an effective method for PIs construction, suffer from high computational load using the stochastic algorithm for inference. This study proposes a variational inference method for the KDBN for the purpose of fast inference, which avoids the time-consuming stochastic sampling. The proposed algorithm contains two stages. The first stage involves the inference of the missing inputs by using a local linearization based variational inference, and based on the computed posterior distributions over the missing inputs the second stage sees a Gaussian approximation for probability over the nodes in future time slices. To verify the effectiveness of the proposed method, a synthetic dataset and a practical dataset of generation flow of blast furnace gas (BFG) are employed with different ratios of missing inputs. The experimental results indicate that the proposed method can provide reliable PIs for the generation flow of BFG and it exhibits shorter computing time than the stochastic based one.      

Tuning the Learning Rate for Stochastic Variational Inference

Stochastic variational inference (SVI) can learn topic models with very big corpora. It optimizes the variational objective by using the stochastic natural gradient algorithm with a decreasing learning rate. This rate is crucial for SVI; however, it is often tuned by hand in real applications. To address this, we develop a novel algorithm, which tunes the learning rate of each iteration adaptively. The proposed algorithm uses the Kullback–Leibler (KL) divergence to measure the similarity between the variational distribution with noisy update and that with batch update, and then optimizes the learning rates by minimizing the KL divergence. We apply our algorithm to two representative topic models: latent Dirichlet allocation and hierarchical Dirichlet process. Experimental results indicate that our algorithm performs better and converges faster than commonly used learning rates.     

基于WB-MMSB模型的微博网络社区发现

提出了一个用于微博网络社区发现的模型WB-MMSB,该模型考虑了微博网络中节点存在的单向关系,节点的社区隶属度从链入主题隶属度和链出主题隶属度两个方面表示。用指数族分布和平均场变分推理方法推导了模型中各变量的表示,并用SVI算法计算模型涉及的参数。实验在新浪微博数据集上进行,采用归一化互信息和困惑度进行评估,结果表明,WB-MMSB模型的社区发现能力优于aMMSB模型,并且其收敛速度快于aMMSB模型。     

利用标注者相关性的深度生成式众包学习

传统监督学习需要训练样本的真实标记信息,而在很多情况下,真实标记并不容易收集.与之对比,众包学习从多个可能犯错的非专家收集标注,通过某种融合方式估计样本的真实标记.注意到现有深度众包学习工作对标注者相关性建模不足,而非深度众包学习方面的工作表明,标注者相关性建模利用有助于改善学习效果.提出一种深度生成式众包学习方法,以...     

Spatially Varying Mixtures Incorporating Line Processes for Image Segmentation

Spatially varying mixture models are characterized by the dependence of their mixing proportions on location (contextual mixing proportions) and they have been widely used in image segmentation. In this work, Gauss-Markov random field (MRF) priors are employed along with spatially varying mixture models to ensure the preservation of region boundaries in image segmentation. To preserve region boundaries, two distinct models for a line process involved in the MRF prior are proposed. The first model considers edge preservation by imposing a Bernoulli prior on the normally distributed local differences of the contextual mixing proportions. It is a discrete line process model whose parameters are computed by variational inference. The second model imposes Gamma prior on the Student’s-t distributed local differences of the contextual mixing proportions. It is a continuous line process whose parameters are also automatically estimated by the Expectation-Maximization (EM) algorithm. The proposed models are numerically evaluated and two important issues in image segmentation by mixture models are also investigated and discussed: the constraints to be imposed on the contextual mixing proportions to be probability vectors and the MRF optimization strategy in the frameworks of the standard and variational EM algorithm.     

动态贝叶斯网络一种自适应的局部抽样粒子滤波算法*

针对传统自适应粒子滤波（APF）对于动态贝叶斯网络推理中高维的问题,提出动态贝叶斯网络一种自适应的局部抽样粒子滤波算法(LSAPF)。LSAPF算法将BK算法分团的思想引入到粒子抽样中,利用策略相关性和局部模型的弱交互性为指导对动态贝叶斯网络进行分割,以降低抽样规模和抽样的状态空间;进而对局部模型用自适应粒子滤波算法进行近似推理,并以粒子的因式积形式近似系统的状态信度。实验结果表明,该算法能很好地兼顾推理精度和推理时间,其性能优于普通PF算法;与APF算法相比,在不增加推理误差的情况下推理时间也有较大的提高。     

Distributed EM Algorithm for Gaussian Mixtures in Sensor Networks

This paper presents a distributed expectation–maximization (EM) algorithm over sensor networks. In the E-step of this algorithm, each sensor node independently calculates local sufficient statistics by using local observations. A consensus filter is used to diffuse local sufficient statistics to neighbors and estimate global sufficient statistics in each node. By using this consensus filter, each node can gradually diffuse its local information over the entire network and asymptotically the estimate of global sufficient statistics is obtained. In the M-step of this algorithm, each sensor node uses the estimated global sufficient statistics to update model parameters of the Gaussian mixtures, which can maximize the log-likelihood in the same way as in the standard EM algorithm. Because the consensus filter only requires that each node communicate with its neighbors, the distributed EM algorithm is scalable and robust. It is also shown that the distributed EM algorithm is a stochastic approximation to the standard EM algorithm. Thus, it converges to a local maximum of the log-likelihood. Several simulations of sensor networks are given to verify the proposed algorithm.      

Variational learning for switching state-space models

We introduce a new statistical model for time series that iteratively segments data into regimes with approximately linear dynamics and learnsthe parameters of each of these linear regimes. This model combines and generalizes two of the most widely used stochastic time-series models -- hidden Markov models and linear dynamical systems -- and is closely related to models that are widely used in the control and econometrics literatures. It can also be derived by extending the mixture of experts neural network (Jacobs, Jordan, Nowlan, & Hinton, 1991) to its fully dynamical version, in which both expert and gating networks are recurrent. Inferring the posterior probabilities of the hidden states of this model is computationally intractable, and therefore the exact expectation maximization (EM) algorithm cannot be applied. However, we present a variational approximation that maximizes a lower bound on the log-likelihood and makes use of both the forward and backward recursions for hidden Markov models and the Kalman filter recursions for linear dynamical systems. We tested the algorithm on artificial data sets and a natural data set of respiration force from a patient with sleep apnea. The results suggest that variational approximations are a viable method for inference and learning in switching state-space models.     

基于结构分析的局部Gibbs抽样自动推理算法

提出一种基于结构分析的局部Gibbs抽样的贝叶斯网络推理算法(S-LGSI).S-LGSI算法基于联合树算法的概率图模型分析思想,对贝叶斯网络进行精确分解,然后根据查询结点和证据结点生成具有强相关性的局部网络模型,进而对局部网络模型进行Gibbs抽样推理.与当前基于抽样的其它近似推理算法相比,该算法降低推理的计算维数.同时,由于局部抽样模型包含了与查询结点相关的重要信息,因此该算法保证局部抽样推理的精度.算法分析和在Alarm网的实验结果表明,S-LGSI算法较显著降低时间复杂度,同时也提高推理精度.S-LGSI算法应用于上海证券交易所股票网络的推理结果与实际情况基本一致,表现出较强的实用性.     

面向大规模数据主题建模的方差减小的随机变分推理算法

随机变分推理（SVI）已被成功应用于在包括主题模型在内的众多类型的模型。虽然它将推理问题映射到涉及随机梯度的优化问题,使其扩展到处理大规模数据集,但是SVI算法中随机梯度固有的噪声使其产生较大的方差,阻碍了快速收敛。为此,对SVI作出改进,提出一种方差减小的SVI （VR-SVI）算法。首先,采取滑动窗口的方法重新计算随机梯度中的噪声项,构建新的随机梯度,减少了噪声对随机梯度的影响;然后,对提出的算法可在SVI基础上使得随机梯度的方差减小进行证明;最后,讨论窗口大小对算法的影响,并分析算法的收敛性。实验结果表明,VR-SVI算法既减小了随机梯度的方差,又节省了计算时间,可达到快速收敛的效果。