Information geometry of mean-field approximation

I present a general theory of mean-field approximation based on information geometry and applicable not only to Boltzmann machines but also to wider classes of statistical models. Using perturbation expansion of the Kullback divergence (or Plefka expansion in statistical physics), a formulation of mean-field approximation of general orders is derived. It includes in a natural way the "naive" mean-field approximation and is consistent with the Thouless-Anderson-Palmer (TAP) approach and the linear response theorem in statistical physics.     

递推信度传播算法——-按良序的信度传播

针对循环信度传播算法在多环的贝叶斯网中迭代次数较多且不一定收敛的问题, 提出了递推信度传播算法. 它与循环信度传播及其推广算法的区别就在于按某一特定顺序(良序)进行信度传播. 该算法经过一轮信度传播便达到不动点, 显著降低了计算量. 按这种顺序传播信度等价于去掉网络中某些边而解除了网络中的环, 从而使信度不再出现环流. 此算法得到的不动点与循环信度传播算法在收敛时得到的不动点是一致的, 也就是网络的Bethe自由能的最小值点. 最后, 实验验证本文所提的算法在实际应用中能有效地降低推理的复杂度.     

两两关系马尔科夫网的自适应组稀疏化学习

稀疏化学习能显著降低无向图模型的参数学习与结构学习的复杂性, 有效地处理无向图模型的学习问题. 两两关系马尔科夫网在多值变量情况下, 每条边具有多个参数, 本文对此给出边参数向量的组稀疏化学习, 提出自适应组稀疏化, 根据参数向量的模大小自适应调整惩罚程度. 本文不仅对比了不同边势情况下的稀疏化学习性能, 为了加速模型在复杂网络中的训练过程, 还对目标函数进行伪似然近似、平均场自由能近似和Bethe自由能近似. 本文还给出自适应组稀疏化目标函数分别使用谱投影梯度算法和投 影拟牛顿算法时的最优解, 并对比了两种优化算法进行稀疏化学习的性能. 实验表明自适 应组稀疏化具有良好的性能.     

Decoding Algorithm of Low-density Parity-check Codes based on Bowman-Levin Approximation

Belief propagation (BP) and the concave-convex procedure (CCCP) are algorithms that use the Bethe free energy as a cost function and are used to solve information processing tasks. We have developed a new algorithm that also uses the Bethe free energy but changes the roles of the master and slave variables. This is called the Bowman-Levin (BL) approximation in the domain of statistical physics. When we applied the BL approximation to decode the regular low-density parity-check (LDPC) codes over an additive white Gaussian noise (AWGN) channel, its average performance was roughly similar to that of either BP or CCCP, but slightly outperforms them if the vast calculation cost is not prohibitive. This implies that our algorithm based on the BL approximation can be successfully applied to other problems to which BP or CCCP have already been applied. We also found that the decoding dynamics of the BL algorithm particularly depend on the number of inner loops. These differences from BP may be important in understanding the complicated landscape of the Bethe free energy.     

A tighter bound for graphical models

We present a method to bound the partition function of a Boltzmann machine neural network with any odd-order polynomial. This is a direct extension of the mean-field bound, which is first order. We show that the third-order bound is strictly better than mean field. Additionally, we derive a third-order bound for the likelihood of sigmoid belief networks. Numerical experiments indicate that an error reduction of a factor of two is easily reached in the region where expansion-based approximations are useful.     

Calculation of the T–P phase diagram for oxygen using the mean field theory

T–P phase diagram of oxygen with the liquid and solid phases (α, ά, ᾱ, β, γ and ϵ) is calculated using the mean field theory. By expanding the free energy in terms of the order parameter, the phase line equations are derived and they are fitted to the experimental T–P data from the literature.Our calculated T–P phase diagram predicts adequately the observed behavior of the transitions between the liquid and solid phases in oxygen (O₂). Some thermodynamic quantities are predicted from the free energy expansion for the liquid–γ and ά–ɛ transitions in O₂.     

Thermodynamics and its prediction and CALPHAD modeling: Review,state of the art,and perspectives

Thermodynamics is a science concerning the state of a system, whether it is stable, metastable, or unstable, when interacting with its surroundings. The combined law of thermodynamics derived by Gibbs about 150 years ago laid the foundation of thermodynamics. In Gibbs combined law, the entropy production due to internal processes was not included, and the 2^nd law was thus practically removed from the Gibbs combined law, so it is only applicable to systems under equilibrium, thus commonly termed as equilibrium or Gibbs thermodynamics. Gibbs further derived the classical statistical thermodynamics in terms of the probability of configurations in a system in the later 1800's and early 1900's. With the quantum mechanics (QM) developed in 1920's, the QM-based statistical thermodynamics was established and connected to classical statistical thermodynamics at the classical limit as shown by Landau in the 1940's. In 1960's the development of density functional theory (DFT) by Kohn and co-workers enabled the QM prediction of properties of the ground state of a system. On the other hand, the entropy production due to internal processes in non-equilibrium systems was studied separately by Onsager in 1930's and Prigogine and co-workers in the 1950's. In 1960's to 1970's the digitization of thermodynamics was developed by Kaufman in the framework of the CALculation of PHAse Diagrams (CALPHAD) modeling of individual phases with internal degrees of freedom. CALPHAD modeling of thermodynamics and atomic transport properties has enabled computational design of complex materials in the last 50 years. Our recently termed zentropy theory integrates DFT and statistical mechanics through the replacement of the internal energy of each individual configuration by its DFT-predicted free energy. The zentropy theory is capable of accurately predicting the free energy of individual phases, transition temperatures and properties of magnetic and ferroelectric materials with free energies of individual configurations solely from DFT-based calculations and without fitting parameters, and is being tested for other phenomena including superconductivity, quantum criticality, and black holes. Those predictions include the singularity at critical points with divergence of physical properties, negative thermal expansion, and the strongly correlated physics. Those individual configurations may thus be considered as the genomic building blocks of individual phases in the spirit of the materials genome®. This has the potential to shift the paradigm of CALPHAD modeling from being heavily dependent on experimental inputs to becoming fully predictive with inputs solely from DFT-based calculations and machine learning models built on those calculations and existing experimental data through newly developed and future open-source tools. Furthermore, through the combined law of thermodynamics including the internal entropy production, it is shown that the kinetic coefficient matrix of independent internal processes is diagonal with respect to the conjugate potentials in the combined law, and the cross phenomena that the phenomenological Onsager flux and reciprocal relationships are due to the dependence of the conjugate potential of a molar quantity on nonconjugate molar quantities and other potentials, which can be predicted by the zentropy theory and CALPHAD modeling.     

An efficient energy-conserving numerical model for the electron energy distribution function in the presence of electron-electron collisions

An efficient algorithm to calculate the contribution of electron-electron collisions in the Boltzmann equation for free electrons, in the two-term approximation is presented. The electron-electron collision term must be energy-conserving, while, due to non-linearity, commonly used algorithms do not satisfy this requirement. The efficiency of the algorithm make feasible the use of a non-linear iterative solver to conserve electron energy in electron-electron collisions.The performance of the proposed algorithm has been compared with standard numerical schemes obtaining: 1) considerable gain in computational time; 2) the conservation of the total electron energy density in e-e collisions under the required tolerance.     

On the uniqueness of loopy belief propagation fixed points

We derive sufficient conditions for the uniqueness of loopy belief propagation fixed points. These conditions depend on both the structure of the graph and the strength of the potentials and naturally extend those for convexity of the Bethe free energy. We compare them with (a strengthened version of) conditions derived elsewhere for pairwise potentials. We discuss possible implications for convergent algorithms, as well as for other approximate free energies.     

Pressure condition for lattice Boltzmann methods on domains with curved boundaries

We propose a lattice Boltzmann algorithm for an average pressure boundary condition at outlets in pipe flow systems. The advantage of this boundary condition is that only the average pressure is used to recover the non-trivial flow fields. The asymptotic analysis shows that this algorithm works for general curved boundaries and renders a second order accurate velocity and a first order accurate pressure approximation of the incompressible Navier–Stokes solution. Here, we verify the accuracy by numerical simulations of a Poiseuille flow and a less symmetric flow with non-trivial pressure field in channels inclined with arbitrary angle, and flows in a pipe with three outlets.     

2,6-二硫代黄嘌呤互变异构体的密度泛函理论研究

在密度泛函B3LYP/6-311G~(**)水平下,对14种气相和水相中可能存在的2,6-二硫代黄嘌呤异构体进行了几何构型全自由度优化,并计算出它们的总能量、焓、熵、吉布斯自由能。Onsager反应场溶剂模型用于水相的计算,计算结果表明,2,6-二硫代黄嘌呤在气相和水相中主要以双硫酮形式存在。在气相和水相中,硫酮-N(7)(H)均比硫酮-N(9)(H)稳定,计算结果同已有实验结果一致。2,6-二硫代黄嘌呤互变异构的熵效应小,对互变异构平衡没有显著影响,而焓变对互变异构产生了主要影响。较详细地讨论了水溶剂化作用对异构体的能量、几何结构、电荷分布和偶极矩的影响。溶剂化自由能与异构体的偶极矩存在相关性。并且以在水相中最稳定的两种异构体为对象,采用超分子模型探讨了它们与水分子之间氢键作用的情况。     

Quartic parameters for acoustic applications of lattice Boltzmann scheme

Using the Taylor expansion method, we show that it is possible to improve the lattice Boltzmann method for acoustic applications. We derive a formal expansion of the eigenvalues of the discrete approximation and fit the parameters of the scheme to enforce fourth order accuracy. The corresponding discrete equations are solved with the help of symbolic manipulation. The solutions are obtained in the case of D3Q27 lattice Boltzmann scheme. Various numerical tests support the coherence of this approach.     

Free energies of ferroelectric crystals from a microscopic approach

The free energy of barium titanate is computed around the Curie temperature as a function of polarization from the first-principles derived Effective Hamiltonian of Zhong, Vanderbilt and Rabe [Phys. Rev. Lett. 73 (1994) 1861], through Molecular Dynamics simulations coupled to the method of the Thermodynamic Integration. The algorithms used to fix the temperature (Nosé-Hoover) and/or the pressure/stress (Parrinello-Rahman), combined with fixed-polarization molecular dynamics, allow to compute a Helmholtz free energy (fixed volume/strain) or a Gibbs free energy (fixed pressure/stress). The main feature of this approach is to calculate the gradient of the free energy in the 3-D space (Px, Py, Pz) from the thermal averages of the forces acting on the local modes, that are obtained by Molecular Dynamics under the constraint of fixed . This work extends the method presented in [Phys. Rev. B 79 (2009) 064101] to the calculation of the Gibbs free energy and presents new features about the computation of the free energy of ferroelectric crystals from a microscopic approach. A careful analysis of the states of constrained polarization is performed at T=280 K (≈15-17 K below Tc) especially at low order parameter. These states are found reasonably homogeneous for small supercell size (L=12 and L=16), until inhomogeneous states are observed at low order parameter for large supercells (L=20). The effect of this evolution towards multidomain configurations on the mean force and free energy curves is shown. However, for reasonable supercell sizes (L=12), the free energy curves obtained are in very good agreement with phenomenological Landau potentials of the literature and the states of constrained polarization are homogeneous. Moreover, the free energy obtained is quite insensitive to the supercell size from L=12 to L=16 at T=280 K, suggesting that interfacial contributions, if any, are negligible at these sizes around Tc. The method allows a numerical estimation of the free energy barrier separating the paraelectric from the ferroelectric phase at Tc (ΔG≈0.012-0.015 meV/5-atom cell). However, our tests evidence phase separation at low temperature and low order parameter, in agreement with the results of Tröster et al. [Phys. Rev. B 72 (2005) 094103]. Finally, the natural decomposition of the forces into onsite, short-range, dipole-dipole and elastic-local mode interaction allows to make the same decomposition of the free energy. Some parts of this decomposition can be directly calculated from the coefficients of the Effective Hamiltonian.     

Theoretical study of vibrational contribution on cluster formation in a binary alloy system

Lattice vibrational effects in the process of precipitation in a binary alloy system has been studied theoretically. We derived a cancellation condition of vibrational free energy change due to cluster formation within the first order approximation, in terms of the nearest bond pair interaction. Vibrational contribution to the free energy is calculated for a specific example of cluster formation from isolated atoms, and it is almost perfectly vanished when the unlike atom potential is assumed to be the geometric mean of their constituent potentials.     

The stability of transition phases in Fe-Mn-Si based alloys

Various transition phases, featured by nH(n>2), are possible in the Fe-Mn-Si based alloys. Using the Chou-Model, their Gibbs free energies are calculated and their stabilities are ranked. The thermodynamical parameters of those structures were deduced from those of fcc and hcp structures which have been established based on the stacking fault energy (SFE) and the SF density. The results indicate that at the temperatures below thermal equilibrium (To) the stability of transition phases is in an order of 2H(hcp)>9H(9R)>4H>6H>fcc and in the reverse order when the temperature is above To, consistent with the experimental observations. In the case that an external stress field exists, the strain energy plays an important role on the phase stabilities and, therefore, their order would be modified.     

Decoupled Energy Stable Schemes for a Phase Field Model of Three-Phase Incompressible Viscous Fluid Flow

We develop a numerical approximation for a hydrodynamic phase field model of three immiscible, incompressible viscous fluid phases. The model is derived from a generalized Onsager principle following an energetic variational formulation and is consisted of the momentum transport equation and coupled phase transport equations. It conserves the volume of each phase and warrants the total energy dissipation in time. Its numerical approximation is given by a set of easy-to-implement, semi-discrete, linear, decoupled elliptic equations at each time step, which can be solved efficiently using fast solvers. We prove that the scheme is energy stable. Mesh refinement tests and three numerical examples of three-phase viscous fluid flows in 3D are presented to benchmark the effectiveness of the model as well as the efficiency of the numerical scheme.     

Annealing by two sets of interactive dynamics

This work derives the mean field approximation to the mean configuration of a stochastic Hopfield neural network under the Boltzmann assumption. The new approximation is realized by two sets of interactive mean field equations, respectively estimating mean activations subject to mean correlations and mean correlations subject to mean activations. The two sets of interactive dynamics are derived based on two dual mathematical frameworks. Each aims to optimize the objective quantified by a combiation of the Kullback-Leibler (KL) divergence and the correlation strength between any two distinct fluctuated variables subject to fixed mean correlations or activations. The new method is applied to the graph bisection problem. By numerical simulations, we show that the new method effectively improves in both performance and relaxation efficiency against the naive mean field equation     

基于多隐层Gibbs采样的深度信念网络训练方法

深度信念网络（Deep belief network，DBN）作为一类非常重要的概率生成模型，在多个领域都有着广泛的用途.现有深度信念网的训练分为两个阶段，首先是对受限玻尔兹曼机（Restricted Boltzmann machine，RBM）层自底向上逐层进行的贪婪预训练，使得每层的重构误差最小，这个阶段是无监督的；随后再对整体的权值使用有监督的反向传播方法进行精调.本文提出了一种新的DBN训练方法，通过多隐层的Gibbs采样，将局部RBM层组合，并在原有的逐层预训练和整体精调之间进行额外的预训练，有效地提高了DBN的精度.本文同时比较了多种隐层的组合方式，在MNIST和ShapeSet以及Cifar10数据集上的实验表明，使用两两嵌套组合方式比传统的方法错误率更低.新的训练方法可以在更少的神经元上获得比以往的训练方法更好的准确度，有着更高的算法效率.