Application of maximum entropy principle for reliability-based design optimization

The maximum entropy principle (MEP) is used to generate a natural probability distribution among the many possible that have the same moment conditions. The MEP can accommodate higher order moment information and therefore facilitate a higher quality PDF model. The performance of the MEP for PDF estimation is studied by using more than four moments. For the case with four moments, the results are compared with those by the Pearson system. It is observed that as accommodating higher order moment, the estimated PDF converges to the original one. A sensitivity analysis formulation of the failure probability based on the MEP is derived for reliability-based design optimization (RBDO) and the accuracy is compared with that by finite difference method (FDM). Two RBDO examples including a realistic three-dimensional wing design are solved by using the derived sensitivity formula and the MEP-based moment method. The results are compared with other methods such as TR-SQP, FAMM + Pearson system, FFMM + Pearson system in terms of accuracy and efficiency. It is also shown that an improvement in the accuracy by including more moment terms can increase numerical efficiency of optimization for the three-dimensional wing design. The moment method equipped with the MEP is found flexible and well adoptable for reliability analysis and design.     

最大熵模型在邮件分类中的应用

邮件分类是指在给定的分类体系下,根据邮件的内容和属性,确定其类别标签的过程。将最大熵模型应用于邮件分类中,给出了邮件的预处理过程,介绍了邮件信头特征,分析比较了特征数量和迭代次数、邮件特征字段对分类结果的影响,以及对层次分类和平面分类的效果进行了比较。实验表明,特征数量和迭代次数分别取2 000和250时为宜;充分利用邮件各字段信息,取得的总体分类效果最好,但对合法邮件,利用邮件头及邮件标题却取得了最好结果,并在层次分类中验证了这点,层次分类效果要优于平面分类。最后进行了总结和展望。     

New measures of weighted fuzzy entropy and their applications for the study of maximum weighted fuzzy entropy principle

Keeping in view the non-probabilistic nature of experiments, two new measures of weighted fuzzy entropy have been introduced and to check their authenticity, the essential properties of these measures have been studied. Under the fact that measures of entropy can be used for the study of optimization principles when certain partial information is available, we have applied the existing as well as the newly introduced weighted measures of fuzzy entropy to study the maximum entropy principle.     

基于最大信息熵模型的异常流量分类方法*

最大信息熵原理已被成功地应用于各种自然语言处理领域,如机器翻译、语音识别和文本自动分类等,提出了将其应用于互联网异常流量的分类。由于最大信息熵模型利用二值特征函数来表达和处理符号特征,而KDD99数据集中存在多种连续型特征,因此采用基于信息熵的离散化方法对数据集进行预处理,并利用CFS算法选择合适的特征子集,形成训练数据集合。最后利用BLVM算法进行参数估计,得到满足最大熵约束的指数形式的概率模型。通过实验,比较了最大信息熵模型和Naive Bayes、Bayes Net、SVM与C4.5决策树方法之间的精度、召回率、F-Measure,发现最大信息熵模型具有良好的综合性能,尤其在训练数据集样本数量有限的情况下仍然能保持较高的分类精度,在实际应用中具有广阔的前景。     

基于最大熵方法汉语基本短语分析

提出了一个汉语基本短语分析模型,将汉语短语的边界划分和短语标识分开,假定这两个过程相互独立,采用最大熵方法分别建立模型解决。最大熵模型的关键是如何选取有效的特征,文中给出了两个步骤相关的特征空间以及特征选择过程和算法。实验表明,模型的短语定界精确率达到95.27％,标注精确率达到96.2％。     

Maximum entropy principle with imprecise side-conditions III: Crisp continuous solutions

In this paper we consider the maximum entropy principle with imprecise side-conditions, where the imprecise side-conditions are modeled as fuzzy sets. In two previous papers our solution produced: (1) fuzzy discrete probability distributions and fuzzy probability density functions; and (2) crisp discrete probability distributions. In this paper we consider only continuous probability density functions and we have the constraint that the solution must be crisp (non-fuzzy).     

基于最大熵模型的语义块切分

语义块切分是HNC理论的重要课题,与以往的处理策略不同,采用统计建模的方法来解决这一问题。采用词语、词性、概念等信息组成特征模板,并应用增量方法进行特征选择,构建了一个基于最大熵模型的语义块切分系统。在HNC标注语料库上的测试取得了较好的效果,开放测试的正确率和召回率分别达到了83.78%和91.17%。     

基于最大熵马尔科夫模型的绩效评价方法

本文提出了一种基于最大熵马尔科夫模型的绩效评价方法.该方法采用马氏模型来定量化建模专家打分过程,采用特征函数表征打分规则,通过在训练集上最大化熵来获得符合专家经验的最优的打分模型.与传统方法相比,所提出的方法可以融合各种打分规则、专家经验和指标逻辑关系得到综合打分结果.为了提高模型的训练和打分的效率,本文提出了基于改进迭代算法的参数估计方法,并利用Viterbi算法进行快速打分计算.利用中国大洋协会绩效评价指标体系历史数据进行的仿真实验表明,与BP神经网络方法和最大熵方法进行对比,本文所提出的方法具有更高的打分正确率.     

基于最大熵模型的评价搭配识别*

在分析酒店评论文本倾向性过程中,针对某些评价词语所产生的歧义性问题,提出一种基于最大熵的评价搭配识别的方法。该方法通过构建极性词表,挖掘出评价词语类别作为语义特征,将其与词、词性、距离、否定词特征结合构成最大熵的复合模板,采用最大熵模型进行评价搭配识别。实验结果证明,采用构建的最大熵复合模板进行评价搭配识别具有较高的准确率和识别性能。     

基于最大熵模型预测蛋白质结构的分类

基于最大熵模型,构建一种简单的预测蛋白质序列结构分类的算法。不同性质的氨基酸组合,在特定结构的蛋白质二级结构中,出现的频率不同,通过在模体数据库Prosite中查找蛋白质序列匹配的模体,以10种氨基酸组合在序列中出现的频率,表示蛋白质序列的特征,构建相应的结构分类预测模型。最大熵模型用来确定蛋白质结构分类预测模型的参数。以自身一致性和Jackknife测试方法验证分类模型的准确性。结果表明新构建的方法简单、准确,综合性能优于一般的预测方法。     

基于最大熵分类器的Deep Web查询接口自动判定

Web中包含着海量的高质量信息,它们通常处在网络深处,无法被传统搜索引擎索引,将这样的资源称为Deep Web。因为查询接口是Deep Web的唯一入口,所以要获取Deep Web信息就必须判定哪些网页表单是Deep Web查询接口。由于最大熵模型可以综合观察到的各种相关或不相关的概率知识,对许多问题的处理都可以达到较好的结果。因此,基于最大熵模型的分类性能,利用最大熵分类算法自动判定查询接口。并通过实验,将最大熵分类法与其它常用分类方法进行了比较,结果显示它的分类性能优于Bayes方法和C4.5方法,与SVM方法相当,表明这是一种非常实用的查询接口分类方法。     

基于二维熵和轮廓特征的非结构化道路检测

针对非结构化道路场景复杂干扰因素较多、检测困难的问题,提出了一种基于轮廓特征和二维最大熵的道路检测算法。采用融合色彩特征不变量的二次二维最大熵分割算法对道路图像进行分割;利用边界跟踪算法提取分割图像的轮廓特征,根据道路区域的位置和几何特性选取最大轮廓;通过改进Mid-to-side算法进行边缘点搜索,用三阶道路模型重建道路边界,并对道路方向进行判断。实验结果表明,所提算法与传统算法相比,对三类不同场景下非结构化道路的检测准确率可提高25%左右,具有较强抗阴影干扰的能力,并能有效识别道路方向。     

On the performance of the flexible maximum entropy distributions within partially adaptive estimation

The partially adaptive estimation based on the assumed error distribution has emerged as a popular approach for estimating a regression model with non-normal errors. In this approach, if the assumed distribution is flexible enough to accommodate the shape of the true underlying error distribution, the efficiency of the partially adaptive estimator is expected to be close to the efficiency of the maximum likelihood estimator based on knowledge of the true error distribution. In this context, the maximum entropy distributions have attracted interest since such distributions have a very flexible functional form and nest most of the statistical distributions. Therefore, several flexible MaxEnt distributions under certain moment constraints are determined to use within the partially adaptive estimation procedure and their performances are evaluated relative to well-known estimators. The simulation results indicate that the determined partially adaptive estimators perform well for non-normal error distributions. In particular, some can be useful in dealing with small sample sizes. In addition, various linear regression applications with non-normal errors are provided.     

基于情感从属和最大熵模型的细粒度观点挖掘

目前许多观点挖掘方法挖掘粒度过大,导致反馈信息不足。为解决该问题,对标准LDA模型进行改进,提出主题情感联合最大熵LDA模型进行细粒度观点挖掘。首先,考虑到词的位置和语义信息,在传统LDA模型中加入最大熵组件来区分背景词、特征词和观点词,并对特征词和观点词进行局部和全局的划分;其次,在主题层和单词层之间加入情感层,实现词语级别的细粒度情感分析,并引入情感转移变量来处理情感从属关系,同时获取整篇评论和每个主题的情感极性,实验验证了所提模型和理论的有效性。     

Parallel energy-efficient coverage optimization with maximum entropy clustering in wireless sensor networks

Energy constraint is an important issue in wireless sensor networks. This paper proposes a parallel energy-efficient coverage optimization mechanism to optimize the positions of mobile sensor nodes based on maximum entropy clustering in large-scale wireless sensor networks. According to the models of coverage and energy, stationary nodes are partitioned into clusters by maximum entropy clustering. After identifying the boundary node of each cluster, the sensing area is divided for parallel optimization. A numerical algorithm is adopted to calculate the coverage metric of each cluster, while the lowest cost paths of the inner cluster are used to define the energy metric in which Dijkstra’s algorithm is utilized. Then cluster heads are assigned to perform parallel particle swarm optimization to maximize the coverage metric and minimize the energy metric where a weight coefficient between the two metrics is employed to achieve a tradeoff between coverage area and energy efficiency. Simulations of the optimization mechanism and a target tracking application verify that coverage performance can be guaranteed by choosing a proper weight coefficient for each cluster and energy efficiency is enhanced by parallel energy-efficient optimization.     

约束条件下BN参数最大熵模型扩展学习算法

在很多智能系统的参数建模时,用户往往面对建模样本稀少的困境。针对在小数据集条件下贝叶斯网络（BN）参数建模的问题,提出了一种约束数据最大熵BN参数学习算法（CDME）。首先利用小数据集估算BN参数,随后把定性的专家经验转换为不等式约束,并利用Bootstrap算法生成满足约束的一组参数候选集,再根据信息最大熵进行加权计算出BN参数。实验结果表明,当数据量充分时,CDME参数学习算法与经典的MLE算法的学习精度近似,表明了算法的正确性;在小数据集条件下,利用CDME算法,可以对BN进行参数建模,学习精度优于MLE算法和QMAP算法。CDME算法在实际故障诊断样本数据相对稀缺的条件下,获取了诊断BN模型参数,在此基础上完成的诊断推理结果也印证了算法的有效性,为小数据集条件下的参数建模提供了一条新途径。     

Social emotion classification of short text via topic-level maximum entropy model

With the rapid proliferation of Web 2.0, the identification of emotions embedded in user-contributed comments at the social web is both valuable and essential. By exploiting large volumes of sentimental text, we can extract user preferences to enhance sales, develop marketing strategies, and optimize supply chain for electronic commerce. Pieces of information in the social web are usually short, such as tweets, questions, instant messages, messages, and news headlines. Short text differs from normal text because of its sparse word co-occurrence patterns, which hampers efforts to apply social emotion classification models. Most existing methods focus on either exploiting the social emotions of individual words or the association of social emotions with latent topics learned from normal documents. In this paper, we propose a topic-level maximum entropy (TME) model for social emotion classification over short text. TME generates topic-level features by modeling latent topics, multiple emotion labels, and valence scored by numerous readers jointly. The overfitting problem in the maximum entropy principle is also alleviated by mapping the features to the concept space. An experiment on real-world short documents validates the effectiveness of TME on social emotion classification over sparse words.     

Design of a complex system based on the maximum entropy principle

A complex system is a system composed of many dynamic elements with mutual interactions. This paper proposes a unified approach for the design of an information processing system using a complex system. The method of design is based on the maximum entropy principle. After a detailed explanation, the proposed method is applied to the design of a spatial filter using a complex system. This work was presented, in part, at the International Symposium on Artificial Life and Robotics, Oita, Japan, February 18–20, 1996.     

基于最大熵模型的汉语标点句缺失话题自动识别初探

本文的任务是判别标点句缺失话题是上句的主语还是宾语,将该任务作为标点句缺失话题自动识别研究的切入点。首先归纳了判别这一任务的一系列字面特征和语义特征,然后结合规则和最大熵模型,进行自动判别实验。结果显示,对特定类别动词的实验F值达到82%。对实验结果的分析说明,动词特征和语义特征对判别该任务的作用最大,规则方法和统计方法在判别任务中不能偏废,精细化的知识对判别的性能有重要影响。     

An approach for constructing complex discriminating surfaces based on Bayesian interference of the maximum entropy

In this paper we present a comprehensive Maximum Entropy (MaxEnt) procedure for the classification tasks. This MaxEnt is applied successfully to the problem of estimating the probability distribution function (pdf) of a class with a specific pattern, which is viewed as a probabilistic model handling the classification task. We propose an efficient algorithm allowing to construct a non-linear discriminating surfaces using the MaxEnt procedure. The experiments that we carried out shows the performance and the various advantages of our approach.