An approach for constructing complex discriminating surfaces based on Bayesian interference of the maximum entropy

In this paper we present a comprehensive Maximum Entropy (MaxEnt) procedure for the classification tasks. This MaxEnt is applied successfully to the problem of estimating the probability distribution function (pdf) of a class with a specific pattern, which is viewed as a probabilistic model handling the classification task. We propose an efficient algorithm allowing to construct a non-linear discriminating surfaces using the MaxEnt procedure. The experiments that we carried out shows the performance and the various advantages of our approach.     

Constructing Maximum Entropy Language Models for Movie Review Subjectivity Analysis

Document subjectivity analysis has become an important aspect of web text content mining. This problem is similar to traditional text categorization, thus many related classification techniques can be adapted here. However, there is one significant difference that more language or semantic information is required for better estimating the subjectivity of a document. Therefore, in this paper, our focuses are mainly on two aspects. One is how to extract useful and meaningful language features, and the other is how to construct appropriate language models efficiently for this special task. For the first issue, we conduct a Global-Filtering and Local-Weighting strategy to select and evaluate language features in a series of n-grams with different orders and within various distance-windows. For the second issue, we adopt Maximum Entropy （MaxEnt） modeling methods to construct our language model framework. Besides the classical MaxEnt models, we have also constructed two kinds of improved models with Gaussian and exponential priors respectively. Detailed experiments given in this paper show that with well selected and weighted language features, MaxEnt models with exponential priors are significantly more suitable for the text subjectivity analysis task.     

A regularization framework for multiclass classification: A deterministic annealing approach

In this paper, we propose a general regularization framework for multiclass classification based on discriminant functions. Since the objective function in the primal optimization problem of this framework is always not differentiable, the optimal solution cannot be obtained directly. With the aid of the deterministic annealing approach, a differentiable objective function is derived subject to a constraint on the randomness of the solution. The problem can be approximated by solving a sequence of differentiable optimization problems, and such approximation converges to the original problem asymptotically. Based on this approach, class-conditional posterior probabilities can be calculated directly without assuming the underlying probabilistic model. We also notice that there is a connection between our approach and some existing statistical models, such as Fisher discriminant analysis and logistic regression.     

Maximum entropy principle with imprecise side-conditions III: Crisp continuous solutions

In this paper we consider the maximum entropy principle with imprecise side-conditions, where the imprecise side-conditions are modeled as fuzzy sets. In two previous papers our solution produced: (1) fuzzy discrete probability distributions and fuzzy probability density functions; and (2) crisp discrete probability distributions. In this paper we consider only continuous probability density functions and we have the constraint that the solution must be crisp (non-fuzzy).     

An integer program and a hybrid genetic algorithm for the university timetabling problem

The university timetabling problem (UTP) has been studied by numerous research groups for decades. In addition to addressing hard and soft constraints, we extend the UTP by considering consecutiveness and periodicity constraints of multi-session lectures, which are common in many eastern Asian universities. Because schedulers can decide the consecutiveness and periodicity constraints for the multi-session lectures within a limited ratio, we consider these novel decision variables in our model. We develop a mixed integer linear program for the UTP. For the analysis, we convert the UTP into the three-dimensional container packing problem (3DCPP) and create a hybrid genetic algorithm (HGA), which has been shown to be efficient in solving the 3DCPP. We also develop a tabu search algorithm based on the existing UTP literature and compare the findings with that of our HGA. The results show that our HGA obtains a better solution than the tabu search algorithm in a reasonable amount of time.     

Minimizing maximum tardiness and delivery costs in a batched delivery system

High delivery costs usually urge manufacturers to dispatch their jobs in batches. However, dispatching the jobs in batches can have profound negative effects on important scheduling objective functions such as minimizing maximum tardiness. This paper considers a single machine scheduling problem with the aim of minimizing the maximum tardiness and delivery costs in a single-machine scheduling problem with batched delivery system. A mathematical model is developed for this problem which can serve to solve it with the help of a commercial solver. However, due to the fact that this model happens to be a mixed integer nonlinear programming model the solver cannot guarantee to reach the global solution. For this reason, a branch and bound algorithm (B&B) is presented to obtain the global solution. Besides, a heuristic algorithm for calculation of the initial upper bound is introduced. Computational results show that the algorithm can be beneficial for solving this problem, especially for large size instances.     

基于Power图求解容量限制P-中值问题

针对稠密需求下连续域上的容量P-中值问题,提出基于质心的容量限制Power图(CCCPD)理论,对连续P-中值问题进行近似建模,并加快计算过程。扩展Balzer试位法构造Power图,施加质心限制满足P-中值要求,施加容量限制满足需求密度下的容量要求。实验结果表明所提算法可快速得到近似可行解,同Alper Murata方法相比,计算效率高;同质心容量限制Voronoi图(CCCVT)相比,具有容量限制精确度高等优点,并能适应各种复杂需求密度函数。     

Consistency techniques for continuous constraints

We consider constraint satisfaction problems with variables in continuous, numerical domains. Contrary to most existing techniques, which focus on computing one single optimal solution, we address the problem of computing a compact representation of the space of all solutions admitted by the constraints. In particular, we show how globally consistent (also called decomposable) labelings of a constraint satisfaction problem can be computed.Our approach is based on approximating regions of feasible solutions by 2 ^k -trees, a representation commonly used in computer vision and image processing. We give simple and stable algorithms for computing labelings with arbitrary degrees of consistency. The algorithms can process constraints and solution spaces of arbitrary complexity, but with a fixed maximal resolution.Previous work has shown that when constraints are convex and binary, path-consistency is sufficient to ensure global consistency. We show that for continuous domains, this result can be generalized to ternary and in fact arbitrary n-ary constraints using the concept of (3,2)-relational consistency. This leads to polynomial-time algorithms for computing globally consistent labelings for a large class of constraint satisfaction problems with continuous variables.     

Interprocessor communication with limited memory

Many parallel applications require periodic redistribution of workloads and associated data. In a distributed memory computer, this redistribution can be difficult if limited memory is available for receiving messages. We propose a model for optimizing the exchange of messages under such circumstances which we call the minimum phase remapping problem. We first show that the problem is NP-complete, and then analyze several methodologies for addressing it. First, we show how the problem can be phrased as an instance of multicommodity flow. Next, we study a continuous approximation to the problem. We show that this continuous approximation has a solution which requires at most two more phases than the optimal discrete solution, but the question of how to consistently obtain a good discrete solution from the continuous problem remains open. We also devise a simple and practical approximation algorithm for the problem with a bound of 1.5 times the optimal number of phases. We also present an empirical study of variations of our algorithms which indicate that our approaches are quite practical.     

On the performance of the flexible maximum entropy distributions within partially adaptive estimation

The partially adaptive estimation based on the assumed error distribution has emerged as a popular approach for estimating a regression model with non-normal errors. In this approach, if the assumed distribution is flexible enough to accommodate the shape of the true underlying error distribution, the efficiency of the partially adaptive estimator is expected to be close to the efficiency of the maximum likelihood estimator based on knowledge of the true error distribution. In this context, the maximum entropy distributions have attracted interest since such distributions have a very flexible functional form and nest most of the statistical distributions. Therefore, several flexible MaxEnt distributions under certain moment constraints are determined to use within the partially adaptive estimation procedure and their performances are evaluated relative to well-known estimators. The simulation results indicate that the determined partially adaptive estimators perform well for non-normal error distributions. In particular, some can be useful in dealing with small sample sizes. In addition, various linear regression applications with non-normal errors are provided.     

Special Issue on Latin-American Research: A Time Based Discretization Approach for Ship Routing and Scheduling with Variable Speed

In this paper we develop a network based model for the routing and scheduling of a heterogeneous tramp fleet. The objective of the problem is to serve a known set of single trip cargo contracts, observing time window constraints at both origin and destination of cargoes, while minimizing total operating cost. A distinctive aspect of the methodology is that time windows for picking and delivering cargoes are discretized. This approach allows for a broad variety of features and practical constraints to be implicitly included in the model. In particular, we consider problems where navigation speed can be used to control fuel consumption, which is a main operating cost in ocean shipping. We performed a computational study on three set of fifteen problem instances each, involving 30, 40 and 50 cargoes per instance, respectively. Each problem instance was solved with two fleet sizes, three levels of discretization, and with constant and variable speed. The numerical results show that our model presents a much better trade-off between solution quality and computing time than a similar constant speed continuous model. For example, discretizing the time windows in as few as 3 points, we obtained solutions that in average were no more than 0,8% worse than the best solution found by the continuous model. Computing time, on the other side, decreased in at least two orders of magnitude. The results also confirm that significant benefits might be obtained by incorporating the navigation speed as a controllable variable in the model.     

基于最大熵原理的汉语词义消歧

词义消歧是自然语言处理中亟待解决的一个关键问题,本文提出一种基于最大熵模型的有监督的机器学习方法,用于汉语词义消歧。该方法综合了词标记、词性、主题等上下文特征,并用一种统一的表示方法规范化特征形式,解决了多种不同特征之间的融合和特征的知识表示。实验对20个汉语高频多义词进行了测试,平均正确率为87％,验证了该方法的有效性。     

The impact of lot-sizing in multiple product environments with congestion

We present a production planning model for a multiple product single machine dynamic lot-sizing problem with congestion. Using queuing models, we develop a set of functions to capture the nonlinear relationship between the output, lot sizes and available work in process inventory levels of all products in the system. We then embed these functions in a nonlinear optimization model with continuous variables, and construct an approximate solution to the original problem by rounding the resulting fractional solution. Computational experiments show that our model with congestion provides significantly better flow time and inventory performance than a benchmark model that does not consider the effects of congestion. These advantages arise from the use of multiple smaller lots in a period instead of a single large lot as suggested by conventional fixed-charge models without congestion.     

The capacitated multi-AGV scheduling problem with conflicting products: Model and a decentralized multi-agent approach

Automated guided vehicles (AGVs) are a key technology to facilitate flexible production systems in the context of Industry 4.0. This paper investigates an optimization model and a solution using a decentralized multi-agent approach for a new capacitated multi-AGV scheduling problem with conflicting products (CMASPCP) to take full advantage of AGVs. The novelty of the problem and our model lies in the introduction of AGV capacity constraints and constraints arising from conflicting products, i.e. products that cannot be transported together. As the new I4.0 paradigm tends towards decentralized control, we also present a decentralized multi-agent approach in which AGVs autonomously coordinate to solve the task. The performance of the proposed decentralized approach is compared to a mixed-integer linear programming model on a set of 110 problem instances with different sizes and degrees of complexity. The obtained results show that the proposed decentralized multi-agent approach is effective and competitive in terms of the solution quality and computational time.     

A new maximum-likelihood phase estimation method for X-ray pulsar signals

X-ray pulsar navigation （XPNAV） is an attractive method for autonomous navigation of deep space in the future. Currently, techniques for estimating the phase of X-ray pulsar radiation involve the maximization of the general non-convex object functions based on the average profile fxom the epoch folding method. This results in the suppression of useful information and highly complex computation. In this paper, a new maximum likelihood （ML） phase estimation method that directly utilizes the measured time of arrivals （TOAs） is presented. The X-ray pulsar radiation will be treated as a cyclo-stationary process and the TOAs of the photons in a period will be redefined as a new process, whose probability distribution function is the normalized standard profile of the pulsar. We demonstrate that the new process is equivalent to the generally used Poisson model. Then, the phase estimation problem is recast as a cyclic shift parameter estimation under the ML estimation, and we also put forward a parallel ML estimation method to improve the ML solution. Numerical simulation results show that the estimator described here presents a higher precision and reduces the computational complexity compared with currently used estimators.     

Maximum entropy principle with imprecise side-conditions II: crisp discrete solutions

In this paper we consider the maximum entropy principle with imprecise side-conditions, where the imprecise side-conditions are modeled as fuzzy sets. In a previous paper our solution produced fuzzy discrete probability distributions and fuzzy probability density functions. In this paper we consider only discrete probability distributions and we have the constraint that the solution must be crisp (non-fuzzy).     

Maximum entropy based significance of itemsets

We consider the problem of defining the significance of an itemset. We say that the itemset is significant if we are surprised by its frequency when compared to the frequencies of its sub-itemsets. In other words, we estimate the frequency of the itemset from the frequencies of its sub-itemsets and compute the deviation between the real value and the estimate. For the estimation we use Maximum Entropy and for measuring the deviation we use Kullback–Leibler divergence. A major advantage compared to the previous methods is that we are able to use richer models whereas the previous approaches only measure the deviation from the independence model. We show that our measure of significance goes to zero for derivable itemsets and that we can use the rank as a statistical test. Our empirical results demonstrate that for our real datasets the independence assumption is too strong but applying more flexible models leads to good results.

结合相似性拟合与空间约束的图像分割

目的 图像中的目标一般含有很多子类,仅仅利用某个子类的特征无法完整地分割出目标区域。针对这一问题,提出一种结合相似性拟合与空间约束的图像交互式分割方法。方法 首先,通过手工标记的样本组成各个目标的字典,通过相似度量搜寻测试样本与各个目标的字典中最相似的原子建立拟合项;再结合图像的空间约束项,构建图像分割模型;最后利用连续最大流算法求解,快速实现图像分割的目的。结果 通过对比实验,本文方法的速度比基于稀疏表示的分类方法的速度提高约13倍,而与归一化切割(N-Cut),逻辑回归(logistic regression)等方法相比,本文方法能取得更稳定和准确的分割结果。此外,本文方法无需过完备字典,只需要训练样本能体现各个子类的信息即可得到稳定的图像分割结果。结论 本文交互式图像分割方法,通过结合相似性拟合以及空间约束建立分割模型,并由连续最大流算法求解,实现图像的快速准确的分割。实验结果表明,该方法能够胜任较准确地对自然图像进行分割以及目标提取等任务。     

Vector Image Segmentation by Piecewise Continuous Approximation

In this article we present an approach to the segmentation problem by a piecewise approximation of the given image with continuous functions. Unlike the common approach of Mumford and Shah in our formulation of the problem the number of segments is a parameter, which can be estimated. The problem can be stated as: Compute the optimal segmentation with a fixed number of segments, then reduce the number of segments until the segmentation result fulfills a given suitability. This merging algorithm results in a multi-objective optimization, which is not only resolved by a linear combination of the contradicting error functions. To constrain the problem we use a finite dimensional vector space of functions in our approximation and we restrict the shape of the segments. Our approach results in a multi-objective optimization: On the one hand the number of segments is to be minimized, on the other hand the approximation error should also be kept minimal. The approach is sound theoretically and practically: We show that for L ²-images a Pareto-optimal solution exists and can be computed for the discretization of the image efficiently.