A hybrid radial basis function and data envelopment analysis neural network for classification

We propose a hybrid radial basis function network-data envelopment analysis (RBFN-DEA) neural network for classification problems. The procedure uses the radial basis function to map low dimensional input data from input space ℜ to a high dimensional ℜ⁺ feature space where DEA can be used to learn the classification function. Using simulated datasets for a non-linearly separable binary classification problem, we illustrate how the RBFN-DEA neural network can be used to solve it. We also show how asymmetric misclassification costs can be incorporated in the hybrid RBFN-DEA model. Our preliminary experiments comparing the RBFN-DEA with feed forward and probabilistic neural networks show that the RBFN-DEA fares very well.     

On the equivalence between nonnegative tensor factorization and tensorial probabilistic latent semantic analysis

Non-negative Matrix Factorization (NMF) and Probabilistic Latent Semantic Analysis (PLSA) are two widely used methods for non-negative data decomposition of two-way data (e.g., document-term matrices). Studies have shown that PLSA and NMF (with the Kullback-Leibler divergence objective) are different algorithms optimizing the same objective function. Recently, analyzing multi-way data (i.e., tensors), has attracted a lot of attention as multi-way data have rich intrinsic structures and naturally appear in many real-world applications. In this paper, the relationships between NMF and PLSA extensions on multi-way data, e.g., NTF (Non-negative Tensor Factorization) and T-PLSA (Tensorial Probabilistic Latent Semantic Analysis), are studied. Two types of T-PLSA models are shown to be equivalent to two well-known non-negative factorization models: PARAFAC and Tucker3 (with the KL-divergence objective). NTF and T-PLSA are also compared empirically in terms of objective functions, decomposition results, clustering quality, and computation complexity on both synthetic and real-world datasets. Finally, we show that a hybrid method by running NTF and T-PLSA alternatively can successfully jump out of each other’s local minima and thus be able to achieve better clustering performance.     

Spatially-weighted nonnegative matrix factorization with application to temporal psychovisual modulation

Nonnegative Matrix Factorization (NMF), which decomposes a target matrix into the product of two matrices with nonnegative elements, has been widely used in various fields of signal processing. In visual signal processing, the spatially nonuniformed distribution of perceptually meaningful information in image and video frames calls for a kind of Spatially-Weighted NMF (swNMF) that applies location dependent weights into the decomposition problem. In this paper we introduce swNMF solution based on the hierarchical alternating least squares (HALS) approach. Then we exemplify its application to a new information display diagram named temporal psychovisual modulation (TPVM) with comparison with traditional HALS method and baseline algorithm of multiplicative update (MU).     

Manufacturing cell formation using an improved P-median model

The p-median model objective function is modified for the cell formation problem to minimize the variability between parts in a group by considering part similarity to all other parts in the group instead of similarity to an arbitrary median. The heuristic vertex substitution method for solution of the part grouping problem is adapted for this objective function and then modified to provide improved starting points. The theoretical lower bound for the heuristic is developed and shown to be valid for all solutions. Worst case run time is shown to be O(n²) or O(n³) for distance matrix or network inputs respectively. Tests on published problems show that the proposed p-median model method provides as good or better objective function value (OFV) than the OFV of a p-median model in which parts are grouped to an arbitrary median. Likewise the new p-median model is shown, for these published problems, to give as good or better OFV than the algorithms reported by the original authors of the problem. The test problems suggest that other measures of solution quality such as bottlenecks and duplicate machines in addition to OFV become important measures of solution quality for dense problems.     

Manifold-respecting discriminant nonnegative matrix factorization

Nonnegative matrix factorization (NMF) is an unsupervised learning method for low-rank approximation of nonnegative data, where the target matrix is approximated by a product of two nonnegative factor matrices. Two important ingredients are missing in the standard NMF methods: (1) discriminant analysis with label information; (2) geometric structure (manifold) in the data. Most of the existing variants of NMF incorporate one of these ingredients into the factorization. In this paper, we present a variation of NMF which is equipped with both these ingredients, such that the data manifold is respected and label information is incorporated into the NMF. To this end, we regularize NMF by intra-class and inter-class k-nearest neighbor (k-NN) graphs, leading to NMF-kNN, where we minimize the approximation error while contracting intra-class neighborhoods and expanding inter-class neighborhoods in the decomposition. We develop simple multiplicative updates for NMF-kNN and present monotonic convergence results. Experiments on several benchmark face and document datasets confirm the useful behavior of our proposed method in the task of feature extraction.     

Polynomial-time primal simplex algorithms for the minimum cost network flow problem

We present two variants of the primal network simplex algorithm which solve the minimum cost network flow problem in at mostO(n ² logn) pivots. Here we define the network simplex method as a method which proceeds from basis tree to adjacent basis tree regardless of the change in objective function value; i.e., the objective function is allowed to increase on some iterations. The first method is an extension of theminimum mean augmenting cycle-canceling method of Goldberg and Tarjan. The second method is a combination of a cost-scaling technique and a primal network simplex method for the maximum flow problem. We also show that the diameter of the primal network flow polytope is at mostn ² m.     

结合空间光谱预处理和约束非负矩阵分解的高光谱图像混合像元分解

目的 混合像元问题在高光谱遥感图像处理分析中普遍存在,非负矩阵分解的方法被引入到高光谱图像解混中。本文提出结合空间光谱预处理和约束非负矩阵分解的混合像元分解流程。方法 结合空间光谱预处理的约束非负矩阵分解,如最小体积约束、流行约束等,通过加入邻域的空间和光谱信息进行预处理获得更优的预选端元,从而对非负矩阵分解的解混结果进行优化。结果 在5组不同信噪比的模拟数据实验中,空间预处理（SPP）和空间光谱预处理（SSPP）均能够有效提高约束非负矩阵分解（最小体积约束的非负矩阵分解和图正则非负矩阵分解）的解混结果,其中SPP在不同信噪比的情况下都能优化约束非负矩阵分解的结果,而SSPP在低信噪比的情况下,预处理效果更佳。利用美国内华达州Cuprite矿区数据进行真实数据实验,SPP提高了约束非负矩阵分解的解混精度,而SSPP在复杂场景下,解混精度更佳。模拟数据和真实数据的实验均表明,空间光谱预处理能够有效地提高约束非负矩阵分解的解混精度,特别是对于信噪比较低的情况下,融合空间和光谱信息对噪声有很好的鲁棒性。结论 本文对约束非负矩阵分解的解混算法添加空间光谱预处理,利用高光谱遥感数据的空间和光谱信息,优化预选端元,加入空间光谱预处理的非负矩阵解混实验流程,在复杂场景情况下,对噪声具有较好的鲁棒性。     

Piecewise cubic monotone interpolation with assigned slopes

An algorithm to construct a monotonicity preserving cubicC ¹ interpolant without modification of the assigned slopes is proposed. AnO(h ⁴) convergence result is obtained when exact function and derivative values are available andO(h ^p ) convergence can be obtained withp=min(4,q) forO(h ^q ) accurate function and derivative values. Numerical experiments carried out on data coming from functions with very different behaviours are presented. The results show that the method can interpolate monotone data in a visually pleasing way, even for data which present rapid variations.     

Business failure prediction using hybrid2 case-based reasoning (H2CBR)

We have investigated business failure prediction (BFP) by a combination of decision-aid, statistical, and artificial intelligence techniques. The goal is to construct a hybrid forecasting method for BFP by combining various outranking preference functions with case-based reasoning (CBR), whose heart is the k-nearest neighbor (k-NN) algorithm, and to empirically test the predictive performance of its modules. The hybrid² CBR (H²CBR) forecasting method was constructed by integrating six hybrid CBR modules. These hybrid CBR modules were built up by combining and modifying six outranking preference functions with the algorithm of k-NN inside CBR. A trial-and-error iterative process was employed to identify the optimal hybrid CBR module of the H²CBR forecasting system. The prediction of the optimal module is the final output of the H²CBR forecasting method. We have compared the predictive performance of the six hybrid CBR modules in BFP of Chinese listed companies. In this empirical study, the classical CBR algorithm based on the Euclidean metric, and the two classical statistical methods of logistic regression (Logit) and multivariate discriminant analysis (MDA) were used as baseline models for comparison. Feature subsets were selected with the stepwise method of MDA. The predictive performance of the H²CBR system is promising; the most preferred hybrid CBR for short-term BFP of Chinese listed companies is based on the ranking-order preference function.     

Efficient Nonnegative Matrix Factorization via projected Newton method

Nonnegative Matrix Factorization (NMF) is a popular decomposition technique in pattern analysis, document clustering, image processing and related fields. In this paper, we propose a fast NMF algorithm via Projected Newton Method (PNM). First, we propose PNM to efficiently solve a nonnegative least squares problem, which achieves a quadratic convergence rate under appropriate assumptions. Second, in the framework of an alternating optimization method, we adopt PNM as an essential subroutine to efficiently solve the NMF problem. Moreover, by exploiting the low rank assumption of NMF, we make PNM very suitable for solving NMF efficiently. Empirical studies on both synthetic and real-world (text and image) data demonstrate that PNM is quite efficient to solve NMF compared with several state of the art algorithms.     

An intelligent forecasting model based on robust wavelet ν-support vector machine

Aiming at the problem of small samples, season character, nonlinearity, randomicity and fuzziness in product demand series, the existing support vector kernel does not approach the random curve of the demands time series in the L²(Rⁿ) space (quadratic continuous integral space). The robust loss function is also proposed to solve the shortcoming of ε-insensitive loss function during handling hybrid noises. A novel robust wavelet support vector machine (RW ν-SVM) is proposed based on wavelet theory and the modified support vector machine. Particle swarm optimization algorithm is designed to select the optimal parameters of RW ν-SVM model in the scope of constraint permission. The results of application in car demand forecasts show that the forecasting approach based on the RW ν-SVM model is effective and feasible, the comparison between the method proposed in this paper and other ones is also given which proves this method is better than RW ν-SVM and other traditional methods.     

基于图正则化和稀疏约束的半监督非负矩阵分解

非负矩阵分解是在矩阵非负约束下的分解算法。为了提高识别率,提出了一种基于稀疏约束和图正则化的半监督非负矩阵分解方法。该方法对样本数据进行低维非负分解时,既保持数据的几何结构,又利用已知样本的标签信息进行半监督学习,而且对基矩阵施加稀疏性约束,最后将它们整合于单个目标函数中。构造了一个有效的更新算法,并且在理论上证明了该算法的收敛性。在多个人脸数据库上的仿真结果表明,相对于NMF、GNMF、CNMF等算法,GCNMFS具有更好的聚类精度和稀疏性。     

基于图正则化MNMF的中文垃圾邮件过滤

利用向量空间模型表示的文本邮件数据具有高维性, 不利于邮件过滤模型的建立, 需要对数据进行降维处理。最大间隔Semi-NMF(max-margin semi-nonnegative matrix factorization, MNMF)能够同时实现维数约减和邮件分类, 而图正则化NMF能保持数据空间的几何结构。基于以上两种NMF改进模型, 提出了图正则化MNMF(graph regularized MNMF, GMNMF)算法, 并设计了一个迭代的求解算法。将GMNMF算法及其他相关算法用于中文垃圾邮件过滤实验, 结果表明GMNMF算法构建的过滤模型要优于其他较好的算法构建的过滤模型。     

基于感知掩蔽的重构非负矩阵分解单通道语音增强算法

针对非负矩阵分解（NMF）语音增强算法在低信噪比（SNR）非稳定环境下存在噪声残留的问题,提出一种基于感知掩蔽的重构NMF（PM-RNMF）单通道语音增强算法。首先,将心理声学掩蔽特性应用于NMF语音增强算法中;其次,对不同频率位采用不同的掩蔽阈值,建立自适应感知掩蔽增益函数,通过阈值约束残余噪声能量和语音失真能量;最后,结合语音存在概率（SPP）进行感知增益修正,重构NMF算法,以此建立新的目标函数。仿真结果表明,在不同SNR的3种非稳定噪声环境下,与NMF、重构NMF（RNMF）、感知掩蔽深度神经网络（PM-DNN）算法相比,PM-RNMF算法的感知语音质量评估（PESQ）平均值分别提高了0.767、0.474、0.162,信源失真比（SDR）平均值分别提高了2.785、1.197、0.948。实验结果表明,无论是在低频还是高频PM-RNMF有更好的降噪效果。     

Efficient storage and retrieval of probabilistic latent semantic information for information retrieval

Probabilistic latent semantic analysis (PLSA) is a method for computing term and document relationships from a document set. The probabilistic latent semantic index (PLSI) has been used to store PLSA information, but unfortunately the PLSI uses excessive storage space relative to a simple term frequency index, which causes lengthy query times. To overcome the storage and speed problems of PLSI, we introduce the probabilistic latent semantic thesaurus (PLST); an efficient and effective method of storing the PLSA information. We show that through methods such as document thresholding and term pruning, we are able to maintain the high precision results found using PLSA while using a very small percent (0.15%) of the storage space of PLSI.     

Unified framework for representing and ranking

In the database retrieval and nearest neighbor classification tasks, the two basic problems are to represent the query and database objects, and to learn the ranking scores of the database objects to the query. Many studies have been conducted for the representation learning and the ranking score learning problems, however, they are always learned independently from each other. In this paper, we argue that there are some inner relationships between the representation and ranking of database objects, and try to investigate their relationships by learning them in a unified way. To this end, we proposed the Unified framework for Representation and Ranking (UR²) of objects for the database retrieval and nearest neighbor classification tasks. The learning of representation parameter and the ranking scores are modeled within one single unified objective function. The objective function is optimized alternately with regard to representation parameter and the ranking scores. Based on the optimization results, iterative algorithms are developed to learn the representation parameter and the ranking scores on a unified way. Moreover, with two different formulas of representation (feature selection and subspace learning), we give two versions of UR². The proposed algorithms are tested on two challenging tasks – MRI image based brain tumor retrieval and nearest neighbor classification based protein identification. The experiments show the advantage of the proposed unified framework over the state-of-the-art independent representation and ranking methods.     

Scheduling problems with two competing agents to minimized weighted earliness–tardiness

We study scheduling problems with two competing agents, sharing the same machines. All the jobs of both agents have identical processing times and a common due date. Each agent needs to process a set of jobs, and has his own objective function. The objective of the first agent is total weighted earliness–tardiness, whereas the objective of the second agent is maximum weighted deviation from the common due date. Our goal is to minimize the objective of the first agent, subject to an upper bound on the objective value of the second agent. We consider a single machine, and parallel (both identical and uniform) machine settings. An optimal solution in all cases is shown to be obtained in polynomial time by solving a number of linear assignment problems. We show that the running times of the single and the parallel identical machine algorithms are O(n^m+3), where n is the number of jobs and m is the number of machines. The algorithm for solving the problem on parallel uniform machine requires O(n^m+3m³) time, and under very reasonable assumptions on the machine speeds, is reduced to O(n^m+3). Since the number of machines is given, these running times are polynomial in the number of jobs.     

A hybrid algorithm based on particle swarm and chemical reaction optimization

In this paper, a hybrid method for optimization is proposed, which combines the two local search operators in chemical reaction optimization with global search ability of for global optimum. This hybrid technique incorporates concepts from chemical reaction optimization and particle swarm optimization, it creates new molecules (particles) either operations as found in chemical reaction optimization or mechanisms of particle swarm optimization. Moreover, some technical bound constraint handling has combined when the particle update in particle swarm optimization. The effects of model parameters like InterRate, γ, Inertia weight and others parameters on performance are investigated in this paper. The experimental results tested on a set of twenty-three benchmark functions show that a hybrid algorithm based on particle swarm and chemical reaction optimization can outperform chemical reaction optimization algorithm in most of the experiments. Experimental results also indicate average improvement and deviate over chemical reaction optimization in the most of experiments.     

A novel classification method: A hybrid approach based on extension of the UTADIS with polynomial and PSO-GA algorithm

In this study, a new multi-criteria classification technique for nominal and ordinal groups is developed by expanding the UTilites Additives DIScriminantes (UTADIS) method with a polynomial of degree T which is used as the utility function rather than using a piecewise linear function as an approximation of the utility function of each attribute. We called this method as PUTADIS. The objective is calculating the coefficients of the polynomial and the threshold limit of classes and weight of attributes such that it minimizes the number of misclassification error. Estimation of unknown parameters of the problem is calculated by using a hybrid algorithm which is a combination of particle swarm optimization algorithm (PSO) and Genetic Algorithm (GA). The results obtained by implementing the model on different datasets and comparing its performance with other previous methods show the high efficiency of the proposed method.     

Uniform convergence and a posteriori error estimation for assumed stress hybrid finite element methods

Assumed stress hybrid methods are known to improve the performance of standard displacement-based finite elements and are widely used in computational mechanics. The methods are based on the Hellinger–Reissner variational principle for the displacement and stress variables. This work analyzes two existing 4-node hybrid stress quadrilateral elements due to Pian and Sumihara [T.H.H. Pian, K. Sumihara, Rational approach for assumed stress finite elements, Int. J. Numer. Methods Engrg. 20 (9) (1984) 1685–1695] and due to Xie and Zhou [X.P. Xie, T.X. Zhou, Optimization of stress modes by energy compatibility for 4-node hybrid quadrilaterals, Int. J. Numer. Methods Engrg. 59 (2004) 293–313], which behave robustly in numerical benchmark tests. For the finite elements, the isoparametric bilinear interpolation is used for the displacement approximation, while different piecewise-independent 5-parameter modes are employed for the stress approximation. We show that the two schemes are free from Poisson-locking, in the sense that the error bound in the a priori estimate is independent of the relevant Lamé constant λ. We also establish the equivalence of the methods to two assumed enhanced strain schemes. Finally, we derive reliable and efficient residual-based a posteriori error estimators for the stress in L²-norm and the displacement in H¹-norm, and verify the theoretical results by some numerical experiments.