密度敏感的半监督谱聚类

聚类通常被认为是一种无监督的数据分析方法,然而在实际问题中可以很容易地获得有限的样本先验信息,如样本的成对限制信息.大量研究表明,在聚类搜索过程中充分利用先验信息会显著提高聚类算法的性能.首先分析了在聚类过程中仅利用成对限制信息存在的不足,尝试探索数据集本身固有的先验信息--空间一致性先验信息,并提出利用这类先验信息的具体方法.接着,将两类先验信息同时引入经典的谱聚类算法中,提出一种密度敏感的半监督谱聚类算法(density-sensitive semi-supervised spectral clustering algorithm,简称DS-SSC).两类先验信息在指导聚类搜索的过程中能够起到相辅相成的作用,这使得DS-SSC算法相对于仅利用成对限制信息的聚类算法在聚类性能上有了显著的提高.在UCI基准数据集、USPS手写体数字集以及TREC的文本数据集上的实验结果验证了这一点.     

The cubic spline solution of practical problems modelled by hyperbolic partial differential equations

An approximation is developed for a general one-dimensional hyperbolic partial differential equation with constant coefficients and function value boundary conditions. The time derivative is replaced by a finite difference representation and the space derivative by a cubic spline. As expected, a three-level finite difference formula in time is obtained giving the solution at each succeeding time level. The spline approximation produces a spline function which can be used on each time level to obtain the solution at any points intermediate to the mesh points. The numerical scheme is extended to the more general variable-coefficient case and for derivative boundary conditions. Truncation errors and stability criteria are produced, and the scheme is rigorously tested on practical problems, a comparison being made with a more well-known fully implicit finite difference scheme.     

基于多视角的多类型错误全面检测方法

随着信息化社会的发展,数据的规模越发庞大,数据的种类也越发丰富.时至今日,数据已经成为国家和企业的重要战略资源,是科学化管理的重要保障.然而,随着社会生活产生的数据日益丰富,大量的脏数据也随之而来,数据质量问题油然而生.如何准确而全面地检测出数据集中所包含的错误数据,一直是数据科学中的痛点问题.尽管已有许多传统方法被广泛用于各行各业,如基于约束与统计的检测方法,但这些方法通常需要丰富的先验知识与昂贵的人力和时间成本.受限于此,这些方法往往难以准确而全面地检测数据.近年来,许多新型错误检测方法利用深度学习技术,通过时序推断、文本解析等方式取得了更好检测效果,但它们通常只适用于特定的领域或特定的错误类型,面对现实生活中的复杂情况,泛用性不足.基于上述情况,结合传统方法与深度学习技术的优点,提出了一个基于多视角的多类型错误全面检测模型CEDM.首先,从模式的角度,结合现有约束条件,在属性、单元和元组层面进行多维度的统计分析,构建出基础检测规则;然后,通过词嵌入捕获数据语义,从语义的角度分析属性相关性、单元关联性与元组相似性,进而基于语义关系,从多个维度上更新、扩展基础规则;最终,联合多个视角...     

Trajectory association across multiple airborne cameras

A camera mounted on an aerial vehicle provides an excellent means for monitoring large areas of a scene. Utilizing several such cameras on different aerial vehicles allows further flexibility, in terms of increased visual scope and in the pursuit of multiple targets. In this paper, we address the problem of associating objects across multiple airborne cameras. Since the cameras are moving and often widely separated, direct appearance-based or proximity-based constraints cannot be used. Instead, we exploit geometric constraints on the relationship between the motion of each object across cameras, to test multiple association hypotheses, without assuming any prior calibration information. Given our scene model, we propose a likelihood function for evaluating a hypothesized association between observations in multiple cameras that is geometrically motivated. Since multiple cameras exist, ensuring coherency in association is an essential requirement, e.g. that transitive closure is maintained between more than two cameras. To ensure such coherency we pose the problem of maximizing the likelihood function as a k-dimensional matching and use an approximation to find the optimal assignment of association. Using the proposed error function, canonical trajectories of each object and optimal estimates of inter-camera transformations (in a maximum likelihood sense) are computed. Finally, we show that as a result of associating objects across the cameras, a concurrent visualization of multiple aerial video streams is possible and that, under special conditions, trajectories interrupted due to occlusion or missing detections can be repaired. Results are shown on a number of real and controlled scenarios with multiple objects observed by multiple cameras, validating our qualitative models, and through simulation quantitative performance is also reported.     

On the computation of reference signal constraints for guaranteed tracking performance

This paper concerns the problem of determining constraints on reference signals for tracking systems such that the tracking performance can be guaranteed within a specified tolerance for any reference signal satisfying the constraints. We first consider the problem of computing derivative constraints, which are linear constraints on the (vector-valued) reference signal and its time derivatives, and present an off-line algorithm for computing inner approximations to supremal derivative constraint sets based on the hyperplane method for generating inner and outer approximations to the reachable set of states for the controlled system. No simulation is required. We then consider planning problems in which a finite number of parameters are selected to generate the reference signal. The derivative constraints are mapped into this parameter space. The simplicial approximation method is proposed as a method for computing an approximation to the set of admissible parameters. The resulting (linear) parameter constraints characterize a class of reference signals which can be successfully executed by the tracking system, thereby permitting supervisory planning and control to be carried out in the reference signal parameter space without simulating detailed models of the underlying system dynamics. We illustrate the computational algorithm and the application of derivative and parameter constraints for the problem of generating trajectories for a two-axis computer numerical control (CNC) cutting tool.     

Some Recent Developments in Superconvergence of Discontinuous Galerkin Methods for Time-Dependent Partial Differential Equations

In this paper, we briefly review some recent developments in the superconvergence of three types of discontinuous Galerkin (DG) methods for time-dependent partial differential equations: the standard DG method, the local discontinuous Galerkin method, and the direct discontinuous Galerkin method. A survey of our own results for various time-dependent partial differential equations is presented and the superconvergence phenomena of the aforementioned three types of DG solutions are studied for: (i) the function value and derivative approximation at some special points, (ii) cell average error and supercloseness.     

The pseudospectral Legendre method for discretizing optimal controlproblems

This paper presents a computational technique for optimal control problems including state and control inequality constraints. The technique is based on spectral collocation methods used in the solution of differential equations. The system dynamics are collocated at Legendre-Gauss-Lobatto points. The derivative x˙(t) of the state x(t) is approximated by the analytic derivative of the corresponding interpolating polynomial. State and control inequality constraints are collocated at Legendre-Gauss-Lobatto nodes. The integral involved in the definition of the performance index is discretized based on the Gauss-Lobatto quadrature rule. The optimal control problem is thereby converted into a mathematical programming program. Thus existing, well-developed optimization algorithms may be used to solve the transformed problem. The method is easy to implement, capable of handling various types of constraints, and yields very accurate results. Illustrative examples are included to demonstrate the capability of the proposed method, and a comparison is made with existing methods in the literature     

Second‐Order Approximation for Variance Reduction in Multiple Importance Sampling

Monte Carlo Techniques are widely used in Computer Graphics to generate realistic images. Multiple Importance Sampling reduces the impact of choosing a dedicated strategy by balancing the number of samples between different strategies. However, an automatic choice of the optimal balancing remains a difficult problem. Without any scene characteristics knowledge, the default choice is to select the same number of samples from different strategies and to use them with heuristic techniques (e.g., balance, power or maximum). In this paper, we introduce a second‐order approximation of variance for balance heuristic. Based on this approximation, we introduce an automatic distribution of samples for direct lighting without any prior knowledge of the scene characteristics. We demonstrate that for all our test scenes (with different types of materials, light sources and visibility complexity), our method actually reduces variance in average. We also propose an implementation with low overhead for offline and GPU applications. We hope that this approach will help developing new balancing strategies.     

Signal and image approximation with level-set constraints

Summary We present a novel variational approach to signal and image approximation using filter statistics (histograms) as constraints. Given a set of linear filters, we study the problem to determine the closest point to given data while constraining the level-sets of the filter outputs. This criterion and the constraints are formulated as a bilevel optimization problem. We develop an algorithm by representing the lower-level problem through complementarity constraints and by applying an interior-penalty relaxation method. Based on a decomposition of the penalty term into the difference of two convex functions, the resulting algorithm approximates the data by solving a sequence of convex programs. Our approach allows to model and to study the generation of image structure through the interaction of two convex processes for spatial approximation and for preserving filter statistics, respectively.      

A new constrained learning algorithm for function approximation by encoding a priori information into feedforward neural networks

In this paper, a new learning algorithm which encodes a priori information into feedforward neural networks is proposed for function approximation problem. The new algorithm considers two kinds of constraints, which are architectural constraints and connection weight constraints, from a priori information of function approximation problem. On one hand, the activation functions of the hidden neurons are specific polynomial functions. On the other hand, the connection weight constraints are obtained from the first-order derivative of the approximated function. The new learning algorithm has been shown by theoretical justifications and experimental results to have better generalization performance and faster convergent rate than other algorithms.     

On robotic optimal path planning in polygonal regions with pseudo-Euclidean metrics.

This paper presents several results on some cost-minimizing path problems in polygonal regions. For these types of problems, an approach often used to compute approximate optimal paths is to apply a discrete search algorithm to a graph G(epsilon) constructed from a discretization of the problem; this graph is guaranteed to contain an epsilon-good approximate optimal path, i.e., a path with a cost within (1 + epsilon) factor of that of an optimal path, between given source and destination points. Here, epsilon > 0 is the user-defined error tolerance ratio. We introduce a class of piecewise pseudo-Euclidean optimal path problems that includes several non-Euclidean optimal path problems previously studied and show that the BUSHWHACK algorithm, which was formerly designed for the weighted region optimal path problem, can be generalized to solve any optimal path problem of this class. We also introduce an empirical method called the adaptive discretization method that improves the performance of the approximation algorithms by placing discretization points densely only in areas that may contain optimal paths. It proceeds in multiple iterations, and in each iteration, it varies the approximation parameters and fine tunes the discretization.     

移动最小二乘法导数近似讨论

在移动最小二乘法（moving least squares method, MLS）构造无网格形函数的数值方法中,通常采用无单元伽辽金法（element-free Galerkin method, EFG）的建议,将系数向量a参与导数运算。为探讨这种导数近似算法在更一般无网格法中的适用性和合理性,针对系数向量a是否应参与运算的问题进行讨论和数值检验。结果表明：单纯从近似意义上讲,这种将系数向量代入导数运算的算法并不具有优势;从数值方法的应用意义上讲,这种导数近似算法对数值求解,特别是强式无网格法,会带来一系列潜在不稳定的问题。建议在MLS导数近似中,系数向量a不应当参与导数运算,并提出采用一种由核基函数代替普通基函数的核近似法。     

考虑量化输入和输出约束的互联系统自适应分散跟踪控制

本文考虑具有量化输入和输出约束的一类非线性互联系统的自适应分散跟踪控制设计. 分别针对量化参数已知和未知两种情况, 基于反推(Backstepping)设计法, 利用神经网络逼近特性, 设计自适应分散跟踪控制策略. 通过定义新的未知常量和非线性光滑函数, 设计自适应参数估计项来消除未知互联项对系统的影响. 进一步考虑量化参数未知的情形, 引入一个新的不等式来转化输入信号, 并构建新的自适应补偿项来处理量化影响. 同时, 障碍李雅普诺夫函数的引入, 确保了系统输出不违反约束条件. 与现有量化输入设计相比, 本文所提方法不要求未知非线性项满足李普希兹条件, 并且允许量化参数未知. 该设计方法保证了闭环系统所有信号最终一致有界, 而且跟踪误差能够收敛到原点的小邻域内, 同时保证输出不违反约束条件. 最后, 仿真算例验证了所提方法具备良好的跟踪控制性能.     

Computation of probabilities of survival/failure of technical,economic systems/structures by means of piecewise linearization of the performance function

The computation of the probability of survival/failure of technical/economic structures and systems is based on an appropriate performance or so-called (limit) state function separating the safe and unsafe states in the space of random model parameters. Starting with the survival conditions, hence, the state equation and the condition for the admissibility of states, an optimizational representation of the state function can be given in terms of the minimum value function of a closely related minimization problem. Selecting a certain number of boundary points of the safe/unsafe domain, hence, on the limit state surface, the safe/unsafe domain is approximated by a convex polyhedron. This convex polyhedron is defined by the intersection of the half spaces in the parameter space generated by the tangent hyperplanes to the safe/unsafe domain at the selected boundary points on the limit state surface. The approximative probability functions are then defined by means of the resulting probabilistic linear constraints in the parameter space. After an appropriate transformation, the probability distribution of the parameter vector can be assumed to be normal with zero mean vector and unit covariance matrix. Working with separate linear constraints, approximation formulas for the probability of survival of the structure are obtained immediately. More exact approximations are obtained by considering joint probability constraints. In a second approximation step, these approximations can be evaluated by using probability inequalities and/or discretizations of the underlying probability distribution.     

Learning Bayesian network parameters under incomplete data with domain knowledge

Bayesian networks (BNs) have gained increasing attention in recent years. One key issue in Bayesian networks is parameter learning. When training data is incomplete or sparse or when multiple hidden nodes exist, learning parameters in Bayesian networks becomes extremely difficult. Under these circumstances, the learning algorithms are required to operate in a high-dimensional search space and they could easily get trapped among copious local maxima. This paper presents a learning algorithm to incorporate domain knowledge into the learning to regularize the otherwise ill-posed problem, to limit the search space, and to avoid local optima. Unlike the conventional approaches that typically exploit the quantitative domain knowledge such as prior probability distribution, our method systematically incorporates qualitative constraints on some of the parameters into the learning process. Specifically, the problem is formulated as a constrained optimization problem, where an objective function is defined as a combination of the likelihood function and penalty functions constructed from the qualitative domain knowledge. Then, a gradient-descent procedure is systematically integrated with the E-step and M-step of the EM algorithm, to estimate the parameters iteratively until it converges. The experiments with both synthetic data and real data for facial action recognition show our algorithm improves the accuracy of the learned BN parameters significantly over the conventional EM algorithm.     

Design sensitivity and reliability-based structural optimization by univariate decomposition

This paper presents a new univariate decomposition method for design sensitivity analysis and reliability-based design optimization of mechanical systems subject to uncertain performance functions in constraints. The method involves a novel univariate approximation of a general multivariate function in the rotated Gaussian space for reliability analysis, analytical sensitivity of failure probability with respect to design variables, and standard gradient-based optimization algorithms. In both reliability and sensitivity analyses, the proposed effort has been reduced to performing multiple one-dimensional integrations. The evaluation of these one-dimensional integrations requires calculating only conditional responses at selected deterministic input determined by sample points and Gauss–Hermite integration points. Numerical results indicate that the proposed method provides accurate and computationally efficient estimates of the sensitivity of failure probability, which leads to accurate design optimization of uncertain mechanical systems.     

A regularization framework for robust dimensionality reduction with applications to image reconstruction and feature extraction

Dimensionality reduction has many applications in pattern recognition, machine learning and computer vision. In this paper, we develop a general regularization framework for dimensionality reduction by allowing the use of different functions in the cost function. This is especially important as we can achieve robustness in the presence of outliers. It is shown that optimizing the regularized cost function is equivalent to solving a nonlinear eigenvalue problem under certain conditions, which can be handled by the self-consistent field (SCF) iteration. Moreover, this regularization framework is applicable in unsupervised or supervised learning by defining the regularization term which provides some types of prior knowledge of projected samples or projected vectors. It is also noted that some linear projection methods can be obtained from this framework by choosing different functions and imposing different constraints. Finally, we show some applications of our framework by various data sets including handwritten characters, face images, UCI data, and gene expression data.     

Exact solutions to some minimum-time problems and their behaviornear inequality state constraints

A heuristic method for generating exact solutions to certain minimum-time problems with inequality state constraints is used to generate solutions to a class of path-planning problems. It is observed that, when the state constraint function has a continuous second derivative, the constraint does not become active for any continuous-time period. Instead, the solution bumps up against the constraint repeatedly at isolated points. The solution method offers some insight into this behavior. It is shown that such a state constraint can become active for a continuous-time period only if the solution path satisfies an overdetermined system of equations. It is argued that the phenomenon is general and will arise in many different optimization problems     

The dual and degrees of freedom of linearly constrained generalized lasso

The lasso and its variants have attracted much attention recently because of its ability of simultaneous estimation and variable selection. When some prior knowledge exists in applications, the performance of estimation and variable selection can be further improved by incorporating the prior knowledge as constraints on parameters. In this article, we consider linearly constrained generalized lasso, where the constraints are either linear inequalities or equalities or both. The dual of the problem is derived, which is a much simpler problem than the original one. As a by-product, a coordinate descent algorithm is feasible to solve the dual. A formula for the number of degrees of freedom is derived. The method for selecting tuning parameter is also discussed.     

Multiple Empirical Kernel Learning with dynamic pairwise constraints

Unlike the traditional Multiple Kernel Learning (MKL) with the implicit kernels, Multiple Empirical Kernel Learning (MEKL) explicitly maps the original data space into multiple feature spaces via different empirical kernels. MEKL has been demonstrated to bring good classification performance and to be much easier in processing and analyzing the adaptability of kernels for the input space. In this paper, we incorporate the dynamic pairwise constraints into MEKL to propose a novel Multiple Empirical Kernel Learning with dynamic Pairwise Constraints method (MEKLPC). It is known that the pairwise constraint provides the relationship between two samples, which tells whether these samples belong to the same class or not. In the present work, we boost the original pairwise constraints and design the dynamic pairwise constraints which can pay more attention onto the boundary samples and thus to make the decision hyperplane more reasonable and accurate. Thus, the proposed MEKLPC not only inherits the advantages of the MEKL, but also owns multiple folds of prior information. Firstly, MEKLPC gets the side-information and boosts the classification performance significantly in each feature space. Here, the side-information is the dynamic pairwise constraints which are constructed by the samples near the decision boundary, i.e. the boundary samples. Secondly, in each mapped feature space, MEKLPC still measures the empirical risk and generalization risk. Lastly, different feature spaces mapped by multiple empirical kernels can agree to their outputs for the same input sample as much as possible. To the best of our knowledge, it is the first time to introduce the dynamic pairwise constraints into the MEKL framework in the present work. The experiments on a number of real-world data sets demonstrate the feasibility and effectiveness of MEKLPC.