Equivalence conditions for behaviors and the Kronecker canonical form

In this paper we explore equivalence conditions and invariants for behaviors given in kernel representations. In case the kernel representation is given in terms of a linear matrix pencil, the invariants for strict equivalence are given by the Kronecker canonical form which, in turn, we interpret in geometric control terms. If the behavior is given in a kernel representation by a higher order rectangular polynomial matrix, the natural equivalence concept is behavior equivalence. These notions are closely related to the Morse group that incorporates state space similarity transformations, state feedback, and output injection. A simple canonical form for behavioral equivalence is given that clearly exhibits the reachable and autonomous parts of the behavior. Using polynomial models we also present a unified approach to pencil equivalence that elucidates the close connections between classification problems from linear algebra, geometric control theory, and behavior theory. We also indicate how to derive the invariants under behavior equivalence from the Kronecker invariants.     

Equivalence of rational representations of behaviors

This article deals with the equivalence of representations of behaviors of linear differential systems. In general, the behavior of a given linear differential system has many different representations. In this paper we restrict ourselves to kernel and image representations. Two kernel representations are called equivalent if they represent one and the same behavior. For kernel representations defined by polynomial matrices, necessary and sufficient conditions for equivalence are well known. In this paper, we deal with the equivalence of rational representations, i. e. kernel and image representations that are defined in terms of rational matrices. As the first main result of this paper, we will derive a new condition for the equivalence of rational kernel representations of possibly noncontrollable behaviors. Secondly we will derive conditions for the equivalence of rational representations of a given behavior in terms of the polynomial modules generated by the rows of the rational matrices. We will also establish conditions for the equivalence of rational image representations. Finally, we will derive conditions under which a given rational kernel representation is equivalent to a given rational image representation.     

Canonical piecewise-linear networks

In this paper, mapping networks will be considered from the viewpoint of the piecewise-linear (PWL) approximation. The so-called canonical representation plays a kernel role in the PWL representation theory. While this theory has been researched intensively in the contents of mathematics and circuit simulations, little has been seen in the research area about the theoretical aspect of neural networks. This paper modifies this theory and applies it as a mathematical support for mapping networks. The main modification is a "higher-level" generalization of the canonical representation with proofs of its availability in the set of PWL functions. The modified theory will first be used to study the canonical PWL feature of the popular multilayer perceptron-like (MLPL) networks. Second, it will be seen that the generalized canonical representation is itself suitable for a network implementation, which is called the standard canonical PWL network. More generally, the family of (generalized) canonical PWL networks is defined as those which may take the canonical PWL representation as a mathematical model. This family is large and practically meaningful. The standard canonical PWL networks may be taken as representatives in the family. The modification of the PWL representation theory as well as the introduction of this theory in the theoretical study of mapping networks, which provide a new concept of mapping networks, i.e., the canonical PWL network family, may be regarded as the main contributions of the paper.     

The input-output and controllable behavioural structures of a state space representation

Given the matrices of a linear state space representation, we find an expression for a universal left annihilator of the matrix zI - A -C and hence derive kernel representations for the input-output behaviour and by duality the controllable part. More generally in the discrete case we derive representations for the L -completion for different values of L, and the subbehaviours of trajectories reachable in a given time interval. The representations are in certain trim canonical forms, which are intimately connected with structure indices. As a by-product of the state elimination procedure, we obtain a minimal state space representation for a given behaviour in terms of an arbitrary one.     

Data representations and generalization error in kernel based learning machines

This paper focuses on the problem of how data representation influences the generalization error of kernel based learning machines like support vector machines (SVM) for classification. Frame theory provides a well founded mathematical framework for representing data in many different ways. We analyze the effects of sparse and dense data representations on the generalization error of such learning machines measured by using leave-one-out error given a finite amount of training data. We show that, in the case of sparse data representations, the generalization error of an SVM trained by using polynomial or Gaussian kernel functions is equal to the one of a linear SVM. This is equivalent to saying that the capacity of separating points of functions belonging to hypothesis spaces induced by polynomial or Gaussian kernel functions reduces to the capacity of a separating hyperplane in the input space. Moreover, we show that, in general, sparse data representations increase or leave unchanged the generalization error of kernel based methods. Dense data representations, on the contrary, reduce the generalization error in the case of very large frames. We use two different schemes for representing data in overcomplete systems of Haar and Gabor functions, and measure SVM generalization error on benchmarked data sets.     

Ordered seminormal default theories and their extensions

Abstract

The concept of extension plays an important role in default logic. The notion of an ordered seminormal default theory has been introduced (Etherington 1987) to characterize a class of seminormal default theories which have extensions. However, the original definition has a drawback because of its dependence on specific representations of the default theory. We introduce the ‘canonical representation’ of a default theory and redefine the orderedness of a default theory based on its canonical representation. We show that under the new definition, the orderedness of a default theory Δ = (W,D) is intrinsic to the theory itself, independent of the specific representations of W and D. We present a modification of the algorithm in Etherington (1987) for computing extensions of a default theory. More importantly, we prove the conjecture (Etherington 1987) that a modified version of the algorithm in Etherington (1987) converges for general ordered, finite seminormal default theories, while the original algorithm was proven (Etherington 1987) to converge for ordered, finite network default theories which form a proper subset of the theories considered in this paper.     

Meaning of state-space representation and its relation to canonical form Part 2. Controllability and constructibility

State space is one of the key concepts of systems theory. In the previous paper (Nakano et al. 1987) a system-theoretic framework about the choice of a state space was developed and the canonical forms of linear control theory were interpreted. Four realizations of a single-input and single-output discrete basic linear system were constructed. The isomorphism between the realizations and the universal state space representation shows systemic meanings of the state space. In this paper, using the same approach, two other realizations—a controllability realization and a constructibility realization—are constructed. By the matrix representations of these realizations the meanings of the controllability and constructibility canonical forms are shown in relation to the universal state-space representation.     

Edit distance-based kernel functions for structural pattern classification

A common approach in structural pattern classification is to define a dissimilarity measure on patterns and apply a distance-based nearest-neighbor classifier. In this paper, we introduce an alternative method for classification using kernel functions based on edit distance. The proposed approach is applicable to both string and graph representations of patterns. By means of the kernel functions introduced in this paper, string and graph classification can be performed in an implicit vector space using powerful statistical algorithms. The validity of the kernel method cannot be established for edit distance in general. However, by evaluating theoretical criteria we show that the kernel functions are nevertheless suitable for classification, and experiments on various string and graph datasets clearly demonstrate that nearest-neighbor classifiers can be outperformed by support vector machines using the proposed kernel functions.     

基于递推规范变量残差和核主元分析的微小故障检测

微小故障由于故障征兆不明显从而很难在故障发生早期对其进行检测. 针对该问题, 本文提出了一种基于递推规范变量残差和核主元分析(RCVD–KPCA)的微小故障检测方法. 首先构造规范变量残差, 从中提取数据的线性特征. 利用指数加权滑动平均法对规范变量残差进行递推滤波处理, 提高规范变量残差对微小故障的敏感程度;然后使用KPCA提取规范变量残差中的非线性主成分作为非线性特征, 根据提取的特征提出了两个新的故障检测统计量; 此外, 利用核密度估计确定故障检测统计量的控制限. 由于同时提取了过程数据的线性和非线性特征, 有效地提高了非线性动态过程中微小故障的可检测性. 以闭环连续搅拌釜式反应器过程为例进行了仿真分析, 仿真结果表明本文所提方法具有较好的故障检测性能.     

Symmetries of differential behaviors and finite group actions on free modules over a polynomial ring

In this paper we study finite group symmetries of differential behaviors (i.e., kernels of linear constant coefficient partial differential operators). They lead us to study the actions of a finite group on free modules over a polynomial ring. We establish algebraic results which are then used to obtain canonical differential representations of symmetric differential behaviors.     

A theory of learning with similarity functions

Kernel functions have become an extremely popular tool in machine learning, with an attractive theory as well. This theory views a kernel as implicitly mapping data points into a possibly very high dimensional space, and describes a kernel function as being good for a given learning problem if data is separable by a large margin in that implicit space. However, while quite elegant, this theory does not necessarily correspond to the intuition of a good kernel as a good measure of similarity, and the underlying margin in the implicit space usually is not apparent in “natural” representations of the data. Therefore, it may be difficult for a domain expert to use the theory to help design an appropriate kernel for the learning task at hand. Moreover, the requirement of positive semi-definiteness may rule out the most natural pairwise similarity functions for the given problem domain. In this work we develop an alternative, more general theory of learning with similarity functions (i.e., sufficient conditions for a similarity function to allow one to learn well) that does not require reference to implicit spaces, and does not require the function to be positive semi-definite (or even symmetric). Instead, our theory talks in terms of more direct properties of how the function behaves as a similarity measure. Our results also generalize the standard theory in the sense that any good kernel function under the usual definition can be shown to also be a good similarity function under our definition (though with some loss in the parameters). In this way, we provide the first steps towards a theory of kernels and more general similarity functions that describes the effectiveness of a given function in terms of natural similarity-based properties.     

Multiple kernel representation and classification of multivariate satellite-image time-series for crop mapping

Multivariate satellite-image time-series (MSITS) are a valuable source of information for a wide range of agricultural applications. Image classification, one of the main applications of this type of data, is a challenging task. It is mainly because MSITS are generated by a complex interaction among several sources of information, which are known as the factors of variation. These factors contain different information with different levels of relevance to a classification task. Thus, a proper representation of MSITS data is required in order to extract and model the most useful information from these factors for classification purpose. To this end, this article proposes three multiple kernel representations of MSITS data. These representations extract the most classification-related information from these data through combining the basis kernels constructed from different factors of variation of the MSITS data. In the proposed representations, the combination of the basis kernels was achieved by using the multiple kernel learning algorithms. The efficiency of the proposed multiple kernel representations was evaluated based both on analysing the relevance of their kernels to the classification task and their classification performances. Two different MSITS data sets composed of 10 RapidEye imageries of an agricultural area were used to evaluate the performances of the proposed methods. In addition, the classification results of both MSITS using a single kernel were considered as the baseline for comparison. The results showed an increase of up to 14% in overall accuracy of the classification maps by using the multiple kernel representations. Moreover, these particular representations for classification of time-series observations were able to handle the undesirable effects in image data such as the presence of clouds and their shadows.     

核典型相关性鉴别分析

提出一种新的基于典型相关性的核鉴别分析,以图片集为基础的人脸识别算法。把每个图片集映射到一个高维特征空间,然后通过核线性鉴别分析(KLDA)处理,得到相应的核子空间。通过计算两典型向量的典型差来估计两个子空间的相似度。根据核Fisher准则,基于类间典型差与类内典型差的比率建立核子空间的相关性来得到核典型相关性鉴别分析(KDCC)算法。在ORL、NUST603、FERNT和XM2VTS人脸库上的实验结果表明,该算法能够更有效提取样本特征,在识别率上要优于典型相关性鉴别分析(DCC)和核鉴别转换(KDT)算法。     

Scale-space derived from B-splines

This paper proposes a scale-space theory based on B-spline kernels. Our aim is twofold: 1) present a general framework, and show how B-splines provide a flexible tool to design various scale-space representations. In particular, we focus on the design of continuous scale-space and dyadic scale-space frame representations. A general algorithm is presented for fast implementation of continuous scale-space at rational scales. In the dyadic case, efficient frame algorithms are derived using B-spline techniques to analyze the geometry of an image. The relationship between several scale-space approaches is explored. The behavior of edge models, the properties of completeness, causality, and other properties in such a scale-space representation are examined in the framework of B-splines. It is shown that, besides the good properties inherited from the Gaussian kernel, the B-spline derived scale-space exhibits many advantages for modeling visual mechanism including the efficiency, compactness, orientation feature and parallel structure     

Instructions for Authors

In biological data, it is often the case that objects are described in two or more representations. In order to perform classification based on such data, we have to combine them in a certain way. In the context of kernel machines, this task amounts to mix several kernel matrices into one. In this paper, we present two ways to mix kernel matrices, where the mixing weights are optimized to minimize the cross validation error. In bacteria classification and gene function prediction experiments, our methods significantly outperformed single kernel classifiers in most cases.     

Redundancies in linear GP,canonical transformation,and its exploitation: a demonstration on image feature synthesis

This paper concerns redundancies in representation of linear genetic programming (GP). We identify the causes of redundancies in linear GP and propose a canonical transformation that converts original linear representations into a canonical form in which structural redundancies are removed. In canonical form, we can easily verify whether two representations represent an identical program. We then discuss exploitation of the proposed canonical transformation, and demonstrate a way to improve search performance of linear GP by avoiding redundant individuals. Experiments were conducted with an image feature synthesis problem. Firstly, we have verified that there are really a lot of redundancies in conventional linear GP. We then investigate the effect of avoiding redundant individuals. The results yield that linear GP with avoidance of redundant individuals obviously outperforms conventional linear GP.     

基于自适应混合核典型变量分析的工业过程质量相关故障检测

质量相关故障检测技术是保障工业过程安全顺行和质量稳定的重要手段,是当前流程工业过程控制领域的研究热点.针对工业过程的非线性与动态特性及其质量相关故障的时变特性,提出一种基于自适应混合核典型变量分析(AMKCVA)的质量相关故障检测方法.该方法通过设计合理的混合核函数和自适应监测统计量,提升了工业过程质量相关故障的检测性...     

Unsupervised feature extraction via kernel subspace techniques

This paper provides a new insight into unsupervised feature extraction techniques based on kernel subspace models. The data projected onto kernel subspace models are new data representations which might be better suited for classification. The kernel subspace models are always described exploiting the dual form for the basis vectors which requires that the training data must be available even during the test phase. By exploiting an incomplete Cholesky decomposition of the kernel matrix, a computationally less demanding implementation is proposed. Online benchmark data sets allow the evaluation of these feature extraction methods comparing the performance of two classifiers which both have as input either the raw data or the new representations.     

基于E2LSH-MKL的视觉语义概念检测

多核学习方法(Multiple kernel learning, MKL)在视觉语义概念检测中有广泛应用, 但传统多核学习大都采用线性平稳的核组合方式而无法准确刻画复杂的数据分布. 本文将精确欧氏空间位置敏感哈希(Exact Euclidean locality sensitive Hashing, E2LSH)算法用于聚类, 结合非线性多核组合方法的优势, 提出一种非线性非平稳的多核组合方法—E2LSH-MKL. 该方法利用Hadamard内积实现对不同核函数的非线性加权,充分利用了不同核函数之间交互得到的信息; 同时利用基于E2LSH哈希原理的聚类算法,先将原始图像数据集哈希聚类为若干图像子集, 再根据不同核函数对各图像子集的相对贡献大小赋予各自不同的核权重, 从而实现多核的非平稳加权以提高学习器性能; 最后,把E2LSH-MKL应用于视觉语义概念检测. 在Caltech-256和TRECVID 2005数据集上的实验结果表明,新方法性能优于现有的几种多核学习方法.