基于DLSR的归纳式迁移学习

传统机器学习方法的有效性依赖于大量的有效训练数据,而这难以满足,因此迁移学习被广泛研究并成为近年来的研究热门.针对由于训练数据严重不足导致多分类场景下分类性能降低的挑战,提出一种基于DLSR(discriminative least squares regressions)的归纳式迁移学习方法(TDLSR).该方法从归纳式迁移学习出发,通过知识杠杆机制,将源域知识迁移到目标域并同目标域数据同时进行模型学习,在提升分类性能的同时保证源域数据的安全性.TDLSR继承了DLSR在多分类任务中扩大类别间间隔的优势,为DLSR注入了迁移能力以适应数据不足的挑战,更加适用于复杂的多分类任务.通过在12个真实UCI数据集上进行实验,验证了所提出方法的有效性.     

Word sense learning based on feature selection and MDL principle

In this paper, we propose a word sense learning algorithm which is capable of unsupervised feature selection and cluster number identification. Feature selection for word sense learning is built on an entropy-based filter and formalized as a constraint optimization problem, the output of which is a set of important features. Cluster number identification is built on a Gaussian mixture model with a MDL-based criterion, and the optimal model order is inferred by minimizing the criterion. To evaluate closeness between the learned sense clusters with the ground-truth classes, we introduce a kind of weighted F-measure to model the effort needed to reconstruct the classes from the clusters. Experiments show that the algorithm can retrieve important features, roughly estimate the class numbers automatically and outperforms other algorithms in terms of the weighted F-measure. In addition, we also try to apply the algorithm to a specific task of adding new words into a Chinese thesaurus.     

Interactive inductive learning

We propose an interactive probabilistic inductive learning model which defines a feedback relationship between the user and the learning program. We extend previously described learning algorithms to a conditional model previously described by the authors, and formulate our Conditional Probabilistic Learning Algorithm (CPLA), applying conditions as introduced by Wasilewska to a probabilistic version of the work of Wong and Wong. We propose the Condition Suggestion Algorithm (CSA) as a way to use the syntactic knowledge in the system to generalize the family of decision rules. We also examine the semantic knowledge of the system implied by the suggested conditions and analyse the effects of conditions on the system. CPLA/CSA has been implemented by the first author and was used to generate the examples presented.     

An inductive learning method for medical diagnosis

In most learning models, the induction methods that are learnable have low expressive power. The learnability of such methods are proved for some concept classes by assuming that the hypothesis space of the method contains the target concept. However, in real-world practical problems the type of target concept being dealt with is almost always unknown. In medical diagnosis, where mistakes can cause fatal results, it is very important to achieve high recognition rates and the use of more expressive methods are common. However, conventional methods are weak at handling irrelevant information which often appears in medical databases. In this paper, we consider test feature classifiers recently introduced by (Lashkia and Aleshin, 2001. IEEE Trans. Syst. Man Cybern. 31 (4), 643–650), and show that they meet all essential requirements to be of practical use in medical decision making, which are: ability to handle irrelevant attributes, high expressive power, high recognition ability, and ability to generate decisions by a set of rules.     

Extended invariance principle for nonautonomous switched systems

Stability analysis is developed for nonlinear nonautonomous switched systems, trajectories of which admit, generally speaking, a nonunique continuation on the right. For these systems Krasovskii-LaSalle's invariance principle is extended in such a manner to remain true even in the nonautonomous case. In addition to a nonsmooth Lyapunov function with negative-semidefinite time derivatives along the system trajectories, the extended invariance principle involves a coupled indefinite function to guarantee asymptotic stability of the system in question. As an illustration of the capabilities of this principle, a switched regulator of a fully-actuated manipulator with frictional joints is constructed.     

Cohesion methods in inductive learning

According to Webster, cohesion is "the act or process of sticking together tightly." Here the term represents the underlying forces that drive the formation of classes during inductive learning. This paper considers several numerical and conceptual induction algorithms, and compares their methods of cohesion. While these algorithms represent several different methods, they also exhibit some significant commonalities.     

Iterative feature construction for improving inductive learning algorithms

Inductive learning algorithms, in general, perform well on data that have been pre-processed to reduce complexity. By themselves they are not particularly effective in reducing data complexity while learning difficult concepts. Feature construction has been shown to reduce complexity of space spanned by input data. In this paper, we present an iterative algorithm for enhancing the performance of any inductive learning process through the use of feature construction as a pre-processing step. We apply the procedure on three learning methods, namely genetic algorithms, C4.5 and lazy learner, and show improvement in performance.     

Extended clause learning

The past decade has seen clause learning as the most successful algorithm for SAT instances arising from real-world applications. This practical success is accompanied by theoretical results showing clause learning as equivalent in power to resolution. There exist, however, problems that are intractable for resolution, for which clause-learning solvers are hence doomed. In this paper, we present extended clause learning, a practical SAT algorithm that surpasses resolution in power. Indeed, we prove that it is equivalent in power to extended resolution, a proof system strictly more powerful than resolution. Empirical results based on an initial implementation suggest that the additional theoretical power can indeed translate into substantial practical gains.     

Class-dependent discretization for inductive learning fromcontinuous and mixed-mode data

Inductive learning systems can be effectively used to acquire classification knowledge from examples. Many existing symbolic learning algorithms can be applied in domains with continuous attributes when integrated with a discretization algorithm to transform the continuous attributes into ordered discrete ones. In this paper, a new information theoretic discretization method optimized for supervised learning is proposed and described. This approach seeks to maximize the mutual dependence as measured by the interdependence redundancy between the discrete intervals and the class labels, and can automatically determine the most preferred number of intervals for an inductive learning application. The method has been tested in a number of inductive learning examples to show that the class-dependent discretizer can significantly improve the classification performance of many existing learning algorithms in domains containing numeric attributes     

A method for attribute selection in inductive learning systems

A computable measure was developed that can be used to discriminate between attributes on the basis of their potential value in the formation of decision rules by the inductive learning process. This relevance measure is the product of extensions to an information-theoretic foundation that address the particular characteristics of a class of inductive learning algorithms. The measure is also conceptually compatible with approaches from pattern recognition. It is described in the context of a generalized model of the expertise development process, and an experiment is presented in which a significant reduction in the number of attributes to be considered was achieved for a complex medical domain     

Integrating statistical and inductive learning methods for knowledge acquisition

Inductive learning is a method for automated knowledge acquisition. It converts a set of training data into a knowledge structure. In the process of knowledge induction, statistical techniques can play a major role in improving performance. In this paper, we investigate the competition and integration between the traditional statistical and the inductive learning methods. First, the competition between these two approaches is examined. Then, a general framework for integrating these two approaches is presented. This framework suggests three possible integrations: (1) statistical methods as preprocessors for inductive learning, (2) inductive learning methods as preprocessors for statistical classification, and (3) the combination of the two methods to develop new algorithms. Finally, empirical evidence concerning these three possible integrations are discussed. The general conclusion is that algorithms integrating statistical and inductive learning concepts are likely to make the most improvement in performance.     

MDL理论的多属性值域划分方法

连续属性值域划分方法是数据挖掘和机器学习领域的重要课题。但已有的大量离散化方法倾向于研究一维属性离散化问题,没有考虑多属性之间的相互关系,难于获得最佳的离散化结果。提出一种基于最小描述长度理论的多属性划分方法,通过定义多属性的模型选择问题,推导出多属性划分衡量函数;设计一种合理的算法来寻找最好的离散化结果。性能评价与分析表明,该方法在Naive贝叶斯分类器上有很好的分类学习能力。     

A principle for learning egocentric-allocentric transformation

Numerous single-unit recording studies have found mammalian hippocampal neurons that fire selectively for the animal's location in space, independent of its orientation. The population of such neurons, commonly known as place cells, is thought to maintain an allocentric, or orientation-independent, internal representation of the animal's location in space, as well as mediating long-term storage of spatial memories. The fact that spatial information from the environment must reach the brain via sensory receptors in an inherently egocentric, or viewpoint-dependent, fashion leads to the question of how the brain learns to transform egocentric sensory representations into allocentric ones for long-term memory storage. Additionally, if these long-term memory representations of space are to be useful in guiding motor behavior, then the reverse transformation, from allocentric to egocentric coordinates, must also be learned. We propose that orientation-invariant representations can be learned by neural circuits that follow two learning principles: minimization of reconstruction error and maximization of representational temporal inertia. Two different neural network models are presented that adhere to these learning principles, the first by direct optimization through gradient descent and the second using a more biologically realistic circuit based on the restricted Boltzmann machine (Hinton, 2002; Smolensky, 1986). Both models lead to orientation-invariant representations, with the latter demonstrating place-cell-like responses when trained on a linear track environment.     

An efficient inductive learning method for object-oriented databaseusing attribute entropy

The data-driven characteristic of the Version Space rule-learning method works efficiently in memory even if the training set is enormous. However, the concept hierarchy of each attribute used to generalize/specialize the hypothesis of a specific/general (S/G) set is processed sequentially and instance by instance, which degrades its performance. As for ID3, the decision tree is generated from the order of attributes according to their entropies to reduce the number of attributes in some of the tree paths. Unlike Version Space, ID3 generates an extremely complex decision tree when the training set is enormous. Therefore, we propose a method called AGE (A_RCH+OG_L+ASE_, where ARCH=“Automatic geneRation of Concept Hierarchies”, OGL=“Optimal Generalization Level”, and ASE=“Attribute Selection by Entropy”), taking advantages of Version Space and ID3 to learn rules from object-oriented databases (OODBs) with the least number of learning features according to the entropy. By simulations, we found the performance of our learning algorithm is better than both Version Space and ID3. Furthermore, AGE's time complexity and space complexity are both linear with the number of training instances     

Intelligent condition monitoring using fuzzy inductive learning

Extensive research has been performed for developing knowledge based intelligent monitoring systems for improving the reliability of manufacturing processes. Due to the high expense of obtaining knowledge from human experts, it is expected to develop new techniques to obtain the knowledge automatically from the collected data using data mining techniques. Inductive learning has become one of the widely used data mining methods for generating decision rules from data. In order to deal with the noise or uncertainties existing in the data collected in industrial processes and systems, this paper presents a new method using fuzzy logic techniques to improve the performance of the classical inductive learning approach. The proposed approach, in contrast to classical inductive learning method using hard cut point to discretize the continuous-valued attributes, uses soft discretization to enable the systems have less sensitivity to the uncertainties and noise. The effectiveness of the proposed approach has been illustrated in an application of monitoring the machining conditions in uncertain environment. Experimental results show that this new fuzzy inductive learning method gives improved accuracy compared with using classical inductive learning techniques.     

Extended distributed learning automata

In this paper, a new structure for cooperative learning automata called extended learning automata (eDLA) is introduced. Based on the new structure, an iterative randomized heuristic algorithm using sampling is proposed for finding an optimal subgraph in a stochastic edge-weighted graph. Stochastic graphs are graphs in which the weights of edges have an unknown probability distribution. The proposed algorithm uses an eDLA to find a policy that leads to a subgraph that satisfy some restrictions such as minimum or maximum weight (length). At each stage of the proposed algorithm, the eDLA determines which edges should be sampled. The proposed eDLA-based sampling method may reduce unnecessary samples and hence decrease the time required for finding an optimal subgraph. It is shown that the proposed method converges to an optimal solution, the probability of which can be made arbitrarily close to 1 by using a sufficiently small learning parameter. A new variance-aware threshold value is also proposed that can significantly improve the convergence rate of the proposed eDLA-based algorithm. It is further shown that our algorithm is competitive in terms of the quality of the solution.     

A new algorithm for automatic knowledge acquisition in inductive learning

The objective of this study is to present a new algorithm, REX-1, developed for automatic knowledge acquisition in Inductive Learning. It aims at eliminating the pitfalls and disadvantages of the techniques and algorithms currently in use. The proposed algorithm makes use of the direct rule extraction approach, rather than the decision tree. For this purpose, it uses a set of examples to induce general rules. Using some widely used set of examples such as IRIS, Balance and Balloons, Monk, Splice, Promoter, Lenses, Zoo, and Vote, our algorithm is compared with other well-known algorithms such as ID3, C4.5, ILA, and Rules Family.