Alien attractors and memory annihilation of structured sets inHopfield networks

This paper considers the encoding of structured sets into Hopfield associative memories. A structured set is a set of vectors with equal Hamming distance h from one another, and its centroid is an external vector that has distance h/2 from every vector of the set. Structured sets having centroids are not infrequent. When such a set is encoded into a noiseless Hopfield associative memory using a bipolar outer-product connection matrix, and the network operates with synchronous neuronal update, the memory of all encoded vectors is annihilated even for sets with as few as three vectors in dimension n>5 (four for n=5). In such self-annihilating structured sets, the centroid emerges as a stable attractor. We call it an alien attractor. For canonical structured sets, self-annihilation takes place only if h相似文献   

On bounds of response time performance achievable by multiclass single-server queues

MulticlassM/G/1 systems in steady-state with work-conserving scheduling strategies are studied. Restricting a system's scheduling strategy to making no direct use of the required service times, every time the server becomes idle its memory is cleared, and service may only be interrupted by newly arriving customers, a conservation law is developed by means of inequalities. The conservation law states that if a response time vector composed of the expected response times of the different classes of a system in steady-state is achievable, then it must belong to a well-defined convex polytope (a set bounded by hyperplanes). Furthermore, on each hyperplane bounding the relevant polytope there lies at least one vertex of the convex set of achievable response time vectors. Therefore, this polytope is the least one including the set of all achievable response time vectors.     

基于聚类的快速支持向量机训练算法*

支持向量机(support vector machine, SVM)具有良好的泛化性能而被广泛应用于机器学习及模式识别领域。然而,当训练集较大时,训练SVM需要极大的时间及空间开销。另一方面,SVM训练所得的判定函数取决于支持向量,使用支持向量集取代训练样本集进行学习,可以在不影响结果分类器分类精度的同时缩短训练时间。采用混合方法来削减训练数据集,实现潜在支持向量的选择,从而降低SVM训练所需的时间及空间复杂度。实验结果表明,该算法在极大提高SVM训练速度的同时,基本维持了原始分类器的泛化性能。     

A Boolean Hebb rule for binary associative memory design

A binary associative memory design procedure that gives a Hopfield network with a symmetric binary weight matrix is introduced in this paper. The proposed method is based on introducing the memory vectors as maximal independent sets to an undirected graph, which is constructed by Boolean operations analogous to the conventional Hebb rule. The parameters of the resulting network is then determined via the adjacency matrix of this graph in order to rind a maximal independent set whose characteristic vector is close to the given distorted vector. We show that the method provides attractiveness for each memory vector and avoids spurious memories whenever the set of given memory vectors satisfy certain compatibility conditions, which implicitly imply sparsity. The applicability of the design method is finally investigated by a quantitative analysis of the compatibility conditions.     

Odd Memory Systems: A New Approach

To reject the use of a prime (or odd) number N of memory banks in a vector processor, it is generally advanced that address computation for such a memory system would require systematic Euclidean division by the number N. We first show that the Chinese Remainder Theorem allows one to define a very simple mapping of data onto the memory banks for which address computation does not require any Euclidean division. Massively parallel SIMD computers may have thousands of processors. When the memory on such a machine is globally shared, routing vectors from memory to the processors is a major difficulty; the control for the interconnection network cannot be generally computed at execution time. When the number of memory banks and processors is a product of prime numbers, the family of permutations needed for routing vectors from memory to the processors through the interconnection network has very specific properties. The Chinese Remainder Network presented in the paper is able to execute all these permutations in a single path and may be easily controlled.     

量化编码的分层可通航小世界图算法

随着大数据和人工智能的高速发展,针对多媒体数据的结构化处理与基于内容的检索受到极大的关注,面对多媒体数据结构化后的海量高维特征向量,如何快速、准确地检索是人工智能处理大规模数据所必须解决的问题。最近提出的分层可通航小世界图HNSW检索算法在多个公开数据集取得了最佳的性能表现,但该算法存在内存开销大的问题。而基于量化编码的检索算法能够压缩数据集向量,大幅度降低内存占用。将量化编码和分层可通航小世界图算法结合,提出了2种基于量化编码改进的HNSW算法,分别是使用标量量化编码向量的HNSWSQ算法和使用乘积量化编码向量的HNSWPQ算法,2种算法使用不同的量化策略存储原始向量编码,以降低内存开销,再通过HNSW算法建立索引达到缩短检索耗时的目的。其中HNSWSQ算法在多个数据集上获得了与HNSW算法相近的查全率和平均检索耗时,而内存开销大幅降低。实验结果表明,HNSWSQ算法在SIFT-1M和GIST-1M数据集上的内存开销比HNSW算法分别降低了45.1%和70.4%。     

Some experiments on human memory and a new neural model

An associative memory with parallel architecture is presented. The neurons are modelled by perceptrons having only binary, rather than continuous valued input. To store m elements each having n features, m neurons each with n connections are needed. The n features are coded as an n-bit binary vector. The weights of the n connections that store the n features of an element has only two values -1 and 1 corresponding to the absence or presence of a feature. This makes the learning very simple and straightforward. For an input corrupted by binary noise, the associative memory indicates the element that is closest (in terms of Hamming distance) to the noisy input. In the case where the noisy input is equidistant from two or more stored vectors, the associative memory indicates two or more elements simultaneously. From some simple experiments performed on the human memory and also on the associative memory, it can be concluded that the associative memory presented in this paper is in some respect more akin to a human memory than a Hopfield model.     

On computing the supremal right-closed control invariant subset of a right-closed set of markings for an arbitrary petri net

A set of non-negative integral vectors is said to be right-closed if the presence of a vector in the set implies all term-wise larger vectors also belong to the set. A set of markings is control invariant with respect to a Petri Net (PN) structure if the firing of any uncontrollable transition at any marking in this set results in a new marking that is also in the set. Every right-closed set of markings has a unique supremal control invariant subset, which is the largest subset that is control invariant with respect to the PN structure. This subset is not necessarily right-closed. In this paper, we present an algorithm that computes the supremal right-closed control invariant subset of a right-closed of markings with respect to an arbitrary PN structure. This set plays a critical role in the synthesis of Liveness Enforcing Supervisory Policies (LESPs) for a class of PN structures, and consequently, the proposed algorithm plays a key role in the synthesis of LESPs for this class of PN structures.

     

Memory annihilation of structured maps in bidirectional associativememories

Structured sets comprise Boolean vectors with equal pair-wise Hamming distances, h. An external vector, if it exists at an equidistance of h/2 from each vector of the structured set, is called the centroid of the set. A structured map is a one-one mapping between structured sets. It is a set of associations between Boolean vectors, where both domain and range vectors are drawn from structured sets. Associations between centroids are called centroidal associations. We show that when structured maps are encoded into bidirectional associative memories using outer-product correlation encoding, the memory of these associations are annihilated under certain mild conditions. When annihilation occurs, the centroidal association emerges as a stable association, and we call it an alien attractor. For the special case of maps where h=2, self-annihilation can take place when either the domain or range dimensions are greater than five. In fact, we show that for dimensions greater than eight, as few as three associations suffice for self-annihilation. As an example shows, annihilation occurs even for the case of bipolar decoding which is well known for its improved error correction capability in such associative memory models.     

Hierarchical Learning of Navigational Behaviors in an Autonomous Robot using a Predictive Sparse Distributed Memory

We describe a general framework for learning perception-based navigational behaviors in autonomous mobile robots. A hierarchical behavior-based decomposition of the control architecture is used to facilitate efficient modular learning. Lower level reactive behaviors such as collision detection and obstacle avoidance are learned using a stochastic hill-climbing method while higher level goal-directed navigation is achieved using a self-organizing sparse distributed memory. The memory is initially trained by teleoperating the robot on a small number of paths within a given domain of interest. During training, the vectors in the sensory space as well as the motor space are continually adapted using a form of competitive learning to yield basis vectors that efficiently span the sensorimotor space. After training, the robot navigates from arbitrary locations to a desired goal location using motor output vectors computed by a saliency-based weighted averaging scheme. The pervasive problem of perceptual aliasing in finite-order Markovian environments is handled by allowing both current as well as the set of immediately preceding perceptual inputs to predict the motor output vector for the current time instant. We describe experimental and simulation results obtained using a mobile robot equipped with bump sensors, photosensors and infrared receivers, navigating within an enclosed obstacle-ridden arena. The results indicate that the method performs successfully in a number of navigational tasks exhibiting varying degrees of perceptual aliasing.     

Hierarchical Learning of Navigational Behaviors in an Autonomous Robot using a Predictive Sparse Distributed Memory

We describe a general framework for learning perception-based navigational behaviors in autonomous mobile robots. A hierarchical behavior-based decomposition of the control architecture is used to facilitate efficient modular learning. Lower level reactive behaviors such as collision detection and obstacle avoidance are learned using a stochastic hill-climbing method while higher level goal-directed navigation is achieved using a self-organizing sparse distributed memory. The memory is initially trained by teleoperating the robot on a small number of paths within a given domain of interest. During training, the vectors in the sensory space as well as the motor space are continually adapted using a form of competitive learning to yield basis vectors that efficiently span the sensorimotor space. After training, the robot navigates from arbitrary locations to a desired goal location using motor output vectors computed by a saliency-based weighted averaging scheme. The pervasive problem of perceptual aliasing in finite-order Markovian environments is handled by allowing both current as well as the set of immediately preceding perceptual inputs to predict the motor output vector for the current time instant. We describe experimental and simulation results obtained using a mobile robot equipped with bump sensors, photosensors and infrared receivers, navigating within an enclosed obstacle-ridden arena. The results indicate that the method performs successfully in a number of navigational tasks exhibiting varying degrees of perceptual aliasing.     

Node splitting: A constructive algorithm for feed-forward neural networks

A constructive algorithm is proposed for feed-forward neural networks which uses node-splitting in the hidden layers to build large networks from smaller ones. The small network forms an approximate model of a set of training data, and the split creates a larger, more powerful network which is initialised with the approximate solution already found. The insufficiency of the smaller network in modelling the system which generated the data leads to oscillation in those hidden nodes whose weight vectors cover regions in the input space where more detail is required in the model. These nodes are identified and split in two using principal component analysis, allowing the new nodes to cover the two main modes of the oscillating vector. Nodes are selected for splitting using principal component analysis on the oscillating weight vectors, or by examining the Hessian matrix of second derivatives of the network error with respect to the weights.     

Efficient cross-validation for kernelized least-squares regression with sparse basis expansions

We propose an efficient algorithm for calculating hold-out and cross-validation (CV) type of estimates for sparse regularized least-squares predictors. Holding out H data points with our method requires O(min(H ² n,Hn ²)) time provided that a predictor with n basis vectors is already trained. In addition to holding out training examples, also some of the basis vectors used to train the sparse regularized least-squares predictor with the whole training set can be removed from the basis vector set used in the hold-out computation. In our experiments, we demonstrate the speed improvements provided by our algorithm in practice, and we empirically show the benefits of removing some of the basis vectors during the CV rounds.     

Support Vector的几何解释及其在联想记忆中的应用

利用闭凸集上的投影解释support vector的几何意义，利用支持超平面讨论线性分类器的设计问题，对线性可分情形，Support vector由一类数据集合闭凸包在另一类数据集合闭凸包上投影的非零系数向量组成，SVM所决定的超平面位于两投影点关于各自数据集合支持超平面的中间，作为应用，文中给出一种设计理想联想记忆前馈神经网络的方法，它是FP算法的一般化。     

Finding Essential Attributes from Binary Data

We consider data sets that consist of n-dimensional binary vectors representing positive and negative examples for some (possibly unknown) phenomenon. A subset S of the attributes (or variables) of such a data set is called a support set if the positive and negative examples can be distinguished by using only the attributes in S. In this paper we study the problem of finding small support sets, a frequently arising task in various fields, including knowledge discovery, data mining, learning theory, logical analysis of data, etc. We study the distribution of support sets in randomly generated data, and discuss why finding small support sets is important. We propose several measures of separation (real valued set functions over the subsets of attributes), formulate optimization models for finding the smallest subsets maximizing these measures, and devise efficient heuristic algorithms to solve these (typically NP-hard) optimization problems. We prove that several of the proposed heuristics have a guaranteed constant approximation ratio, and we report on computational experience comparing these heuristics with some others from the literature both on randomly generated and on real world data sets.     

Pseudopolynomial algorithms for certain computationally hard vector subset and cluster analysis problems

We analyze several NP-hard problems related to clustering and searching, in a given set of vectors in a Euclidean space, for a subset of vectors of fixed size. An important data mining problem related to sum of squares optimization reduces to these problems. We show pseudopolynomial algorithms that are guaranteed to find an optimum in these problems in case when vector components have integer values and the dimension is fixed.     

The Object Perceptron Learning Algorithm on Generalised Hopfield Networks for Associative Memory

We present a study of generalised Hopfield networks for associative memory. By analysing the radius of attraction of a stable state, the Object Perceptron Learning Algorithm (OPLA) and OPLA scheme are proposed to store a set of sample patterns (vectors) in a generalised Hopfield network with their radii of attraction as large as we require. OPLA modifies a set of weights and a threshold in a way similar to the perceptron learning algorithm. The simulation results show that the OPLA scheme is more effective for associative memory than both the sum-of-outer produce scheme with a Hopfield network and the weighted sum-of-outer product scheme with an asymmetric Hopfield network.     

{\cal SMDP}: minimizing buffer requirements for continuous media servers

Excessive buffer requirement to handle continuous-media playbacks is an impediment to cost- effective provisioning for on-line video retrieval. Given the skewed distribution of video popularity, it is expected that often there are concurrent playbacks of the same video file within a short time interval. This creates an opportunity to batch multiple requests and to service them with a single stream from the disk without violating the on-demand constraint. However, there is a need to keep data in memory between successive uses to do this. This leads to a buffer space trade-off between servicing a request in memory mode vs. servicing it in disk-mode. In this work, we develop a novel algorithm to minimize the buffer requirement to support a set of concurrent playbacks. One of the beauties of the proposed scheme is that it enables the server to dynamically adapt to the changing workload while minimizing the total buffer space requirement. Our algorithm makes a significant contribution in decreasing the total buffer requirement, especially when the user access pattern is biased in favor of a small set of files. The idea of the proposed scheme is modeled in detail using an analytical formulation, and optimality of the algorithm is proved. An analytical framework is developed so that the proposed scheme can be used in combination with various existing disk-scheduling strategies. Our simulation results confirm that under certain circumstances, it is much more resource efficient to support some of the playbacks in memory mode and subsequently the proposed scheme enables the server to minimize the overall buffer space requirement.     

Parallel 1D and 2D vector quantizers using a Kohonen neural network

The process of reconstructing an original image from a compressed one is a difficult problem, since a large number of original images lead to the same compressed image and solutions to the inverse problem cannot be uniquely determined. Vector quantization is a compression technique that maps an input set of k-dimensional vectors into an output set of k-dimensional vectors, such that the selected output vector is closest to the input vector according to a selected distortion measure. In this paper, we show that adaptive 2D vector quantization of a fast discrete cosine transform of images using Kohonen neural networks outperforms other Kohonen vector quantizers in terms of quality (i.e. less distortion). A parallel implementation of the quantizer on a network of SUN Sparcstations is also presented.     

Consecutive multiline addressing: A scheme for addressing PMOLEDs

Abstract— Consecutive multiline addressing (CMLA) has been developed to increase a PMOLED display's lifetime, resolution, and power efficiency. Mathematically, it decomposes an image matrix into a set of multiline matrices and a residual single‐line matrix. The decomposition is lossless and implemented by a combinatorial algorithm allowing small chip size for the logic and high processing speed, e.g., for video applications. The additional memory needed for CMLA is just a fraction of the graphic data memory (GDRAM). The printed‐circuit‐board (PCB) prototype with a field programmable gate array (FPGA) proves that the CMLA produces images of the same visual quality as the conventional single‐line addressing (SLA), while the power efficiency is substantially higher.