The LBG-U Method for Vector Quantization – an Improvement over LBG Inspired from Neural Networks

A new vector quantization method (LBG-U) closely related to a particular class of neural network models (growing self-organizing networks) is presented. LBG-U consists mainly of repeated runs of the well-known LBG algorithm. Each time LBG converges, however, a novel measure of utility is assigned to each codebook vector. Thereafter, the vector with minimum utility is moved to a new location, LBG is run on the resulting modified codebook until convergence, another vector is moved, and so on. Since a strictly monotonous improvement of the LBG-generated codebooks is enforced, it can be proved that LBG-U terminates in a finite number of steps. Experiments with artificial data demonstrate significant improvements in terms of RMSE over LBG combined with only modestly higher computational costs.     

Consensus-based fault-tolerant total order multicast

While total order broadcast (or atomic broadcast) primitives have received a lot of attention, this paper concentrates on total order multicast to multiple groups in the context of asynchronous distributed systems in which processes may suffer crash failures. “Multicast to Multiple Groups” means that each message is sent to a subset of the process groups composing the system, distinct messages possibly having distinct destination groups. “Total Order” means that all message deliveries must be totally ordered. This paper investigates a consensus-based approach to solve this problem and proposes a corresponding protocol to implement this multicast primitive. This protocol is based on two underlying building blocks, namely, uniform reliable multicast and uniform consensus. Its design characteristics lie in the two following properties. The first one is a minimality property, more precisely, only the sender of a message and processes of its destination groups have to participate in the total order multicast of the message. The second property is a locality property: No execution of a consensus has to involve processes belonging to distinct groups (i.e., consensus is executed on a “per group” basis). This locality property is particularly useful when one is interested in using the total order multicast primitive in large-scale distributed systems. In addition to a correctness proof, an improvement that reduces the cost of the protocol is also suggested     

Growing Grid — a self-organizing network with constant neighborhood range and adaptation strength

We present a novel self-organizing network which is generated by a growth process. The application range of the model is the same as for Kohonen’s feature map: generation of topology-preserving and dimensionality-reducing mappings, e.g., for the purpose of data visualization. The network structure is a rectangular grid which, however, increases its size during self-organization. By inserting complete rows or columns of units the grid may adapt its height/width ratio to the given pattern distribution. Both the neighborhood range used to co-adapt units in the vicinity of the winning unit and the adaptation strength are constant during the growth phase. This makes it possible to let the network grow until an application-specific performance criterion is fulfilled or until a desired network size is reached. A final approximation phase with decaying adaptation strength finetunes the network.     

Fast learning with incremental RBF networks

We present a new algorithm for the construction of radial basis function (RBF) networks. The method uses accumulated error information to determine where to insert new units. The diameter of the localized units is chosen based on the mutual distances of the units. To have the distance information always available, it is held up-to-date by a Hebbian learning rule adapted from the Neural Gas≓ algorithm. The new method has several advantages over existing methods and is able to generate small, well-generalizing networks with comparably few sweeps through the training data.