From Perturbation Analysis to Markov Decision Processes and Reinforcement Learning

The goals of perturbation analysis (PA), Markov decision processes (MDPs), and reinforcement learning (RL) are common: to make decisions to improve the system performance based on the information obtained by analyzing the current system behavior. In this paper, we study the relations among these closely related fields. We show that MDP solutions can be derived naturally from performance sensitivity analysis provided by PA. Performance potential plays an important role in both PA and MDPs; it also offers a clear intuitive interpretation for many results. Reinforcement learning, TD(), neuro-dynamic programming, etc., are efficient ways of estimating the performance potentials and related quantities based on sample paths. The sensitivity point of view of PA, MDP, and RL brings in some new insight to the area of learning and optimization. In particular, gradient-based optimization can be applied to parameterized systems with large state spaces, and gradient-based policy iteration can be applied to some nonstandard MDPs such as systems with correlated actions, etc. Potential-based on-line approaches and their advantages are also discussed.     

Global culture, local cultures and the internet: The Thai example

This paper addresses the questions of whether and, if so, how and to what extent the Internet brings about homogenisation of local cultures in the world. It examines a particular case, that of Thai culture, through an investigation and interpretation of a Usenet newsgroup, soc.culture.thai. Two threads of discussion in the newsgroup are selected. One deals with criticisms of the Thai government and political leaders, and the other focuses on whether the Thai language should be a medium, or perhaps the only medium, of communication in the newsgroup. It is found that, instead of erasing local cultural boundaries, creating a worldwide monolithic culture, the Internet reduplicates the existing cultural boundaries. What the Internet does, on the contrary, is to create an umbrella cosmopolitan culture which is necessary for communication among people from disparate cultures. That culture, however, is devoid of thick backgrounds, in Michael Walzer's sense.     

The historical workstation project

Since 1978 research in the development of software dedicated to the specific problems of historical research has been undertaken at the Max-Planck-Institute für Geschichte in Göttingen. From a background of practical experiences during these years, a concept of what an appropriate workstation for an historian would be has been derived. It stresses the necessity of three components: (a) software, derived from a detailed analysis of what differentiates information contained in historical sources from such present in current material, (b) databases which are as easily available as printed books and (c) knowledge bases which allow software and data bases to draw upon the information contained in historical reference works. A loose network of European research projects, dedicated to the realization of such a setup, is described.Manfred Thaller has a Ph.D. (1975) in Modern and Medieval History and held a post-doctoral fellowship in sociology at the Institute for Advanced Studies (Vienna). Since 1978 he has been at the Max-Planck-Institute for History, where he is a research fellow for Historical Information Science.     

A predicate transformer for the progress property ‘to-always’

The temporal property to-always has been proposed for specifying progress properties of concurrent programs. Although the to-always properties are a subset of the leads-to properties for a given program, to-always has more convenient proof rules and in some cases more accurately describes the desired system behavior. In this paper, we give a predicate transformerwta, derive some of its properties, and use it to define to-always. Proof rules for to-always are derived from the properties ofwta. We conclude by briefly describing two application areas, nondeterministic data flow networks and self-stabilizing systems where to-always properties are useful.     

On the Axioms of Scale Space Theory

We consider alternative scale space representations beyond the well-established Gaussian case that satisfy all reasonable axioms. One of these turns out to be subject to a first order pseudo partial differential equation equivalent to the Laplace equation on the upper half plane {(x, s) ^d × | s > 0}. We investigate this so-called Poisson scale space and show that it is indeed a viable alternative to Gaussian scale space. Poisson and Gaussian scale space are related via a one-parameter class of operationally well-defined intermediate representations generated by a fractional power of (minus) the spatial Laplace operator.     

Fast Online Q(λ)

Q()-learning uses TD()-methods to accelerate Q-learning. The update complexity of previous online Q() implementations based on lookup tables is bounded by the size of the state/action space. Our faster algorithm's update complexity is bounded by the number of actions. The method is based on the observation that Q-value updates may be postponed until they are needed.     

Automatic surface reconstruction with alpha-shape method

In this paper, we present an algorithm for the reconstruction of piecewise linear surfaces from unorganized sample points with an improved -shape. Alpha-shape provides a nice mathematical definition of the shape of a set of points, but the selection of an value is tricky in surface reconstruction. F or solving this problem and the non-uniform distribution, we scale the ball according to the points density. The method discussed in this paper might be applied for surface reconstruction, and the process is fully automatic.     

Automatic Extraction of Collocations From Korean Text

In this paper, we propose a statistical method to automaticallyextract collocations from Korean POS-tagged corpus. Since a large portion of language is represented by collocation patterns, the collocational knowledge provides a valuable resource for NLP applications. One difficulty of collocation extraction is that Korean has a partially free word order, which also appears in collocations. In this work, we exploit four statistics, frequency,randomness, convergence, and correlation' in order to take into account the flexible word order of Korean collocations. We separate meaningful bigrams using an evaluation function based on the four statistics and extend the bigrams to n-gram collocations using a fuzzy relation. Experiments show that this method works well for Korean collocations.     

A Problem of Control of Random Processes

A control problem for a random process depending additively on the fractional Brownian motion with the Hurst parameter H (1/2, 1) is analyzed.     

Design and analysis of two-wire focusing actuator for small-sized ODD with linear VCM type’s actuator

Recent advancements in mobile devices have fueled a requirement for information storage systems with characteristics such as subminiature size, low cost, and minimum power consumption. Small optical disk drives could provide a good solution, because their storage media is cheaper than those of hard disk drives or flash memories. In this study, we design and analysis a linear voice coil motor (VCM) types actuator for the coarse and fine tracking motions and a two-wire spring types actuator for a fine focusing motion in order to apply them to small-sized optical disk drives of high data capacity. And, we make the mathematical model of a two-wire spring types actuator and find proper conditions to suppress the tilt motion of the lens holder for various focusing offset positions without sacrificing the sensitivity. Based on them, we design and modify the magnetic circuit and the structural part. Finally, we verify that the designed actuator had the proper dynamic characteristics and that it could be used in small optical disk drives by simulations and experiments.     

Mechanical procedure for proof construction via closed terms in typed λ calculus

In this paper is presented an algorithm for constructing natural deduction proofs in the propositional intuitionistic and classical logics according to the analogy relating intuitionistic propositional formulas and natural deduction proofs, respectively, to types and terms of simple type theory. Proofs are constructed as closed terms in the simple typed calculus. The soundness and completeness of this method are proved.     

The π-Calculus in Direct Style

We introduce a calculus which is a direct extension of both the and the calculi. We give a simple type system for it, that encompasses both Curry's type inference for the -calculus, and Milner's sorting for the -calculus as particular cases of typing. We observe that the various continuation passing style transformations for -terms, written in our calculus, actually correspond to encodings already given by Milner and others for evaluation strategies of -terms into the -calculus. Furthermore, the associated sortings correspond to well-known double negation translations on types. Finally we provide an adequate CPS transform from our calculus to the -calculus. This shows that the latter may be regarded as an assembly language, while our calculus seems to provide a better programming notation for higher-order concurrency. We conclude by discussing some alternative design decisions.     

Complex-Weighted One-Unit ‘Rigid-Bodies’ Learning Rule for Independent Component Analysis

Over the recent years, noticeable theoretical efforts have been devoted to the understanding of the role of networks' parameter spaces in neural learning. One of the contributions in this field concerns the study of weight-flows on Stiefel manifold, which is the natural parameter-space's algebraic-structure in some unsupervised (information-theoretic) learning task. An algorithm belonging to the class of learning equations generating Stiefel-flows is based on the rigid-body theory, introduced by the present Author in 1996. The aim of this Letter is to present an investigation on the capability of a complex-weighted neuron, trained by a rigid-bodies learning theory, with application to blind source separation of complex-valued independent signals for telecommunication systems.     

Dynamics of Lifeboat and Boom in Deployment from a Moving Support

The equations governing the boom-assisted deployment of a lifeboat from a moving support are formulated in three dimensions using Kanes method, accounting for the coupling of the dynamics of the boom and lifeboat. The effect of wind loading is included. At splashdown we assume a controlled water entry and hence the buoyancy force, but not impact force, is modeled. The equations are solved numerically using a standard Runge-Kutta MATLAB routine. Comparison with a known solution for a special case is illustrated.     

Formal Foundations of Operational Semantics

In this paper we report on the results of a sophisticated and substantial use of PVS to establish a recent result in operational semantics. The result we establish is a context lemma for operational equivalence for very wide class of programming languages, known as the CIU theorem. The proof uses the annotated holes technique to represent contexts and compute with them. Thus this paper demonstrates that that it is possible to use PVS as a tool in the development of modern operational techniques, and a productive tool at that. The process of formalizing the CIU theorem revealed several gaps in published proof. The proof of the CIU theorem in PVS took approximately six months to develop. The actual machine checked proof involves the proving of around one thousand facts, and takes PVS slightly less than three hours of CPU time running on a Linux machine configured with 2 GBytes of main memory and four 550 MHz Xeon PIII processors.     

From Pre-Historic to Post-Modern Symbolic Model Checking

Symbolic model checking, which enables the automatic verification of large systems, proceeds by calculating expressions that represent state sets. Traditionally, symbolic model-checking tools are based on backward state traversal; their basic operation is the function pre, which, given a set of states, returns the set of all predecessor states. This is because specifiers usually employ formalisms with future-time modalities, which are naturally evaluated by iterating applications of pre. It has been shown experimentally that symbolic model checking can perform significantly better if it is based, instead, on forward state traversal; in this case, the basic operation is the function post, which, given a set of states, returns the set of all successor states. This is because forward state traversal can ensure that only parts of the state space that are reachable from an initial state and relevant for the satisfaction or violation of the specification are explored; that is, errors can be detected as soon as possible.In this paper, we investigate which specifications can be checked by symbolic forward state traversal. We formulate the problems of symbolic backward and forward model checking by means of two -calculi. The pre- calculus is based on the pre operation, and the post- calculus is based on the post operation. These two -calculi induce query logics, which augment fixpoint expressions with a boolean emptiness query. Using query logics, we are able to relate and compare the symbolic backward and forward approaches. In particular, we prove that all -regular (linear-time) specifications can be expressed as post- queries, and therefore checked using symbolic forward state traversal. On the other hand, we show that there are simple branching-time specifications that cannot be checked in this way.     

Globally Optimal Trajectory Planning for Redundant Manipulators using State Space Augmentation Method

The off-line global trajectory planning for kinematically redundant manipulators is formulated as an optimization problem whose solution is obtained by applying the Pontryagins Maximum Principle. The state space augmentation method is developed to obtain a set of optimal joint trajectories corresponding to a singularity-free Cartesian path which avoids joint limits and conserves joint configuration in cyclic motion. Results of computer simulation conducted on a three-degree-of-freedom planar manipulator are presented and discussed.     

Modal Pure Type Systems

We present a framework for intensional reasoning in typed -calculus. In this family of calculi, called Modal Pure Type Systems (MPTSs), a propositions-as-types-interpretation can be given for normal modal logics. MPTSs are an extension of the Pure Type Systems (PTSs) of Barendregt (1992). We show that they retain the desirable meta-theoretical properties of PTSs, and briefly discuss applications in the area of knowledge representation.