Editorial: Special issue on approximate dynamic programming and reinforcement learning

We are extremely pleased to present this special issue of the Journal of Control Theory and Applications.Approximate dynamic programming (ADP) is a general and effective approach for solving optimal control and estimation problems by adapting to uncertain environments over time.ADP optimizes the sensing objectives accrued over a future time interval with respect to an adaptive control law,conditioned on prior knowledge of the system,its state,and uncertainties.A numerical search over the present value of the control minimizes a Hamilton-Jacobi-Bellman (HJB) equation providing a basis for real-time,approximate optimal control.     

Polygon subdivision for pocket machining process planning

This paper presents a new approach to improve tool selection for arbitrary shaped pockets based on an approximate polygon subdivision technique. The pocket is subdivided into smaller sub-polygons and tools are selected separately for each sub-polygon. A set of tools for the entire pocket is obtained based on both machining time and the number of tools used. In addition, the sub-polygons are sequenced to eliminate the requirement of multiple plunging operations. In process planning for pocket machining, selection of tool sizes and minimizing the number of plunging operations can be very important factors. The approach presented in this paper is an improvement over previous work in its use of a polygon subdivision strategy to improve the machining time as well as reducing the number of plunges. The implementation of this technique suggests that using a subdivision approach can reduce machining time when compared to solving for the entire polygonal region.     

Succinctness of pattern-based schema languages for XML

Martens et al. defined a pattern-based specification language equivalent in expressive power to the widely adopted XML Schema definitions (XSDs). This language consists of rules of the form $(r,s)$ where r and s are regular expressions and can be seen as a type-free extension of DTDs with vertical regular expressions. Sets of such rules can be interpreted both in an existential or universal way. In the present paper, we study the succinctness of both semantics w.r.t. each other and w.r.t. the common abstraction of XSDs in terms of single-type extended DTDs. The investigation is carried out relative to three kinds of vertical pattern languages: regular, linear, and strongly linear patterns. We also consider the complexity of the simplification problem for each of the considered pattern-based schemas.     

Uncountable automatic classes and learning

In this paper we consider uncountable classes recognizable by ω-automata and investigate suitable learning paradigms for them. In particular, the counterparts of explanatory, vacillatory and behaviourally correct learning are introduced for this setting. Here the learner reads in parallel the data of a text for a language L from the class plus an ω-index α and outputs a sequence of ω-automata such that all but finitely many of these ω-automata accept the index α if and only if α is an index for L.It is shown that any class is behaviourally correct learnable if and only if it satisfies Angluin’s tell-tale condition. For explanatory learning, such a result needs that a suitable indexing of the class is chosen. On the one hand, every class satisfying Angluin’s tell-tale condition is vacillatorily learnable in every indexing; on the other hand, there is a fixed class such that the level of the class in the hierarchy of vacillatory learning depends on the indexing of the class chosen.We also consider a notion of blind learning. On the one hand, a class is blind explanatorily (vacillatorily) learnable if and only if it satisfies Angluin’s tell-tale condition and is countable; on the other hand, for behaviourally correct learning, there is no difference between the blind and non-blind version.This work establishes a bridge between the theory of ω-automata and inductive inference (learning theory).     

QDENSITY—A Mathematica Quantum Computer simulation

This Mathematica 5.2 package¹ is a simulation of a Quantum Computer. The program provides a modular, instructive approach for generating the basic elements that make up a quantum circuit. The main emphasis is on using the density matrix, although an approach using state vectors is also implemented in the package. The package commands are defined in Qdensity.m which contains the tools needed in quantum circuits, e.g., multiqubit kets, projectors, gates, etc. Selected examples of the basic commands are presented here and a tutorial notebook, Tutorial.nb is provided with the package (available on our website) that serves as a full guide to the package. Finally, application is made to a variety of relevant cases, including Teleportation, Quantum Fourier transform, Grover's search and Shor's algorithm, in separate notebooks: QFT.nb, Teleportation.nb, Grover.nb and Shor.nb where each algorithm is explained in detail. Finally, two examples of the construction and manipulation of cluster states, which are part of “one way computing” ideas, are included as an additional tool in the notebook Cluster.nb. A Mathematica palette containing most commands in QDENSITY is also included: QDENSpalette.nb.

Program summary

Title of program: QDENSITYCatalogue identifier: ADXH_v1_0Program summary URL:http://cpc.cs.qub.ac.uk/summaries/ADXH_v1_0Program available from: CPC Program Library, Queen's University of Belfast, N. IrelandOperating systems: Any which supports Mathematica; tested under Microsoft Windows XP, Macintosh OS X, and Linux FC4Programming language used: Mathematica 5.2No. of bytes in distributed program, including test data, etc.: 180 581No. of lines in distributed program, including test data, etc.: 19 382Distribution format: tar.gzMethod of solution: A Mathematica package is provided which contains commands to create and analyze quantum circuits. Several Mathematica notebooks containing relevant examples: Teleportation, Shor's Algorithm and Grover's search are explained in detail. A tutorial, Tutorial.nb is also enclosed.     

MCMC data association and sparse factorization updating for real time multitarget tracking with merged and multiple measurements

In several multitarget tracking applications, a target may return more than one measurement per target and interacting targets may return multiple merged measurements between targets. Existing algorithms for tracking and data association, initially applied to radar tracking, do not adequately address these types of measurements. Here, we introduce a probabilistic model for interacting targets that addresses both types of measurements simultaneously. We provide an algorithm for approximate inference in this model using a Markov chain Monte Carlo (MCMC)-based auxiliary variable particle filter. We Rao-Blackwellize the Markov chain to eliminate sampling over the continuous state space of the targets. A major contribution of this work is the use of sparse least squares updating and downdating techniques, which significantly reduce the computational cost per iteration of the Markov chain. Also, when combined with a simple heuristic, they enable the algorithm to correctly focus computation on interacting targets. We include experimental results on a challenging simulation sequence. We test the accuracy of the algorithm using two sensor modalities, video, and laser range data. We also show the algorithm exhibits real time performance on a conventional PC     

Void formation in an Al-Mg-Si alloy under different precipitation conditions after irradiation with 100 keV Al ions

The influence of different precipitation treatments upon the radiation-induced void formation was studied on an Al-Mg-Si alloy and the results were compared with those obtained on high-purity Al. The changes in the microstructure and the void formation were investigated by TEM methods for the dose range 0.8 to 80 dpa and the temperature range 55 to 250° C. The irradiations were carried out using 100 keV Al ions. Highpurity Al showed void formation over the whole temperature range investigated with a maximum volume increase V/V of 3.4% at 150° C. The behaviour of the Al-Mg-Si alloys depends strongly upon the thermal pretreatments. The alloy in its homogenized state shows no voids between 55 and 140° C. Trapping of vacancies on solute Si atoms could be the explanation. An ageing treatment leading to coherent precipitates results in the complete suppression of void formation for the temperature region of highest swelling (0.35 T _sT0.44 T _s). On the other hand, treatments which cause incoherent or partially coherent precipitates result in swelling. The amount of swelling, however, is lower compared to pure Al and the temperature of the swell maximum is shifted to lower temperatures. These results could be explained by the mechanism of trapping vacancies by solute atoms, on the one hand, and the coherent precipitates acting as recombination centres, on the other.     

Prerequisites for offering ultrasound services in general practice]

In the course of the past few years, sonography has become firmly established in clinical practice. The accuracy of the investigations does not only depend on the quality of the equipment used but also to a considerable degree on the standards of investigator training. To reduce the discrepancy between the number and quality of the investigations in the German Federal Republic, the "Kassen?rztliche Vereinigung" has released guidelines requiring the users of sonographic equipment to undergo theoretical and practical training. These guidelines are reviewed and commented.