Editorial: Special issue on approximate dynamic programming and reinforcement learning

We are extremely pleased to present this special issue of the Journal of Control Theory and Applications.Approximate dynamic programming (ADP) is a general and effective approach for solving optimal control and estimation problems by adapting to uncertain environments over time.ADP optimizes the sensing objectives accrued over a future time interval with respect to an adaptive control law,conditioned on prior knowledge of the system,its state,and uncertainties.A numerical search over the present value of the control minimizes a Hamilton-Jacobi-Bellman (HJB) equation providing a basis for real-time,approximate optimal control.     

From Images to Shape Models for Object Detection

We present an object class detection approach which fully integrates the complementary strengths offered by shape matchers. Like an object detector, it can learn class models directly from images, and can localize novel instances in the presence of intra-class variations, clutter, and scale changes. Like a shape matcher, it finds the boundaries of objects, rather than just their bounding-boxes. This is achieved by a novel technique for learning a shape model of an object class given images of example instances. Furthermore, we also integrate Hough-style voting with a non-rigid point matching algorithm to localize the model in cluttered images. As demonstrated by an extensive evaluation, our method can localize object boundaries accurately and does not need segmented examples for training (only bounding-boxes).     

Enhancing structural software coverage by incrementally computing branch executability

Structural code coverage criteria have been studied since the early seventies, and now they are well supported by commercial and open-source tools and are commonly embedded in several advanced industrial processes. Most industrial applications still refer to simple criteria, like statement and branch coverage, and consider complex criteria, like modified condition decision coverage, only rarely and often driven by the requirements of certification agencies. The industrial value of structural criteria is limited by the difficulty of achieving high coverage, due to both the complexity of deriving test cases that execute specific uncovered elements and the presence of many infeasible elements in the code. In this paper, we propose a technique that both generates test cases that execute yet uncovered branches and identifies infeasible branches that can be eliminated from the computation of the branch coverage. In this way, we can increase branch coverage to closely approximate full coverage, thus improving its industrial value. The algorithm combines symbolic analysis, abstraction refinement, and a novel technique named coarsening, to execute unexplored branches, identify infeasible ones, and mitigate the state space explosion problem. In the paper, we present the technique and illustrate its effectiveness through a set of experimental results obtained with a prototype implementation.     

Comparing sensitivities of differently oriented multi-walled carbon nanotubes integrated on silicon wafer for electrochemical biosensors

In this study, we report on multi-walled carbon nanotubes fabricated on silicon substrate with four different orientations via chemical vapor deposition. It is well-known that chemical treatments improve the nanotube electrochemical reactivity by creating edge-like defects on their exposed sidewalls. Before use, we performed an acid treatment on carbon nanotubes. To prove the effect of the treatment on these nanostructured electrodes, contact angles were measured. Then, sensitivities and detection limits were evaluated performing cyclic voltammetry. Two target molecules were used: potassium ferricyanide, an inorganic electroactive molecule, and hydrogen peroxide that is a product of reactions catalyzed by many enzymes, such as oxidases and peroxidases. Carbon nanotubes with tilted tips become hydrophilic after the treatment showing a contact angle of 22° ± 2°. This kind of electrode has shown also the best electrochemical performance. Sensitivity and detection limit values are 110.0 ± 0.5 μA/(mM cm²) and 8 μM for potassium ferricyanide solutions and 16.4 ± 0.1 μA/(mM cm²) and 24 μM using hydrogen peroxide as target compound. Considering the results of wettability and voltammetric measurements, nanotubes with tilted tips-based electrodes are found to be the most promising for future biosensing applications.     

Experimental setup for lensless imaging via soft x-ray resonant scattering

We have developed a setup for measuring holographically formed interference patterns using an integrated sample-mask design. The direct space image of the sample is obtained via a two-dimensional Fourier transform of the X-ray diffraction pattern. We present the details of our setup, commenting on the influence of geometrical parameters on the imaging capabilities. As an example, we present and discuss the results of test experiments on a patterned Co film.     

SusyMath: A Mathematica package for quantum superfield calculations

SusyMath is a Mathematica package for quantum superfield calculations. It defines a standard form to translate the correction to the effective action corresponding to a given supergraph into a Mathematica expression, which is then evaluated and simplified. Several functions for manipulations of these expressions are provided, and the package also has the ability to save the outcomes of its calculations in form.

Program summary

Title of program: SusyMathCatalogue identifier:ADYQ_v1_0Program summary URL:http://cpc.cs.qub.ac.uk/summaries/ADYQ_v1_0Program obtainable from: CPC Program Library, Queen's University of Belfast, N. Ireland, also at http://fma.if.usp.br/~alysson/SusyMathLicensing provisions: LGPL, CPC non-profit use licenseProgramming language: MathematicaPlatform: Any platform supporting Mathematica 4.0 or higherComputer tested on: PC (Athlon64 X2 +3800); 1 GB RAMOperating system under which the program has been tested: Linux (Debian 4.0); XOrg 7.0.22; Mathematica 5.2No. of lines in distributed program, including test data, etc.:42 472No. of bytes in distributed program, including test data, etc.:471 596Distribution format:tar.gzNature of the problem: Evaluate quantum corrections to the effective action of supersymmetric field theories, formulated in the superfield formalism, both in three- and four-spacetime dimensions.Solution method: A set of procedures for integration by parts, application of the algebra of covariant derivatives and Grassman integration, along with several auxiliary functions, is introduced.Restrictions: At the moment, the background field method is not implemented, but the system is designed to be further generalized.Running time: Depends on the complexity of the problem. From seconds for simpler one-loop diagrams to several hours for simple two-loop graphs.     

MPGS: an interactive tool for the specification and generation ofmultimedia presentations

Multimedia presentations are composed of objects belonging to different data types such as video, audio, text and image. An important aspect is that, quite often, the user defining a presentation needs to express sophisticated temporal and spatial constraints among the objects composing the presentation. We present a system (called MPGS-Multimedia Presentation Generator System) which supports the specification of constraints among multimedia objects and the generation of multimedia presentations according to the specified constraints. The constraint model provided by MPGS is very flexible and powerful in terms of the kinds of object constraints it can represent. A large number of innovative features are supported including: asynchronous and simultaneous spatial constraints; components of interest and priority levels; motion functions. Obviously, the flexibility provided to the users requires the development of nontrivial techniques to check constraint consistency and to generate a presentation satisfying the specified constraints. We illustrate the solutions we have devised in the framework of MPGS     

Learning extended FSA from software: An empirical assessment

A number of techniques that infer finite state automata from execution traces have been used to support test and analysis activities. Some of these techniques can produce automata that integrate information about the data-flow, that is, they also represent how data values affect the operations executed by programs.The integration of information about operation sequences and data values into a unique model is indeed conceptually useful to accurately represent the behavior of a program. However, it is still unclear whether handling heterogeneous types of information, such as operation sequences and data values, necessarily produces higher quality models or not.In this paper, we present an empirical comparative study between techniques that infer simple automata and techniques that infer automata extended with information about data-flow. We investigate the effectiveness of these techniques when applied to traces with different levels of sparseness, produced by different software systems. To the best of our knowledge this is the first work that quantifies both the effect of adding data-flow information within automata and the effectiveness of the techniques when varying sparseness of traces.