Editorial: Special issue on approximate dynamic programming and reinforcement learning

We are extremely pleased to present this special issue of the Journal of Control Theory and Applications.Approximate dynamic programming (ADP) is a general and effective approach for solving optimal control and estimation problems by adapting to uncertain environments over time.ADP optimizes the sensing objectives accrued over a future time interval with respect to an adaptive control law,conditioned on prior knowledge of the system,its state,and uncertainties.A numerical search over the present value of the control minimizes a Hamilton-Jacobi-Bellman (HJB) equation providing a basis for real-time,approximate optimal control.     

A model-based approximate λ-policy iteration approach to online evasive path planning and the video game Ms. Pac-Man

This paper presents a model-based approximate λ-policy iteration approach using temporal differences for optimizing paths online for a pursuit-evasion problem, where an agent must visit several target positions within a region of interest while simultaneously avoiding one or more actively pursuing adversaries. This method is relevant to applications, such as robotic path planning, mobile-sensor applications, and path exposure. The methodology described utilizes cell decomposition to construct a decision tree and implements a temporal difference-based approximate λ-policy iteration to combine online learning with prior knowledge through modeling to achieve the objectives of minimizing the risk of being caught by an adversary and maximizing a reward associated with visiting target locations. Online learning and frequent decision tree updates allow the algorithm to quickly adapt to unexpected movements by the adversaries or dynamic environments. The approach is illustrated through a modified version of the video game Ms. Pac-Man, which is shown to be a benchmark example of the pursuit-evasion problem. The results show that the approach presented in this paper outperforms several other methods as well as most human players.     

Semantic Web computing in industry

The Semantic Web has attracted significant attention during the last decade. On the one hand, many research groups have changed their focus towards Semantic Web research and research funding agencies particularly in Europe have explicitly mentioned Semantic Web in their calls for proposals. On the other hand, industry has also begun to watch developments with interest and a number of large companies have started to experiment with Semantic Web technologies to ascertain if these new technologies can be leveraged to add more value for their customers or internally within the company, while there are already several offers of vendors of Semantic Web solutions on the market. The essence of the Semantic Web is to structure Web-based information to make it more interoperable, machine-readable and thereafter to provide a means to relate various information concepts more easily and in a reusable way. The Semantic Web acts as an additional layer on the top of the Web, and is built around explicit representations of information concepts and their relationships such as ontologies and taxonomies. Furthermore, Semantic Web technologies are not only valuable on an open environment like the Web, but also in closed systems such as in industrial settings. Hence, these technologies can be efficiently deployed for domains including Web Services, Enterprise Application Integration, Knowledge Management and E-Commerce, fulfilling existing gaps in current applications. This paper focuses on this synthesis between Semantic Web technologies and systems problems within industrial applications. There will be a short review of Semantic Web standards, languages and technologies followed by a more detailed review of applications of Semantic Web computing in industry. The paper covers theoretical considerations as well as use cases and experience reports on the topic, and we also present some current challenges and opportunities in the domain.     

Usability evaluation methods for the web: A systematic mapping study

Context

In recent years, many usability evaluation methods (UEMs) have been employed to evaluate Web applications. However, many of these applications still do not meet most customers’ usability expectations and many companies have folded as a result of not considering Web usability issues. No studies currently exist with regard to either the use of usability evaluation methods for the Web or the benefits they bring.

Objective

The objective of this paper is to summarize the current knowledge that is available as regards the usability evaluation methods (UEMs) that have been employed to evaluate Web applications over the last 14 years.

Method

A systematic mapping study was performed to assess the UEMs that have been used by researchers to evaluate Web applications and their relation to the Web development process. Systematic mapping studies are useful for categorizing and summarizing the existing information concerning a research question in an unbiased manner.

Results

The results show that around 39% of the papers reviewed reported the use of evaluation methods that had been specifically crafted for the Web. The results also show that the type of method most widely used was that of User Testing. The results identify several research gaps, such as the fact that around 90% of the studies applied evaluations during the implementation phase of the Web application development, which is the most costly phase in which to perform changes. A list of the UEMs that were found is also provided in order to guide novice usability practitioners.

Conclusions

From an initial set of 2703 papers, a total of 206 research papers were selected for the mapping study. The results obtained allowed us to reach conclusions concerning the state-of-the-art of UEMs for evaluating Web applications. This allowed us to identify several research gaps, which subsequently provided us with a framework in which new research activities can be more appropriately positioned, and from which useful information for novice usability practitioners can be extracted.     

To Score or Not to Score? Tripling Insights for Participatory Design

For evaluating visual-analytics tools, many studies confine to scoring user insights into data. For participatory design of those tools, we propose a three-level methodology to make more out of users' insights. The Relational Insight Organizer (RIO) helps to understand how insights emerge and build on one each other.     

Visual methods for analyzing time-oriented data

Providing appropriate methods to facilitate the analysis of time-oriented data is a key issue in many application domains. In this paper, we focus on the unique role of the parameter time in the context of visually driven data analysis. We will discuss three major aspects - visualization, analysis, and the user. It will be illustrated that it is necessary to consider the characteristics of time when generating visual representations. For that purpose we take a look at different types of time and present visual examples. Integrating visual and analytical methods has become an increasingly important issue. Therefore, we present our experiences in temporal data abstraction, principal component analysis, and clustering of larger volumes of time-oriented data. The third main aspect we discuss is supporting user-centered visual analysis. We describe event-based visualization as a promising means to adapt the visualization pipeline to needs and tasks of users.     

Human cytidine deaminase: a three-dimensional homology model of a tetrameric metallo-enzyme inferred from the crystal structure of a distantly related dimeric homologue

Cytidine deaminase (CDA) is a cytosolic metalloprotein whose functional unit can be either a homotetramer (T-CDA) or a homodimer (D-CDA), depending on the species. In 1994, the first crystal structure of the dimeric Escherichia coli CDA has been published. However, a crystal structure of a tetrameric CDA was not determined until 2002. Prior to the disclosure of the experimentally elucidated structure of a tetrameric CDA, we derived a homology model of the human T-CDA employing the crystal structure of the dimeric E. coli CDA as a template. The comparison of our theoretical model with the crystal structure of the human T-CDA, subsequently published in 2004, validates our prediction: not only of the structural features of the monomer and the details of the binding site, but also the multimeric arrangement of the subunits were determined with high accuracy in our model. By means of a phylogenetic analysis conducted on CDAs from various organisms, we demonstrate that the E. coli CDA is one of the furthest known homologues of the human enzyme. Nonetheless, despite the evolutionary distance and, more importantly, the different multimeric arrangement of their functional units, the E. coli CDA proved to have all the necessary information to accurately infer the structure of its human homologue.     

Analytical and graphical evaluation of local quasi-stable angular points during ion bombardment at oblique incidence

An implied analytical condition, connecting the incidence angles _o and _e between the two intersecting microfacet normals and the ion beam direction, has been inferred for the existence of local quasi-stable intersections (angular points) during ion erosion. This condition, which can also be deduced on the basis of the erosion slowness curve, leads to a higher degree algebraic equation with a parameter, applicable for any dependence S=S (), expressed as an algebraic polynomial expression in cos . A new graphical method for evaluation of orientations _e compatible with a given orientation _o, based on a known polar diagram cursor-type, has also been proposed.     

SEM investigations of iron surface ion erosion as a function of specimen temperature and incidence angle

Ion bombardment of iron surfaces produced an ion eroded surface microtopography strongly dependent on the incidence angle and specimen temperature. For bombardment temperatures exceeding the self-diffusion temperature, T _d, a microstructural smoothing process due to the thermally activated diffusion effects, assisted by ion bombardment, can be observed by SEM. The role of migrational processes which should be taken into account for a correct interpretation of the final surface topography formed by hot sputtering, was also considered.     

First Report on Genome Editing via Ribonucleoprotein (RNP) in Castanea sativa Mill.

Castanea sativa is an important tree nut species worldwide, highly appreciated for its multifunctional role, in particular for timber and nut production. Nowadays, new strategies are needed to achieve plant resilience to diseases, climate change, higher yields, and nutritional quality. Among the new plant breeding techniques (NPBTs), the CRISPR/Cas9 system represents a powerful tool to improve plant breeding in a short time and inexpensive way. In addition, the CRISPR/Cas9 construct can be delivered into the cells in the form of ribonucleoproteins (RNPs), avoiding the integration of exogenous DNA (GMO-free) through protoplast technology that represents an interesting material for gene editing thanks to the highly permeable membrane to DNA. In the present study, we developed the first protoplast isolation protocol starting from European chestnut somatic embryos. The enzyme solution optimized for cell wall digestion contained 1% cellulase Onozuka R-10 and 0.5% macerozyme R-10. After incubation for 4 h at 25 °C in dark conditions, a yield of 4,500,000 protoplasts/mL was obtained (91% viable). The transfection capacity was evaluated using the GFP marker gene, and the percentage of transfected protoplasts was 51%, 72 h after the transfection event. The direct delivery of the purified RNP was then performed targeting the phytoene desaturase gene. Results revealed the expected target modification by the CRISPR/Cas9 RNP and the efficient protoplast editing.