Editorial: Special issue on approximate dynamic programming and reinforcement learning

We are extremely pleased to present this special issue of the Journal of Control Theory and Applications.Approximate dynamic programming (ADP) is a general and effective approach for solving optimal control and estimation problems by adapting to uncertain environments over time.ADP optimizes the sensing objectives accrued over a future time interval with respect to an adaptive control law,conditioned on prior knowledge of the system,its state,and uncertainties.A numerical search over the present value of the control minimizes a Hamilton-Jacobi-Bellman (HJB) equation providing a basis for real-time,approximate optimal control.     

A model-based approximate λ-policy iteration approach to online evasive path planning and the video game Ms. Pac-Man

This paper presents a model-based approximate λ-policy iteration approach using temporal differences for optimizing paths online for a pursuit-evasion problem, where an agent must visit several target positions within a region of interest while simultaneously avoiding one or more actively pursuing adversaries. This method is relevant to applications, such as robotic path planning, mobile-sensor applications, and path exposure. The methodology described utilizes cell decomposition to construct a decision tree and implements a temporal difference-based approximate λ-policy iteration to combine online learning with prior knowledge through modeling to achieve the objectives of minimizing the risk of being caught by an adversary and maximizing a reward associated with visiting target locations. Online learning and frequent decision tree updates allow the algorithm to quickly adapt to unexpected movements by the adversaries or dynamic environments. The approach is illustrated through a modified version of the video game Ms. Pac-Man, which is shown to be a benchmark example of the pursuit-evasion problem. The results show that the approach presented in this paper outperforms several other methods as well as most human players.     

Usability evaluation methods for the web: A systematic mapping study

Context

In recent years, many usability evaluation methods (UEMs) have been employed to evaluate Web applications. However, many of these applications still do not meet most customers’ usability expectations and many companies have folded as a result of not considering Web usability issues. No studies currently exist with regard to either the use of usability evaluation methods for the Web or the benefits they bring.

Objective

The objective of this paper is to summarize the current knowledge that is available as regards the usability evaluation methods (UEMs) that have been employed to evaluate Web applications over the last 14 years.

Method

A systematic mapping study was performed to assess the UEMs that have been used by researchers to evaluate Web applications and their relation to the Web development process. Systematic mapping studies are useful for categorizing and summarizing the existing information concerning a research question in an unbiased manner.

Results

The results show that around 39% of the papers reviewed reported the use of evaluation methods that had been specifically crafted for the Web. The results also show that the type of method most widely used was that of User Testing. The results identify several research gaps, such as the fact that around 90% of the studies applied evaluations during the implementation phase of the Web application development, which is the most costly phase in which to perform changes. A list of the UEMs that were found is also provided in order to guide novice usability practitioners.

Conclusions

From an initial set of 2703 papers, a total of 206 research papers were selected for the mapping study. The results obtained allowed us to reach conclusions concerning the state-of-the-art of UEMs for evaluating Web applications. This allowed us to identify several research gaps, which subsequently provided us with a framework in which new research activities can be more appropriately positioned, and from which useful information for novice usability practitioners can be extracted.     

To Score or Not to Score? Tripling Insights for Participatory Design

For evaluating visual-analytics tools, many studies confine to scoring user insights into data. For participatory design of those tools, we propose a three-level methodology to make more out of users' insights. The Relational Insight Organizer (RIO) helps to understand how insights emerge and build on one each other.     

Visual methods for analyzing time-oriented data

Providing appropriate methods to facilitate the analysis of time-oriented data is a key issue in many application domains. In this paper, we focus on the unique role of the parameter time in the context of visually driven data analysis. We will discuss three major aspects - visualization, analysis, and the user. It will be illustrated that it is necessary to consider the characteristics of time when generating visual representations. For that purpose we take a look at different types of time and present visual examples. Integrating visual and analytical methods has become an increasingly important issue. Therefore, we present our experiences in temporal data abstraction, principal component analysis, and clustering of larger volumes of time-oriented data. The third main aspect we discuss is supporting user-centered visual analysis. We describe event-based visualization as a promising means to adapt the visualization pipeline to needs and tasks of users.     

Human cytidine deaminase: a three-dimensional homology model of a tetrameric metallo-enzyme inferred from the crystal structure of a distantly related dimeric homologue

Cytidine deaminase (CDA) is a cytosolic metalloprotein whose functional unit can be either a homotetramer (T-CDA) or a homodimer (D-CDA), depending on the species. In 1994, the first crystal structure of the dimeric Escherichia coli CDA has been published. However, a crystal structure of a tetrameric CDA was not determined until 2002. Prior to the disclosure of the experimentally elucidated structure of a tetrameric CDA, we derived a homology model of the human T-CDA employing the crystal structure of the dimeric E. coli CDA as a template. The comparison of our theoretical model with the crystal structure of the human T-CDA, subsequently published in 2004, validates our prediction: not only of the structural features of the monomer and the details of the binding site, but also the multimeric arrangement of the subunits were determined with high accuracy in our model. By means of a phylogenetic analysis conducted on CDAs from various organisms, we demonstrate that the E. coli CDA is one of the furthest known homologues of the human enzyme. Nonetheless, despite the evolutionary distance and, more importantly, the different multimeric arrangement of their functional units, the E. coli CDA proved to have all the necessary information to accurately infer the structure of its human homologue.     

Development and validation of “grey-box” models for refrigeration applications: A review of key concepts

“Grey-box” modelling combines the use of first-principle based “white-box” models and empirical “black-box” models, offering particular benefits when: (a) there is a lack of fundamental theory to describe the system or process modelled; (b) there is a scarcity of suitable experimental data for validation or (c) there is a need to decrease the complexity of the model. The grey-box approach has been used, for example, to create mathematical models to predict the shelf life of chilled products or the thermal behaviour of imperfectly mixed fluids, or to create models that combine artificial neural networks and dynamic differential equations for control-related applications. This paper discusses the main characteristics of white-box, black-box and their integration into grey-box models, the requirements and sourcing of accurate data for model development and important validation concepts and measures.     

First Report on Genome Editing via Ribonucleoprotein (RNP) in Castanea sativa Mill.

Castanea sativa is an important tree nut species worldwide, highly appreciated for its multifunctional role, in particular for timber and nut production. Nowadays, new strategies are needed to achieve plant resilience to diseases, climate change, higher yields, and nutritional quality. Among the new plant breeding techniques (NPBTs), the CRISPR/Cas9 system represents a powerful tool to improve plant breeding in a short time and inexpensive way. In addition, the CRISPR/Cas9 construct can be delivered into the cells in the form of ribonucleoproteins (RNPs), avoiding the integration of exogenous DNA (GMO-free) through protoplast technology that represents an interesting material for gene editing thanks to the highly permeable membrane to DNA. In the present study, we developed the first protoplast isolation protocol starting from European chestnut somatic embryos. The enzyme solution optimized for cell wall digestion contained 1% cellulase Onozuka R-10 and 0.5% macerozyme R-10. After incubation for 4 h at 25 °C in dark conditions, a yield of 4,500,000 protoplasts/mL was obtained (91% viable). The transfection capacity was evaluated using the GFP marker gene, and the percentage of transfected protoplasts was 51%, 72 h after the transfection event. The direct delivery of the purified RNP was then performed targeting the phytoene desaturase gene. Results revealed the expected target modification by the CRISPR/Cas9 RNP and the efficient protoplast editing.     

Characterisation of Progressive Skeletal Muscle Fibrosis in the Mdx Mouse Model of Duchenne Muscular Dystrophy: An In Vivo and In Vitro Study

Duchenne muscular dystrophy (DMD) is a rare genetic disease leading to progressive muscle wasting, respiratory failure, and cardiomyopathy. Although muscle fibrosis represents a DMD hallmark, the organisation of the extracellular matrix and the molecular changes in its turnover are still not fully understood. To define the architectural changes over time in muscle fibrosis, we used an mdx mouse model of DMD and analysed collagen and glycosaminoglycans/proteoglycans content in skeletal muscle sections at different time points during disease progression and in comparison with age-matched controls. Collagen significantly increased particularly in the diaphragm, quadriceps, and gastrocnemius in adult mdx, with fibrosis significantly correlating with muscle degeneration. We also analysed collagen turnover pathways underlying fibrosis development in cultured primary quadriceps-derived fibroblasts. Collagen secretion and matrix metalloproteinases (MMPs) remained unaffected in both young and adult mdx compared to wt fibroblasts, whereas collagen cross-linking and tissue inhibitors of MMP (TIMP) expression significantly increased. We conclude that, in the DMD model we used, fibrosis mostly affects diaphragm and quadriceps with a higher collagen cross-linking and inhibition of MMPs that contribute differently to progressive collagen accumulation during fibrotic remodelling. This study offers a comprehensive histological and molecular characterisation of DMD-associated muscle fibrosis; it may thus provide new targets for tailored therapeutic interventions.