Transfer in variable-reward hierarchical reinforcement learning

Transfer learning seeks to leverage previously learned tasks to achieve faster learning in a new task. In this paper, we consider transfer learning in the context of related but distinct Reinforcement Learning (RL) problems. In particular, our RL problems are derived from Semi-Markov Decision Processes (SMDPs) that share the same transition dynamics but have different reward functions that are linear in a set of reward features. We formally define the transfer learning problem in the context of RL as learning an efficient algorithm to solve any SMDP drawn from a fixed distribution after experiencing a finite number of them. Furthermore, we introduce an online algorithm to solve this problem, Variable-Reward Reinforcement Learning (VRRL), that compactly stores the optimal value functions for several SMDPs, and uses them to optimally initialize the value function for a new SMDP. We generalize our method to a hierarchical RL setting where the different SMDPs share the same task hierarchy. Our experimental results in a simplified real-time strategy domain show that significant transfer learning occurs in both flat and hierarchical settings. Transfer is especially effective in the hierarchical setting where the overall value functions are decomposed into subtask value functions which are more widely amenable to transfer across different SMDPs.     

Self-managed topologies in P2P networks

The problem of efficient resource location is an important open issue in P2P systems. This paper introduces DANTE, a self-adapting P2P system that changes its peer links to form topologies where resources are located in an efficient manner via random walks. Additionally, this same self-adaptation capacity makes DANTE capable of reacting to events like changes in the system load or attacks on well-connected nodes by adjusting the topology to the new scenario. This adaptive behavior emerges as the global result of the individual work of nodes, without the intervention of any central entity or the need for global knowledge. Simulations show that this adaptation process makes the system scalable, resilient to attacks, and tolerant to a high transitivity of peers. Simulations are also used to compare this solution with other well-known self-adapting P2P system. From these results it can be concluded that the topologies achieved by DANTE offer better performance.     

An artificial life approach to dense stereo disparity

This article presents an adaptive approach to improving the infection algorithm that we have used to solve the dense stereo matching problem. The algorithm presented here incorporates two different epidemic automata along a single execution of the infection algorithm. The new algorithm attempts to provide a general behavior of guessing the best correspondence between a pair of images. Our aim is to provide a new strategy inspired by evolutionary computation, which combines the behaviors of both automata into a single correspondence problem. The new algorithm will decide which automata will be used based on the transmission of information and mutation, as well as the attributes, texture, and geometry, of the input images. This article gives details about how the rules used in the infection algorithm are coded. Finally, we show experiments with a real stereo pair, as well as with a standard test bed, to show how the infection algorithm works.     

Phosphofructokinase from mantle tissue of Mytilus galloprovincialis. Purification and effects of phosphorylation on the enzymatic activity

Phosphofructokinase purified from mantle tissue of the sea mussel Mytilus galloprovincialis, was phosphorylated "in vitro" by the catalytic subunit of cyclic AMP-dependent protein kinase. The incorporation of phosphate gave rise to an activation of the enzyme by increasing its affinity for fructose-6-phosphate, by decreasing its sensitivity to the inhibition by ATP and by enhancing the effect of allosteric activators (5'-AMP and fructose-2,6-bisphosphate). In addition, the effects of phosphorylation on the catalytic activity are pH-dependent.     

Engineering linear F(ab')2 fragments for efficient production in Escherichia coli and enhanced antiproliferative activity

We developed a novel bivalent antibody fragment, the linear(L-) F(ab')₂, comprising tandem repeats of a heavy chain fragmentV_H–C_H1–V_H–C_H1 cosecreted with a light chain.Functional humanized L-F(ab')₂ directed against p185^HER2 wassecreted from Escherichia coli at high titer (100 mg/l) andpurified to homogeneity. The L-F(ab')₂ binds two equivalentsof antigen with an apparent affinity (K_d = 0.46 nM) that iswithin 3-fold of the corresponding thioether-linked F(ab')₂fragment The N-terminal site binds antigen with an affinity(K_d = 1.2 nM) that is 4-fold greater than that for the C-terminalsite, as shown by the comparison of L-F(ab')₂ variants containinga single functional binding site. L-F(ab')₂ has greater antiproliferativeactivity than the thioether-linked F(ab')₂ against the p185^HER2-overexpressingtumor cell line BT474. Linear and thioether-linked F(ab')₂ havevery similar pharmacokinetic properties in normal mice, andtheir serum permanence times are respectively 7- and 8-foldlonger than the corresponding Fab fragment L-F(ab')₂ offersa facile route to bivalent antibody fragments that are potentiallysuitable for clinical applications, and that may have improvedbiological activity compared with thioether-linked F(ab')₂ fragments.     

Equivalent-optical-waveguide model for the analysis of optical waveguides by means of an asymptotic effective-index method

We present a theoretical method that makes it possible to analyze three-dimensional (3-D) integrated optical waveguides with arbitrary refractive-index profiles. With this method it is easy to obtain effective indexes, propagation constants, and coupling-switching properties of planar and channel optical waveguides. This theoretical approach involves one's modeling the original optical waveguide by means of an equivalent optical waveguide whose effective index is evaluated by the application of a technique that we call the asymptotic effective-index method. The numerical values show good convergence and accuracy for effective indexes, propagation constants, and coupling-switching characteristics. Theoretical and experimental results are given.     

Criticality Analysis for Maintenance Purposes: A Study for Complex In‐service Engineering Assets

The purpose of this paper is to establish a basis for a criticality analysis, considered here as a prerequisite, a first required step to review the current maintenance programs, of complex in‐service engineering assets. Review is understood as a reality check, a testing of whether the current maintenance activities are well aligned to actual business objectives and needs. This paper describes an efficient and rational working process and a model resulting in a hierarchy of assets, based on risk analysis and cost–benefit principles, which will be ranked according to their importance for the business to meet specific goals. Starting from a multicriteria analysis, the proposed model converts relevant criteria impacting equipment criticality into a single score presenting the criticality level. Although detailed implementation of techniques like Root Cause Failure Analysis and Reliability Centered Maintenance will be recommended for further optimization of the maintenance activities, the reasons why criticality analysis deserves the attention of engineers and maintenance and reliability managers are precisely explained here. A case study is presented to help the reader understand the process and to operationalize the model. Copyright © 2015 John Wiley & Sons, Ltd.     

GraftFast Surface Engineering to Improve MOF Nanoparticles Furtiveness

Controlling the outer surface of nanometric metal–organic frameworks (nanoMOFs) and further understanding the in vivo effect of the coated material are crucial for the convenient biomedical applications of MOFs. However, in most studies, the surface modification protocol is often associated with significant toxicity and/or lack of selectivity. As an alternative, how the highly selective and general grafting GraftFast method leads, through a green and simple process, to the successful attachment of multifunctional biopolymers (polyethylene glycol (PEG) and hyaluronic acid) on the external surface of nanoMOFs is reported. In particular, effectively PEGylated iron trimesate MIL‐100(Fe) nanoparticles (NPs) exhibit suitable grafting stability and superior chemical and colloidal stability in different biofluids, while conserving full porosity and allowing the adsorption of bioactive molecules (cosmetic and antitumor agents). Furthermore, the nature of the MOF–PEG interaction is deeply investigated using high‐resolution soft X‐ray spectroscopy. Finally, a cell penetration study using the radio‐labeled antitumor agent gemcitabine monophosphate (³H‐GMP)‐loaded MIL‐100(Fe)@PEG NPs shows reduced macrophage phagocytosis, confirming a significant in vitro PEG furtiveness.