Integration of imitation learning using GAIL and reinforcement learning using task-achievement rewards via probabilistic graphical model

The integration of reinforcement learning (RL) and imitation learning (IL) is an important problem that has long been studied in the field of intelligent robotics. RL optimizes policies to maximize the cumulative reward, whereas IL attempts to extract general knowledge about the trajectories demonstrated by experts, i.e, demonstrators. Because each has its own drawbacks, many methods combining them and compensating for each set of drawbacks have been explored thus far. However, many of these methods are heuristic and do not have a solid theoretical basis. This paper presents a new theory for integrating RL and IL by extending the probabilistic graphical model (PGM) framework for RL, control as inference. We develop a new PGM for RL with multiple types of rewards, called probabilistic graphical model for Markov decision processes with multiple optimality emissions (pMDP-MO). Furthermore, we demonstrate that the integrated learning method of RL and IL can be formulated as a probabilistic inference of policies on pMDP-MO by considering the discriminator in generative adversarial imitation learning (GAIL) as an additional optimality emission. We adapt the GAIL and task-achievement reward to our proposed framework, achieving significantly better performance than policies trained with baseline methods.     

A 300-MHz 25-/spl mu/A/Mb-leakage on-chip SRAM module featuring process-variation immunity and low-leakage-active mode for mobile-phone application processor

An on-chip 1-Mb SRAM suitable for embedding in the application processor used in mobile cellular phones was developed. This SRAM supports three operating modes - high-speed active mode, low-leakage low-speed active mode, and standby mode - and uses a subdivisional power-line control (SPC) scheme. The combination of three operating modes and the SPC scheme realizes low-power operation under actual usage conditions. It operates at 300 MHz, with leakage of 25 /spl mu/A/Mb in standby mode, and 50 /spl mu/A/Mb at the low-leakage active mode. This SRAM also uses a self-bias write scheme that decreases of minimum operating voltage by about 100 mV.     

Amyloid beta-protein deposition in the leptomeninges and cerebral cortex

To further investigate the process of amyloid beta-protein (Abeta) deposition, we determined, using sensitive enzyme immunoassays, the levels of Abeta40 and Abeta42 (Abetas) in the soluble and insoluble fractions of the leptomeninges (containing arachnoid mater and leptomeningeal vessels) and cerebral cortices from elderly control subjects showing various stages of Abeta deposition and from patients affected by Alzheimer's disease (AD). In both locations, insoluble Abeta levels were higher by orders of magnitude than soluble Abeta levels. Soluble Abeta levels in cortices were much lower than those in leptomeninges. In insoluble Abeta in the cortex, Abeta42 was by far the predominant species, and Abeta42 in AD cortices was characterized by the highest degree of modifications in the amino terminus. In contrast, this Abeta42 predominance was not observed in insoluble Abeta in the leptomeninges, which were found to be able to accumulate Abetas to an extent similar to that in the cortex, on a weight basis. The levels of insoluble Abeta in the leptomeninges or cortex generally correlated with the degree of cerebral amyloid angiopathy or the abundance of senile plaque, respectively. However, the presence of plaque-free cortical samples showing significant levels of insoluble Abeta42 suggests that biochemically detectable Abeta accumulation precedes immunocytochemically detectable Abeta deposition in the cortex.     

Ureteral obstruction enhances eicosanoid production in cortical and medullary tubules of rat kidneys

We examined prostaglandin (PG) E2, 6-keto PGF1alpha, and thromboxane B2 (TxB2) production in cortical and medullary tubules from sham-operated control (SOC) rats and rats with bilateral ureteral obstruction (BUO) of 24 h duration. In SOC rats medullary tubules produced significantly greater amounts of the three eicosanoids than cortical tubules. Again, the production of PGE2, 6-keto PGF1alpha, and TxB2 by cortical and medullary tubules was significantly greater in BUO rats than in SOC rats. To elucidate the mechanisms involved, we examined the activity of phospholipase A2 (PLA2) reactive against phosphatidylcholine or phosphatidylethanolamine (PE), the activity of phospholipase C (PLC), and the levels of cyclooxygenase (COX) in cortical and medullary tubules from SOC and BUO rats. In SOC rats the activity of phosphatidylcholine-PLA2 and PE-PLA2, the activity of PLC, and the mass of COX were significantly greater in medullary tubules than in cortical tubules. On the other hand, the activity of PLC in membranes of cortical tubules and the activity of PE-PLA2 and PLC in membranes of medullary tubules, which were in active location, were significantly greater in BUO rats than in SOC rats. COX levels were also significantly greater in cortical and medullary tubules of BUO rats than in those of SOC rats. Thus, we indicate that medullary tubules from SOC rats have greater production of eicosanoids through increased activity of the PLA2 and PLC-COX pathway than cortical tubules from the same group of rats. Again, in rats with BUO, the tubular eicosanoid production may be enhanced via activation of the PLC-COX pathway in cortical tubules or through activation of the PE-PLA2 and PLC-COX pathway in medullary tubules. The enhanced production of tubular eicosanoids observed in rats with BUO may affect tubular function, particularly sodium and water reabsorption.     

The molecular interaction of Fas and FAP-1. A tripeptide blocker of human Fas interaction with FAP-1 promotes Fas-induced apoptosis

Fas (APO-1/CD95), which is a member of the tumor necrosis factor receptor superfamily, is a cell surface receptor that induces apoptosis. A protein tyrosine phosphatase, Fas-associated phosphatase-1 (FAP-1), that was previously identified as a Fas binding protein interacts with the C-terminal 15 amino acids of the regulatory domain of the Fas receptor. To identify the minimal region of the Fas C-terminal necessary for binding to FAP-1, we employed an in vitro inhibition assay of Fas/FAP-1 binding using a series of synthetic peptides as well as a screen of random peptide libraries by the yeast two-hybrid system. The results showed that the C-terminal three amino acids (SLV) of human Fas were necessary and sufficient for its interaction with the third PDZ (GLGF) domain of FAP-1. Furthermore, the direct cytoplasmic microinjection of this tripeptide (Ac-SLV) resulted in the induction of Fas-mediated apoptosis in a colon cancer cell line that expresses both Fas and FAP-1. Since t(S/T)X(V/L/I) motifs in the C termini of several other receptors have been shown to interact with PDZ domain in signal transducing molecules, this may represent a general motif for protein-protein interactions with important biological functions.     

A method for locating fault section by using sequential information at substations

This paper presents a new real-time method for locating fault section at substations. When a fault occurs, considerable information resulting from protective relay operation and circuit-breaker operation is recorded. Using this information an attempt is made to locate the fault section, especially by considering the sequential relationship of the information and by dividing fault areas. Primarily, in this method, the fault area is divided into several sections based on protection area and operation time of protective relays. Next, expecting subsequent operation of backup relays, the “waiting time” for reasoning is given to each divided section. After a lapse of “waiting time,” all the suspected fault sections are extracted and given priority based on the empirical knowledge of experts. A prototype of the expert system was developed for fault section location, which was applied to various complicated fault cases. The effectiveness of the method was proved even in case of multiple faults and no-operation of protective relays the circuit breakers.     

Performance of compression/absorption hybrid refrigeration cycle with propane/mineral oil combination

This paper discusses the feasibility of a vapor compression/absorption hybrid refrigeration cycle for energy saving and utilization of waste heat. The cycle employs propane as a natural refrigerant and a refrigeration oil as an absorbent. A prototype of the cycle is constructed, in which a compressor and an absorption unit are combined in series. The performance of the cycle is examined both theoretically and experimentally. Although the solubility of the propane with the oil is not enough as a working pair in the absorption unit, the theoretical calculation shows that the hybrid cycle has a potential to achieve a higher performance in comparison with a simple vapor compression cycle by using the waste heat. In the experiment, the prototype cycle is operated successfully and it is found that an improvement of an absorber is necessary to achieve the good performance close to the theoretical one. The application of an AHE (absorber heat exchanger) can reduce the heat input to a generator. Further examinations on some other combinations of refrigerant/refrigeration oil and additives are desirable.     

Systematic analysis of neutron yields from thick targets bombarded by heavy ions and protons with moving source model

A simple phenomenological analysis using the moving source model has been performed on the neutron energy spectra produced by bombarding thick targets with high energy heavy ions which have been systematically measured at the Heavy-Ion Medical Accelerator (HIMAC) facility (located in Chiba, Japan) of the National Institute of Radiological Sciences (NIRS). For the bombardment of both heavy ions and protons in the energy region of 100–500 MeV per nucleon, the moving source model incorporating the knock-on process could be generally successful in reproducing the measured neutron spectra within a factor of two margin of accuracy. This phenomenological analytical equation is expressed having several parameters as functions of atomic number Z_p, mass number A_p, energy per nucleon E_p for projectile, and atomic number Z_T, mass number A_T for target. By inputting these basic data for projectile and target into this equation we can easily estimate the secondary neutron energy spectra at an emission angle of 0–90° for bombardment with heavy ions and protons in the aforementioned energy region. This method will be quite useful to estimate the neutron source term in the neutron shielding design of high energy proton and heavy ion accelerators.