MOSAIC for multiple-reward environments

Reinforcement learning (RL) can provide a basic framework for autonomous robots to learn to control and maximize future cumulative rewards in complex environments. To achieve high performance, RL controllers must consider the complex external dynamics for movements and task (reward function) and optimize control commands. For example, a robot playing tennis and squash needs to cope with the different dynamics of a tennis or squash racket and such dynamic environmental factors as the wind. In addition, this robot has to tailor its tactics simultaneously under the rules of either game. This double complexity of the external dynamics and reward function sometimes becomes more complex when both the multiple dynamics and multiple reward functions switch implicitly, as in the situation of a real (multi-agent) game of tennis where one player cannot observe the intention of her opponents or her partner. The robot must consider its opponent's and its partner's unobservable behavioral goals (reward function). In this article, we address how an RL agent should be designed to handle such double complexity of dynamics and reward. We have previously proposed modular selection and identification for control (MOSAIC) to cope with nonstationary dynamics where appropriate controllers are selected and learned among many candidates based on the error of its paired dynamics predictor: the forward model. Here we extend this framework for RL and propose MOSAIC-MR architecture. It resembles MOSAIC in spirit and selects and learns an appropriate RL controller based on the RL controller's TD error using the errors of the dynamics (the forward model) and the reward predictors. Furthermore, unlike other MOSAIC variants for RL, RL controllers are not a priori paired with the fixed predictors of dynamics and rewards. The simulation results demonstrate that MOSAIC-MR outperforms other counterparts because of this flexible association ability among RL controllers, forward models, and reward predictors.     

Thermoelectric properties of Bi and Cu co-doped Ca3Co2O6single crystals

Single crystals of Bi and Cu-doped Ca₃Co₂O₆were synthesized in a molten K₂CO₃flux. Using an obtained single crystal of (Ca_0.985(5)Bi_0.015(5))₃(Co_0.990(3)Cu_0.010(3))₂O₆elongated to the c-axis direction of the crystal structure, the electric resistivity (ρ) and Seebeck coefficient (S) were measured from room temperature to over 1000 K in air. The single crystal showed p-type semiconducting behavior with ρ values of 1.8 Ω cm at 303 K and 0.017 Ω cm at 1000 K. The S values were +254 μ VK^{− 1} at 325 K, +360 μ VK^{− 1} at 420 K, and +214 μ VK^{− 1} at 1000 K. The power factor (S ² ρ ^{− 1}) increased with an increase of temperature and attained 2.70 × 10^{− 4} Wm^{− 1}K^{− 2} at 1000 K.     

Rolling and swaying motion estimation for a mobile robot by using omnidirectional optical flows

Described here is a method for estimating rolling and swaying motions of a mobile robot using optical flow. We have proposed an image sensor with a hyperboloidal mirror for the vision-based navigation of a mobile robot. Its name is HyperOmni Vision. The radial component of optical flow in HyperOmni Vision has a periodic characteristic. The circumferential component of optical flow has a symmetric characteristic. The proposed method makes use of these characteristic to estimate robustly the rolling and swaying motion of the mobile robot. Correspondence to: Y. Yagi e-mail: y-yagi@sys.es.osaka-u.ac.jp     

Quantum Lattice-Gas Automata Simulation of Electronic Wave Propagation in Nanostructures

We report the two- and three-dimensional quantum lattice-gas automata simulation for one-particle electronic wave propagation in nanostructures. The transmission coefficient of the electronic wave through the two-dimensional quantum point contact is investigated taking account of the surface roughness of the confinement wall. It is demonstrated that the electron transmission is significantly affected by the surface roughness pattern even if the same roughness parameter is assumed. We also perform the three-dimensional simulation, and the wave propagation in the structure like an ultrathin-body MOSFET is visualized.     

Advances in nature-guided materials processing

The center of excellence (COE) titled ‘The Creation of Nature-Guided Materials Processing’ has been established in Nagoya University as the 21st Century COE Program supported by Ministry of Education, Culture, Sports, Science and Technology. In the Nature COE, various activities on the education and research are being performed through learning the laws of nature, namely, methods of attaining ‘appearance of the maximum function under the minimum substance and energy consumption’, which the nature and living organisms have acquired through their evolution in long period. Together with such educational programs for PhD students as research incentive, oversea training, and external evaluation programs, an ‘Open-Cluster Program’ was originated for promoting researches proposed by research groups consisting of young researchers in and out of the university and also for fostering them.

The researches are being advanced on materials used for living bodies, mimicking structures which nature or living bodies are forming, and producing materials by mimicking processes to form the structures observed in the nature or the living bodies. In this COE, these researches are conducted by four groups to extend the processes observed in the natural world to a new type of processing, that is, thoroughly examined and rationalized by plunging a scalpel of engineering and to establish a new academic field of materials science and engineering.     

Development and Use of a Kinetical and Real-Time Monitoring System to Analyze the Replication of Hepatitis C Virus

In microbiological research, it is important to understand the time course of each step in a pathogen’s lifecycle and changes in the host cell environment induced by infection. This study is the first to develop a real-time monitoring system that kinetically detects luminescence reporter activity over time without sampling cells or culture supernatants for analyzing the virus replication. Subgenomic replicon experiments with hepatitis C virus (HCV) showed that transient translation and genome replication can be detected separately, with the first peak of translation observed at 3–4 h and replication beginning around 20 h after viral RNA introduction into cells. From the bioluminescence data set measured every 30 min (48 measurements per day), the initial rates of translation and replication were calculated, and their capacity levels were expressed as the sums of the measured signals in each process, which correspond to the areas on the kinetics graphs. The comparison of various HuH-7-derived cell lines showed that the bioluminescence profile differs among cell lines, suggesting that both translation and replication capacities potentially influence differences in HCV susceptibility. The effects of RNA mutations within the 5′ UTR of the replicon on viral translation and replication were further analyzed in the system developed, confirming that mutations to the miR-122 binding sites primarily reduce replication activity rather than translation. The newly developed real-time monitoring system should be applied to the studies of various viruses and contribute to the analysis of transitions and progression of each process of their life cycle.     

Detection and Quantitation of Left-to-Right Shunts from Radionuclide Angiocardiography Using the Homomorphic Deconvolution Technique

The homomorphic filtering method is described for the detection and quantitation of left-to-right shunts from radionuclide angiocardiography. First, the method is investigated theoreticatly and numerically using a realistic model with systemic recirculation. It is demonstrated that even in a noisy situation the shunt flow fraction can be relatively well estimated, if only the characteristics of the fiters used are suitable. Second, the method is applied to real data, i.e., the pulmonary time-activity curves from radionucUde anglocardiography. The presence or absence of a left-to-right shunt is determined using the ratio AT/MTT. The pulmonary-to-systemic flow ratios calculated from the areas of the two decomposed curves are compared with those from oximetry at cardiac catheterization. Good agreement between the Qp/Qs ratios calculated by oximetry and radionucUide angiocardiography is obtained.     

Downregulation of miR-122-5p Activates Glycolysis via PKM2 in Kupffer Cells of Rat and Mouse Models of Non-Alcoholic Steatohepatitis

Non-alcoholic steatohepatitis (NASH) has pathological characteristics similar to those of alcoholic hepatitis, despite the absence of a drinking history. The greatest threat associated with NASH is its progression to cirrhosis and hepatocellular carcinoma. The pathophysiology of NASH is not fully understood to date. In this study, we investigated the pathophysiology of NASH from the perspective of glycolysis and the Warburg effect, with a particular focus on microRNA regulation in liver-specific macrophages, also known as Kupffer cells. We established NASH rat and mouse models and evaluated various parameters including the liver-to-body weight ratio, blood indexes, and histopathology. A quantitative phosphoproteomic analysis of the NASH rat model livers revealed the activation of glycolysis. Western blotting and immunohistochemistry results indicated that the expression of pyruvate kinase muscle 2 (PKM2), a rate-limiting enzyme of glycolysis, was upregulated in the liver tissues of both NASH models. Moreover, increases in PKM2 and p-PKM2 were observed in the early phase of NASH. These observations were partially induced by the downregulation of microRNA122-5p (miR-122-5p) and occurred particularly in the Kupffer cells. Our results suggest that the activation of glycolysis in Kupffer cells during NASH was partially induced by the upregulation of PKM2 via miR-122-5p suppression.     

One-step route to a hybrid TiO2/TixW1?xN nanocomposite by in situ selective carbothermal nitridation

Metal oxide/nitride nanocomposites have many existing and potential applications, e.g. in energy conversion or ammonia synthesis. Here, a hybrid oxide/nitride nanocomposite (anatase/Ti_xW_1−xN) was synthesized by an ammonia-free sol–gel route. Synchrotron x-ray diffraction, complemented with electron microscopy and thermogravimetric analysis, was used to study the structure, composition and mechanism of formation of the nanocomposite. The nanocomposite contained nanoparticles (<5 nm diameter) of two highly intermixed phases. This was found to arise from controlled nucleation and growth of a single oxide intermediate from the gel precursor, followed by phase separation and in situ selective carbothermal nitridation. Depending on the preparation conditions, the composition varied from anatase/Ti_xW_1−xN at low W content to an isostructural mixture of Ti-rich and W-rich Ti_xW_1−xN at high W content. In situ selective carbothermal nitridation offers a facile route to the synthesis of nitride-oxide nanocomposites. This conceptually new approach is a significant advance from previous methods, which generally require ammonolysis of a pre-synthesized oxide.     

Te concentration dependent photoemission and inverse-photoemission study of FeSe1?xTex

We have characterized the electronic structure of FeSe_1−xTe_x for various x values using soft x-ray photoemission spectroscopy (SXPES), high-resolution photoemission spectroscopy (HRPES) and inverse photoemission spectroscopy (IPES). The SXPES valence band spectral shape shows that the 2 eV feature in FeSe, which was ascribed to the lower Hubbard band in previous theoretical studies, becomes less prominent with increasing x. HRPES exhibits systematic x dependence of the structure near the Fermi level (E_F): its splitting near E_F and filling of the pseudogap in FeSe. IPES shows two features, near E_F and approximately 6 eV above E_F; the former may be related to the Fe 3d states hybridized with chalcogenide p states, while the latter may consist of plane-wave-like and Se d components. In the incident electron energy dependence of IPES, the density of states near E_F for FeSe and FeTe has the Fano lineshape characteristic of resonant behavior. These compounds exhibit different resonance profiles, which may reflect the differences in their electronic structures. By combining the PES and IPES data the on-site Coulomb energy was estimated at 3.5 eV for FeSe.