Integration of imitation learning using GAIL and reinforcement learning using task-achievement rewards via probabilistic graphical model

The integration of reinforcement learning (RL) and imitation learning (IL) is an important problem that has long been studied in the field of intelligent robotics. RL optimizes policies to maximize the cumulative reward, whereas IL attempts to extract general knowledge about the trajectories demonstrated by experts, i.e, demonstrators. Because each has its own drawbacks, many methods combining them and compensating for each set of drawbacks have been explored thus far. However, many of these methods are heuristic and do not have a solid theoretical basis. This paper presents a new theory for integrating RL and IL by extending the probabilistic graphical model (PGM) framework for RL, control as inference. We develop a new PGM for RL with multiple types of rewards, called probabilistic graphical model for Markov decision processes with multiple optimality emissions (pMDP-MO). Furthermore, we demonstrate that the integrated learning method of RL and IL can be formulated as a probabilistic inference of policies on pMDP-MO by considering the discriminator in generative adversarial imitation learning (GAIL) as an additional optimality emission. We adapt the GAIL and task-achievement reward to our proposed framework, achieving significantly better performance than policies trained with baseline methods.     

A method for locating fault section by using sequential information at substations

This paper presents a new real-time method for locating fault section at substations. When a fault occurs, considerable information resulting from protective relay operation and circuit-breaker operation is recorded. Using this information an attempt is made to locate the fault section, especially by considering the sequential relationship of the information and by dividing fault areas. Primarily, in this method, the fault area is divided into several sections based on protection area and operation time of protective relays. Next, expecting subsequent operation of backup relays, the “waiting time” for reasoning is given to each divided section. After a lapse of “waiting time,” all the suspected fault sections are extracted and given priority based on the empirical knowledge of experts. A prototype of the expert system was developed for fault section location, which was applied to various complicated fault cases. The effectiveness of the method was proved even in case of multiple faults and no-operation of protective relays the circuit breakers.     

Systematic analysis of neutron yields from thick targets bombarded by heavy ions and protons with moving source model

A simple phenomenological analysis using the moving source model has been performed on the neutron energy spectra produced by bombarding thick targets with high energy heavy ions which have been systematically measured at the Heavy-Ion Medical Accelerator (HIMAC) facility (located in Chiba, Japan) of the National Institute of Radiological Sciences (NIRS). For the bombardment of both heavy ions and protons in the energy region of 100–500 MeV per nucleon, the moving source model incorporating the knock-on process could be generally successful in reproducing the measured neutron spectra within a factor of two margin of accuracy. This phenomenological analytical equation is expressed having several parameters as functions of atomic number Z_p, mass number A_p, energy per nucleon E_p for projectile, and atomic number Z_T, mass number A_T for target. By inputting these basic data for projectile and target into this equation we can easily estimate the secondary neutron energy spectra at an emission angle of 0–90° for bombardment with heavy ions and protons in the aforementioned energy region. This method will be quite useful to estimate the neutron source term in the neutron shielding design of high energy proton and heavy ion accelerators.     

Subacute toxicity of piperonyl butoxide in ICR mice

Piperonyl butoxide, alpha-[2-(2-butoxyethoxy)ethoxy]-4,5-methylenedioxy-2-propyltol uene, is a pesticide synergist. ICR mice of both sexes were maintained on diet containing 0, 0.1, 0.3 or 0.9% of piperonyl butoxide for 20 days. At the end of the experimental period, they were necropsied. Selected organs were weighed and serum chemistries were analyzed. In male and female mice of the 0.9% group, body weight, kidney and spleen weight were depressed in comparison to those of control group. Liver weight of the 0.3 and 0.9% group of both sexes were significantly higher than those of control group. Mice of the 0.9% group of both sexes had increased serum levels of cholesterol, total protein, gamma-glutamyl transpeptidase. Histological examination of livers from mice of the 0.9% group by light microscopy showed enlarged hepatocytes, anisonucleosis and single cell necrosis. The results indicated that subacute toxicity of piperonyl butoxide in ICR mice was directed primarily at liver.     

Preparation of oriented gold plate/cellulose nanofiber composite films by using TEMPO-oxidized cellulose nanofiber as a reducing agent

Microsystem Technologies - TEMPO-oxidized cellulose nanofiber (TOCN) has been drawing much attention as a useful material for preparation of transparent and stiff films or substrates. In this work,...     

Role of tungsten in promoting selective reduction of NO with CO over Ir/WO3–SiO2 catalysts

Catalytic hydrogenation of naphthalene to decalin was studied over a carbon-supported rhodium catalyst in supercritical carbon dioxide solvent at 333 K, and the results were compared with those in an organic solvent. cis-, trans-Decalin and tetralin were formed from the beginning of the reaction in supercritical carbon dioxide. Higher concentration of hydrogen in carbon dioxide solvent and on the active site, and also the suppression of desorption of partially hydrogenated tetralin molecules from the active site would be responsible for higher selectivity to cis-decalin in supercritical carbon dioxide than that in an organic solvent.     

Creation of the active site for methanol synthesis on a Cu/SiO2 catalyst

The effect of ZnO/SiO₂ in a physical mixture of Cu/SiO₂ and ZnO/SiO₂ on methanol synthesis from CO₂ and H₂ was studied to clarify the role of ZnO in Cu/ZnO-based catalysts. An active Cu/SiO₂ was prepared by the following procedure: the Cu/SiO₂ and ZnO/SiO₂ catalysts with a different SiO₂ particle size were mixed and reduced with H₂ at 523-723 K, and the Cu/SiO₂ was then separated from the mixture using a sieve. The methanol synthesis activity of the Cu/SiO₂ catalyst increased with the reduction temperature and was in fairly good agreement with that previously obtained for the physical mixture of Cu/SiO₂ and ZnO/SiO₂. These results indicated that the active site for methanol synthesis was created on the Cu/SiO₂ upon reduction of the physical mixture with H₂. It was also found that ZnO itself had no promotional effect on the methanol synthesis activity except for the role of ZnO to create the active site. The active site created on the Cu/SiO₂ catalyst was found not to promote the formation of formate from CO₂ and H₂ on the Cu surface based on in situ FT-IR measurements. A special formate species unstable at 523 K with an OCO asymmetric peak at ~1585 cm^-1 was considered to be adsorbed on the active site. This revised version was published online in July 2006 with corrections to the Cover Date.     

Buoyancy engine developed for underwater gliders

A buoyancy engine with a swashplate-type axial piston pump was developed. Its oil extrusion and drawing properties under high hydraulic pressure were evaluated. This buoyancy engine is now installed in an underwater glider that will achieve long-term monitoring of ocean environments up to 2100 m depth in a designated area with lower operational costs. This bidirectionally functioning pump can control the amount of oil in extrusion and draw operations. When drawing oil under high pressure, the hydraulic pump and the electric motor, respectively, act as a hydraulic motor and an electric generator. The generated electric power is absorbed by a damping resistor. The oil-drawing and extrusion properties were measured using a large hyperbaric chamber that is able to produce an almost identical environment to that of actual operations. Results confirmed stable oil extrusion operations up to 21 MPa. Regarding oil-drawing properties, although it was measured only up to 10 MPa in the hyperbaric chamber, it can be inferred that the system can draw the oil and can control the buoyancy precisely up to 21 MPa by replacing the two-way ball valve with an electromagnetic latching solenoid valve.     

Hydrocarbons with a homoconjugated polyene system and their monoepoxy derivatives: Sex attractants of geometrid and noctuid moths distributed in Japan

Although several sex pheromones of the family Geometridae have been characterized, investigations on Japanese species are limited. In order to obtain more information, screening using known sex pheromones and their analogs was carried out. The (Z,Z,Z)-3,6,9-triunsaturated and (Z,Z)-6,9-diunsaturated hydrocarbons with straight C¹⁹-C²¹ chains were synthesized by the Grignard coupling reaction as a key step starting from linolenic and linoleic acids, respectively. Oxidation of the homoconjugated trienes withm-chloroperoxybenzoic acid yielded a 111 mixture of three monoepoxy derivatives that could be separated by silica gel chromatography. The chemical structure of each positional isomer was confirmed using two-dimensional NMR techniques and MS measurements, which enabled characteristic fragment ions from the isomers to be identified. Field tests using lures incorporating only one of the above six hydrocarbons or nine epoxides were carried out in a forest in Tokyo. Consequently, attraction of male moths of 14 geometrid species in addition to four species in another family, the Noctuidae, was observed. It was concluded that hydrocarbons with a homoconjugated polyene system and the monoepoxy derivatives are important components of sex pheromones produced by Japanese lepidopterous insects, particularly the geometrid moths.     

A multi-spectral non-local method for retrieval of boundary layer cloud properties from optical remote sensing data

A multi-spectral non-local (MSN) method is developed for advanced retrieval of boundary layer cloud properties from remote sensing data, as an alternative to the independent pixel approximation (IPA) method. The non-local method uses data at both the target pixel and neighboring pixels to retrieve cloud properties such as pixel-averaged cloud optical thickness and effective droplet radius. Radiance data to be observed from space were simulated by a three-dimensional (3D) radiation model and a stochastic boundary layer cloud model with two-dimensional (horizontal and vertical) variability in cloud liquid water and effective radius. An adiabatic assumption is used for each cloud column to model the geometrical thickness and vertical profiles of cloud liquid water content and effective droplet radius, neglecting drizzle and cloud brokenness for simplicity. The dependence of radiative smoothing and roughening on horizontal scale, optical thickness and single scattering albedo are investigated. Then, retrieval methods using 250-m horizontal resolution data onboard new generation satellites are discussed. The regression model for the MSN method was trained based on datasets from numerical simulations. The training was performed with respect to various domain averages of optical thickness and effective radius, because smoothing and roughening effects are strongly dependent on the two variables. Retrieval accuracy is discussed here with datasets independent of those used in the training, towards assessing the generality of the technique. It is demonstrated that retrieval accuracy of cloud optical thickness, which is often retrieved from single-spectral visible-wavelength data, is improved the most using neighboring pixel data and secondly using multi-spectral data, and ideally with both. When the IPA retrieval method is applied to optical thickness and effective radius, the root-mean-square relative errors can be 15-90%, depending on solar and view directions. In contrast, the MSN method has errors of 4-10%, which is smaller than IPA by a factor of 2-10. It is also suggested that the accuracy of the MSN method is insensitive to some assumptions in the inhomogeneous cloud input data used to train the regression model.