Integration of imitation learning using GAIL and reinforcement learning using task-achievement rewards via probabilistic graphical model

The integration of reinforcement learning (RL) and imitation learning (IL) is an important problem that has long been studied in the field of intelligent robotics. RL optimizes policies to maximize the cumulative reward, whereas IL attempts to extract general knowledge about the trajectories demonstrated by experts, i.e, demonstrators. Because each has its own drawbacks, many methods combining them and compensating for each set of drawbacks have been explored thus far. However, many of these methods are heuristic and do not have a solid theoretical basis. This paper presents a new theory for integrating RL and IL by extending the probabilistic graphical model (PGM) framework for RL, control as inference. We develop a new PGM for RL with multiple types of rewards, called probabilistic graphical model for Markov decision processes with multiple optimality emissions (pMDP-MO). Furthermore, we demonstrate that the integrated learning method of RL and IL can be formulated as a probabilistic inference of policies on pMDP-MO by considering the discriminator in generative adversarial imitation learning (GAIL) as an additional optimality emission. We adapt the GAIL and task-achievement reward to our proposed framework, achieving significantly better performance than policies trained with baseline methods.     

A method for locating fault section by using sequential information at substations

This paper presents a new real-time method for locating fault section at substations. When a fault occurs, considerable information resulting from protective relay operation and circuit-breaker operation is recorded. Using this information an attempt is made to locate the fault section, especially by considering the sequential relationship of the information and by dividing fault areas. Primarily, in this method, the fault area is divided into several sections based on protection area and operation time of protective relays. Next, expecting subsequent operation of backup relays, the “waiting time” for reasoning is given to each divided section. After a lapse of “waiting time,” all the suspected fault sections are extracted and given priority based on the empirical knowledge of experts. A prototype of the expert system was developed for fault section location, which was applied to various complicated fault cases. The effectiveness of the method was proved even in case of multiple faults and no-operation of protective relays the circuit breakers.     

Systematic analysis of neutron yields from thick targets bombarded by heavy ions and protons with moving source model

A simple phenomenological analysis using the moving source model has been performed on the neutron energy spectra produced by bombarding thick targets with high energy heavy ions which have been systematically measured at the Heavy-Ion Medical Accelerator (HIMAC) facility (located in Chiba, Japan) of the National Institute of Radiological Sciences (NIRS). For the bombardment of both heavy ions and protons in the energy region of 100–500 MeV per nucleon, the moving source model incorporating the knock-on process could be generally successful in reproducing the measured neutron spectra within a factor of two margin of accuracy. This phenomenological analytical equation is expressed having several parameters as functions of atomic number Z_p, mass number A_p, energy per nucleon E_p for projectile, and atomic number Z_T, mass number A_T for target. By inputting these basic data for projectile and target into this equation we can easily estimate the secondary neutron energy spectra at an emission angle of 0–90° for bombardment with heavy ions and protons in the aforementioned energy region. This method will be quite useful to estimate the neutron source term in the neutron shielding design of high energy proton and heavy ion accelerators.     

Preparation of oriented gold plate/cellulose nanofiber composite films by using TEMPO-oxidized cellulose nanofiber as a reducing agent

Microsystem Technologies - TEMPO-oxidized cellulose nanofiber (TOCN) has been drawing much attention as a useful material for preparation of transparent and stiff films or substrates. In this work,...     

Role of tungsten in promoting selective reduction of NO with CO over Ir/WO3–SiO2 catalysts

Catalytic hydrogenation of naphthalene to decalin was studied over a carbon-supported rhodium catalyst in supercritical carbon dioxide solvent at 333 K, and the results were compared with those in an organic solvent. cis-, trans-Decalin and tetralin were formed from the beginning of the reaction in supercritical carbon dioxide. Higher concentration of hydrogen in carbon dioxide solvent and on the active site, and also the suppression of desorption of partially hydrogenated tetralin molecules from the active site would be responsible for higher selectivity to cis-decalin in supercritical carbon dioxide than that in an organic solvent.     

Buoyancy engine developed for underwater gliders

A buoyancy engine with a swashplate-type axial piston pump was developed. Its oil extrusion and drawing properties under high hydraulic pressure were evaluated. This buoyancy engine is now installed in an underwater glider that will achieve long-term monitoring of ocean environments up to 2100 m depth in a designated area with lower operational costs. This bidirectionally functioning pump can control the amount of oil in extrusion and draw operations. When drawing oil under high pressure, the hydraulic pump and the electric motor, respectively, act as a hydraulic motor and an electric generator. The generated electric power is absorbed by a damping resistor. The oil-drawing and extrusion properties were measured using a large hyperbaric chamber that is able to produce an almost identical environment to that of actual operations. Results confirmed stable oil extrusion operations up to 21 MPa. Regarding oil-drawing properties, although it was measured only up to 10 MPa in the hyperbaric chamber, it can be inferred that the system can draw the oil and can control the buoyancy precisely up to 21 MPa by replacing the two-way ball valve with an electromagnetic latching solenoid valve.     

Hydrocarbons with a homoconjugated polyene system and their monoepoxy derivatives: Sex attractants of geometrid and noctuid moths distributed in Japan

Although several sex pheromones of the family Geometridae have been characterized, investigations on Japanese species are limited. In order to obtain more information, screening using known sex pheromones and their analogs was carried out. The (Z,Z,Z)-3,6,9-triunsaturated and (Z,Z)-6,9-diunsaturated hydrocarbons with straight C¹⁹-C²¹ chains were synthesized by the Grignard coupling reaction as a key step starting from linolenic and linoleic acids, respectively. Oxidation of the homoconjugated trienes withm-chloroperoxybenzoic acid yielded a 111 mixture of three monoepoxy derivatives that could be separated by silica gel chromatography. The chemical structure of each positional isomer was confirmed using two-dimensional NMR techniques and MS measurements, which enabled characteristic fragment ions from the isomers to be identified. Field tests using lures incorporating only one of the above six hydrocarbons or nine epoxides were carried out in a forest in Tokyo. Consequently, attraction of male moths of 14 geometrid species in addition to four species in another family, the Noctuidae, was observed. It was concluded that hydrocarbons with a homoconjugated polyene system and the monoepoxy derivatives are important components of sex pheromones produced by Japanese lepidopterous insects, particularly the geometrid moths.     

A multi-spectral non-local method for retrieval of boundary layer cloud properties from optical remote sensing data

A multi-spectral non-local (MSN) method is developed for advanced retrieval of boundary layer cloud properties from remote sensing data, as an alternative to the independent pixel approximation (IPA) method. The non-local method uses data at both the target pixel and neighboring pixels to retrieve cloud properties such as pixel-averaged cloud optical thickness and effective droplet radius. Radiance data to be observed from space were simulated by a three-dimensional (3D) radiation model and a stochastic boundary layer cloud model with two-dimensional (horizontal and vertical) variability in cloud liquid water and effective radius. An adiabatic assumption is used for each cloud column to model the geometrical thickness and vertical profiles of cloud liquid water content and effective droplet radius, neglecting drizzle and cloud brokenness for simplicity. The dependence of radiative smoothing and roughening on horizontal scale, optical thickness and single scattering albedo are investigated. Then, retrieval methods using 250-m horizontal resolution data onboard new generation satellites are discussed. The regression model for the MSN method was trained based on datasets from numerical simulations. The training was performed with respect to various domain averages of optical thickness and effective radius, because smoothing and roughening effects are strongly dependent on the two variables. Retrieval accuracy is discussed here with datasets independent of those used in the training, towards assessing the generality of the technique. It is demonstrated that retrieval accuracy of cloud optical thickness, which is often retrieved from single-spectral visible-wavelength data, is improved the most using neighboring pixel data and secondly using multi-spectral data, and ideally with both. When the IPA retrieval method is applied to optical thickness and effective radius, the root-mean-square relative errors can be 15-90%, depending on solar and view directions. In contrast, the MSN method has errors of 4-10%, which is smaller than IPA by a factor of 2-10. It is also suggested that the accuracy of the MSN method is insensitive to some assumptions in the inhomogeneous cloud input data used to train the regression model.     

Structure and properties of biaxially stretched poly(ethylene terephthalate) sheets

Huge numbers of PET (poly[ethylene terephthalate]) bottles are produced in the world. Especially in Japan, the number of hot-fillable PET bottles used is extremely large and is still increasing. This type of bottle is generally manufactured by the heat-set method using hot molds after stretch-blow molding. Herein, we examined how the PET sheet stretching condition affects the PET heat-shrinkage behavior at 85°C, which is the hot-filling temperature. Sheets stretched at a higher temperature and higher speed had higher thermal stability for a wider range of draw ratios. This is because those sheets have a higher crystallinity and relaxed amorphous regions. The higher stretch speed gives the sheet a higher crystallinity with self heat generation during rapid deformation. A higher stretch temperature makes the molecular segments relaxed.     

Catalytic Performance of Aged Rh/CeO2–ZrO2 for NO–C3H6–O2 Reaction Under a Stoichiometric Condition

The activity of Rh/CeO₂ for NO reduction by C₃H₆ was gradually deceased by mixing with ZrO₂ until 68 mol%. Rh supported on CeO₂–ZrO₂ with higher OSC was found to show lower catalytic activity. High OSC of CeO₂–ZrO₂ would probably stabilize the surface of Rh in oxidized state, resulting in low activity and low efficiency of C₃H₆ utilization for NO reduction. In situ FT-IR spectroscopy suggested that mononitrosyl species such as Rh(NO)^δ? and Rh(NO)^δ+ are reaction intermediates in the NO–C₃H₆–O₂ reaction over Rh/CeO₂–ZrO₂ catalysts.