Integration of imitation learning using GAIL and reinforcement learning using task-achievement rewards via probabilistic graphical model

The integration of reinforcement learning (RL) and imitation learning (IL) is an important problem that has long been studied in the field of intelligent robotics. RL optimizes policies to maximize the cumulative reward, whereas IL attempts to extract general knowledge about the trajectories demonstrated by experts, i.e, demonstrators. Because each has its own drawbacks, many methods combining them and compensating for each set of drawbacks have been explored thus far. However, many of these methods are heuristic and do not have a solid theoretical basis. This paper presents a new theory for integrating RL and IL by extending the probabilistic graphical model (PGM) framework for RL, control as inference. We develop a new PGM for RL with multiple types of rewards, called probabilistic graphical model for Markov decision processes with multiple optimality emissions (pMDP-MO). Furthermore, we demonstrate that the integrated learning method of RL and IL can be formulated as a probabilistic inference of policies on pMDP-MO by considering the discriminator in generative adversarial imitation learning (GAIL) as an additional optimality emission. We adapt the GAIL and task-achievement reward to our proposed framework, achieving significantly better performance than policies trained with baseline methods.     

A method for locating fault section by using sequential information at substations

This paper presents a new real-time method for locating fault section at substations. When a fault occurs, considerable information resulting from protective relay operation and circuit-breaker operation is recorded. Using this information an attempt is made to locate the fault section, especially by considering the sequential relationship of the information and by dividing fault areas. Primarily, in this method, the fault area is divided into several sections based on protection area and operation time of protective relays. Next, expecting subsequent operation of backup relays, the “waiting time” for reasoning is given to each divided section. After a lapse of “waiting time,” all the suspected fault sections are extracted and given priority based on the empirical knowledge of experts. A prototype of the expert system was developed for fault section location, which was applied to various complicated fault cases. The effectiveness of the method was proved even in case of multiple faults and no-operation of protective relays the circuit breakers.     

Systematic analysis of neutron yields from thick targets bombarded by heavy ions and protons with moving source model

A simple phenomenological analysis using the moving source model has been performed on the neutron energy spectra produced by bombarding thick targets with high energy heavy ions which have been systematically measured at the Heavy-Ion Medical Accelerator (HIMAC) facility (located in Chiba, Japan) of the National Institute of Radiological Sciences (NIRS). For the bombardment of both heavy ions and protons in the energy region of 100–500 MeV per nucleon, the moving source model incorporating the knock-on process could be generally successful in reproducing the measured neutron spectra within a factor of two margin of accuracy. This phenomenological analytical equation is expressed having several parameters as functions of atomic number Z_p, mass number A_p, energy per nucleon E_p for projectile, and atomic number Z_T, mass number A_T for target. By inputting these basic data for projectile and target into this equation we can easily estimate the secondary neutron energy spectra at an emission angle of 0–90° for bombardment with heavy ions and protons in the aforementioned energy region. This method will be quite useful to estimate the neutron source term in the neutron shielding design of high energy proton and heavy ion accelerators.     

Preparation of oriented gold plate/cellulose nanofiber composite films by using TEMPO-oxidized cellulose nanofiber as a reducing agent

Microsystem Technologies - TEMPO-oxidized cellulose nanofiber (TOCN) has been drawing much attention as a useful material for preparation of transparent and stiff films or substrates. In this work,...     

A secure cookie scheme

Cookies are the primary means for web applications to authenticate HTTP requests and to maintain client states. Many web applications (such as those for electronic commerce) demand a secure cookie scheme. Such a scheme needs to provide the following four services: authentication, confidentiality, integrity, and anti-replay. Several secure cookie schemes have been proposed in previous literature; however, none of them are completely satisfactory. In this paper, we propose a secure cookie scheme that is effective, efficient, and easy to deploy. In terms of effectiveness, our scheme provides all of the above four security services. In terms of efficiency, our scheme does not involve any database lookup or public key cryptography. In terms of deployability, our scheme can be easily deployed on existing web services, and it does not require any change to the Internet cookie specification. We implemented our secure cookie scheme using PHP and conducted experiments. The experimental results show that our scheme is very efficient on both the client side and the server side.A notable adoption of our scheme in industry is that our cookie scheme has been used by Wordpress since version 2.4. Wordpress is a widely used open source content management system.     

Role of tungsten in promoting selective reduction of NO with CO over Ir/WO3–SiO2 catalysts

Catalytic hydrogenation of naphthalene to decalin was studied over a carbon-supported rhodium catalyst in supercritical carbon dioxide solvent at 333 K, and the results were compared with those in an organic solvent. cis-, trans-Decalin and tetralin were formed from the beginning of the reaction in supercritical carbon dioxide. Higher concentration of hydrogen in carbon dioxide solvent and on the active site, and also the suppression of desorption of partially hydrogenated tetralin molecules from the active site would be responsible for higher selectivity to cis-decalin in supercritical carbon dioxide than that in an organic solvent.     

Home Automation Using Augmented Reality (HAAR)

Wireless Personal Communications - The development of Smart Home Controllers has seen rapid growth in recent years, especially for smart devices, that can utilize the Internet of Things (IoT)....     

Hop chains: Secure routing and the establishment of distinct identities

We present a secure routing protocol that is immune to Sybil attacks and that can tolerate collusion of Byzantine routers. It can tolerate either initial collusion of Byzantine routers or runtime collusion of non-adjacent Byzantine routers, both in the absence of runtime collusion between adjacent routers. For these settings, the calculated distance from a destination to a node is not smaller than the actual shortest distance from the destination to the node. The protocol can also simultaneously tolerate initial collusion of Byzantine routers and runtime collusion of adjacent Byzantine routers but in the absence of runtime collusion between non-adjacent routers. For this setting, it guarantees a bound on the difference between the calculated distance and the actual shortest distance. The bound depends on the number of Byzantine routers on a path. The protocol makes very weak timing assumptions and requires synchronization only between neighbors or second neighbors. We propose to use this protocol for secure localization of routers using hop-count distances, which can be then used as a proof of identity of nodes.     

Buoyancy engine developed for underwater gliders

A buoyancy engine with a swashplate-type axial piston pump was developed. Its oil extrusion and drawing properties under high hydraulic pressure were evaluated. This buoyancy engine is now installed in an underwater glider that will achieve long-term monitoring of ocean environments up to 2100 m depth in a designated area with lower operational costs. This bidirectionally functioning pump can control the amount of oil in extrusion and draw operations. When drawing oil under high pressure, the hydraulic pump and the electric motor, respectively, act as a hydraulic motor and an electric generator. The generated electric power is absorbed by a damping resistor. The oil-drawing and extrusion properties were measured using a large hyperbaric chamber that is able to produce an almost identical environment to that of actual operations. Results confirmed stable oil extrusion operations up to 21 MPa. Regarding oil-drawing properties, although it was measured only up to 10 MPa in the hyperbaric chamber, it can be inferred that the system can draw the oil and can control the buoyancy precisely up to 21 MPa by replacing the two-way ball valve with an electromagnetic latching solenoid valve.     

Hydrocarbons with a homoconjugated polyene system and their monoepoxy derivatives: Sex attractants of geometrid and noctuid moths distributed in Japan

Although several sex pheromones of the family Geometridae have been characterized, investigations on Japanese species are limited. In order to obtain more information, screening using known sex pheromones and their analogs was carried out. The (Z,Z,Z)-3,6,9-triunsaturated and (Z,Z)-6,9-diunsaturated hydrocarbons with straight C¹⁹-C²¹ chains were synthesized by the Grignard coupling reaction as a key step starting from linolenic and linoleic acids, respectively. Oxidation of the homoconjugated trienes withm-chloroperoxybenzoic acid yielded a 111 mixture of three monoepoxy derivatives that could be separated by silica gel chromatography. The chemical structure of each positional isomer was confirmed using two-dimensional NMR techniques and MS measurements, which enabled characteristic fragment ions from the isomers to be identified. Field tests using lures incorporating only one of the above six hydrocarbons or nine epoxides were carried out in a forest in Tokyo. Consequently, attraction of male moths of 14 geometrid species in addition to four species in another family, the Noctuidae, was observed. It was concluded that hydrocarbons with a homoconjugated polyene system and the monoepoxy derivatives are important components of sex pheromones produced by Japanese lepidopterous insects, particularly the geometrid moths.