Developing reinforcement learning for adaptive co-construction of continuous high-dimensional state and action spaces

Engineers and researchers are paying more attention to reinforcement learning (RL) as a key technique for realizing adaptive and autonomous decentralized systems. In general, however, it is not easy to put RL into practical use. Our approach mainly deals with the problem of designing state and action spaces. Previously, an adaptive state space construction method which is called a ??state space filter?? and an adaptive action space construction method which is called ??switching RL??, have been proposed after the other space has been fixed. Then, we have reconstituted these two construction methods as one method by treating the former method and the latter method as a combined method for mimicking an infant??s perceptual and motor developments and we have proposed a method which is based on introducing and referring to ??entropy??. In this paper, a computational experiment was conducted using a so-called ??robot navigation problem?? with three-dimensional continuous state space and two-dimensional continuous action space which is more complicated than a so-called ??path planning problem??. As a result, the validity of the proposed method has been confirmed.     

Development and motion control of a biped walking robot based on passive walking theory

This research aims to develop the biped walking robot that can walk on the horizontal ground and improve walking efficiency by utilizing the theory of the passive walking robot, namely the pendulum principle. For that, two motors were installed on the hip of the robot to generate the control torques to perform a walking motion. The computer simulations with dynamic model were carried out to investigate the walking capability of the system. Experimental robot was developed considering the calculated results. The proportional control law was used in walking experiment. The robot can walk on the horizontal ground with the proposed method.     

Innovative metabolic pathway design for efficient l-glutamate production by suppressing CO2 emission

In the pathway of L-glutamic acid (L-Glu) biosynthesis in Corynebacterium glutamicum, 1 mol of L-Glu is synthesized from 1 mol of glucose at a cost of 1 mol of carbon dioxide (CO(2)), with a maximum theoretical yield of 81.7% by weight. We have designed an innovative pathway for efficient L-Glu production employing phosphoketolase (PKT) to bypass the CO(2)-releasing pyruvate dehydrogenase reaction, thereby increasing the maximum theoretical yield of L-Glu from glucose to up to 98.0% by weight (120% mol/mol L-Glu produced/glucose consumed). The xfp gene encoding PKT was cloned from Bifidobacterium animalis and overexpressed under the strong cspB promoter in C. glutamicum. A functional enzyme was detected in an L-Glu-producing strain of C. glutamicum (odhA). When cells of this producer strain with the xfp gene and those without the xfp gene were cultivated in a controlled fermentation system, the L-Glu production yield of the strain expressing the xfp gene was much higher than that of the original strain, coupled with the suppression of CO(2) emission. Consequently, we could successfully enhance L-glutamate production by installing the PKT pathway of B. animalis into C. glutamicuml-Glu metabolism, and this novel metabolic design will be able to increase L-Glu production yield beyond the maximum theoretical yield obtained from the conventional metabolic pathway of biosynthesis from glucose.     

Development and performance evaluation of a GC-FPD method for quantification of ethephon in feeds

An analytical method of ethephon in feeds by GC-FPD was developed. Ethephon was extracted with ethyl acetate-hydrochloric acid (100 : 1) from feed samples. The extract was treated with added trimethylsilyldiazomethane in acetone-acetic acid (99 : 1) and this methylation procedure was repeated three times. Methylated ethephon was cleaned up on a graphitized carbon mini-column and a silicagel mini-column, and determined by GC-FPD. A method performance study in 7 laboratories was conducted with three kinds of samples spiked with ethephon at 10 mg/kg, 1 mg/kg and 0.5 mg/kg. The recovery of ethephon ranged from 81.8% to 90.8% (the reproducibility standard deviation (RSDr) were within 11%) and HorRat values were 0.58 to 0.94. The limit of detection (S/N≧3) and the limit of quantitation (S/N≧10) of ethephon in samples except hay were 0.02 mg/kg and 0.05 mg/kg, respectively. In the case of hay, the corresponding values were 0.2 mg/kg and 0.5 mg/kg, respectively.     

Solid‐state characterization and the thermal properties of stereoregular poly(vinyl chloride) prepared by urea clathrate polymerization

The solid‐state characterization of highly stereoregular poly(vinyl chloride) (PVC) prepared by urea clathrate polymerization was carried out by using various instrumental analyses. The structural differences of PVC appeared most remarkably in solubility to organic solvents, IR, WAXD, and solid‐state ¹³C‐NMR spectra. The value of the glass transition temperature (T_g) was about 90°C, not as high as expected, although its detection was quite difficult. The thermal stability was poor, as evidenced by the easy discoloration of this polymer by heat treatment, which was related to the absence of a termination reaction. Dynamic ESR spectra in the solid state clearly indicate that the radical formation occurs at such a low temperature as 160°C in the initial degradation stage. The degradation characteristics of urea clathrate PVC were critically discussed. © 1999 John Wiley & Sons, Inc. J Appl Polym Sci 74: 2820–2825, 1999     

Fracture Model of Fine Bulk Powder Structures

ABSTRACT

Using several line powders (particle size 15-65 μ m). the rheological parameters of tensile strength ( σ_f,mb) and plastic deformation coefficient (Y) were experimentally measured at ambient and elevated temperatures. In order to be aeratable. the rheological parameters of a specific powder should satisfy the equation of σ_f,mb = 0.11Y^{0 89.}The formation of agglomerates or dead zones can be predicted. By introducing the “Quasi porous solid body model,” the theoretical derivation of the criterion characteristic curve was accomplished using solid fracture theory.     

Study on combustion and ignition characteristics of natural gas components in a micro flow reactor with a controlled temperature profile

Combustion and ignition characteristics of natural gas components such as methane, ethane, propane and n-butane were investigated experimentally and computationally using a micro flow reactor with a controlled temperature profile. Special attention was paid to weak flames which were observed in a low flow velocity region. The observed weak flame responses for the above fuels were successfully simulated by one-dimensional computations with a detailed kinetic model for natural gas. Since the position of the weak flame indicates the ignition characteristics as well as the reactivity of each fuel, the experimental and computational results were compared with research octane number (RON) which is a general index for ignition characteristics of ordinary fuels. At 1 atm, ethane showed the highest reactivity among these fuels, although RON of ethane (115) is between those of methane (120) and propane (112). Since the pressure conditions are different between the present experiment and the general RON test, weak flame responses to the pressure were investigated computationally for these fuels. The order of the fuel reactivity by the reactor agreed with that by RON test when the pressure was higher than 4 atm. Reaction path analysis was carried out to clarify the reasons of the highest reactivity of ethane at 1 atm among the employed fuels in this study. The analysis revealed that C₂H₅ + O₂ ⇔ C₂H₄ + HO₂ is a key reaction and promotes ethane oxidation at 1 atm. The effect of the pressure on the fuel oxidation process in the present reactor was also clarified by the analysis. In addition, weak flame responses to various mixing ratios of methane/n-butane blends were investigated experimentally and computationally. The results indicated a significant effect of n-butane addition in the blends on combustion and ignition characteristics of the blended fuels.     

Stabilized three-stage oxidation of DME/air mixture in a micro flow reactor with a controlled temperature profile

Ignition and combustion characteristics of a stoichiometric dimethyl ether (DME)/air mixture in a micro flow reactor with a controlled temperature profile which was smoothly ramped from room temperature to ignition temperature were investigated. Special attention was paid to the multi-stage oxidation in low temperature condition.Normal stable flames in a mixture flow in the high velocity region, and non-stationary pulsating flames and/or repetitive extinction and ignition (FREI) in the medium velocity region were experimentally confirmed as expected from our previous study on a methane/air mixture. In addition, stable double weak flames were observed in the low velocity region for the present DME/air mixture case. It is the first observation of stable double flames by the present methodology. Gas sampling was conducted to obtain major species distributions in the flow reactor. The results indicated that existence of low-temperature oxidation was conjectured by the production of CH₂O occured in the upstream side of the experimental first luminous flame, while no chemiluminescence from it was seen.One-dimensional computation with detailed chemistry and transport was conducted. At low mixture velocities, three-stage oxidation was confirmed from profiles of the heat release rate and major chemical species, which was broadly in agreement with the experimental results.Since the present micro flow reactor with a controlled temperature profile successfully presented the multi-stage oxidations as spatially separated flames, it is shown that this flow reactor can be utilized as a methodology to separate sets of reactions, even for other practical fuels, at different temperature.