EM-based policy hyper parameter exploration: application to standing and balancing of a two-wheeled smartphone robot

This paper proposes a novel policy search algorithm called EM-based Policy Hyper Parameter Exploration (EPHE) which integrates two reinforcement learning algorithms: Policy Gradient with Parameter Exploration (PGPE) and EM-based Reward-Weighted Regression. Like PGPE, EPHE evaluates a deterministic policy in each episode with the policy parameters sampled from a prior distribution given by the policy hyper parameters (mean and variance). Based on EM-based Reward-Weighted Regression, the policy hyper parameters are updated by reward-weighted averaging so that gradient calculation and tuning of the learning rate are not required. The proposed method is tested in the benchmarks of pendulum swing-up task, cart-pole balancing task and simulation of standing and balancing of a two-wheeled smartphone robot. Experimental results show that EPHE can achieve efficient learning without learning rate tuning even for a task with discontinuities.     

Tetra-Repeat Microsatellite Markers for the Masu Salmon (Oncorhynchus masou masou) and Its Application in Cross-Subspecies Amplification

We developed tetranucleotide-repeat microsatellite markers for the masu salmon (Oncorhynchus masou) complex. 454 pyrosequencing was used to discover repeat motifs, and seven polymorphic microsatellite-primer sets were identified. The number of alleles detected at each locus ranged from four to 24 and the expected heterozygosity varied from 0.57 to 0.92. Cross-subspecies amplification for O. m. masou, O. m. ishikawae and O. m. subsp. was successful. These microsatellites can be utilized in studies of genetic structure, genetic diversity, and intra- and inter-subspecific hybridization, making a contribution to conservation and management of the Oncorhynchus masou complex.     

Influence of wood species on the properties of biopolyurethane prepared from liquefied wood with residue

Biopolyurethane prepared from liquefied wood with the residue of the liquefied wood product was investigated in this article. Previous results indicated that the residue of the liquefaction product was composed mostly of compounds originated from lignin. The chemical structures of lignin in softwood and hardwood are different. The influence of soft‐ and hardwood species on the chemical structure and mechanical properties of biopolyurethane prepared from liquefied wood with residue was investigated by tensile testing and Fourier transform infrared spectroscopy. The experimental results showed that the liquefaction of softwood occurs within a shorter time than that of hardwood and the biopolyurethane prepared from softwood was harder than that prepared from hardwood, which suggests that the properties of the liquefaction product and biopolyurethane are influenced by the chemical structure of the lignin. © 2010 Wiley Periodicals, Inc. J Appl Polym Sci, 2010     

Synthesis of charge transporting block copolymers containing 2,7-dimethoxycarbazole units for light emitting device

The block copolymers consisting of 2,7-dimethoxycarbazole- and oxadiazole-containing segments as hole and electron transporting units, respectively, were synthesized by NMRP manner. OLED devices were fabricated using block copolymer, random copolymer, and polymer blend for matrix of the emitting layer with Ir(ppy)₃ as a phosphorescent dopant in order to investigate morphological effect on the performance. From the finding that the block copolymer system overwhelmed the others in EQE, we assumed that a morphology with dimethoxycarbazole units assembled to the surface of PEDOT:PSS played a considerable role for effective recombination of charges as well as sufficient charge injection into the emitting layer.     

Photocatalytic hydrogenation of acetophenone derivatives and diaryl ketones on polycrystalline titanium dioxide

In the absence of molecular oxygen, various aromatic ketones such as acetophenone derivatives and diaryl ketones were photocatalytically hydrogenated on polycrystalline titanium dioxide (Degussa P25) under UV light irradiation (> 340 nm). The desired secondary alcohols were obtained with excellent chemical efficiency (almost 100% yields for 10 examples) by choosing ethanol as a sacrificial hole scavenger, which was oxidized into acetaldehyde.     

EFFECT OF HEATING MODES ON INTERNAL STRAIN-STRESS FORMATION DURING DRYING OF MOLDED CERAMICS

Parametric analyses in drying processes of molded ceramics are performed to investigate the influence of heating modes on the formation of drying-induced strain-stress as well as the drying characteristic. The transient three-dimensional problem of strain-stress and heat and moisture transfer in a slab is solved simultaneously by the finite element method. Three modes of hot air, intermittent and internal heating are compared by modeling in the normalized parameters. The tensile and compressive stresses fluctuate, and fall remarkably during the low Biot number period when the slab is heated intermittently. In the internal heating mode, the drying rate is the fastest but stress formation is maintained at the lowest level among the three modes. This effectiveness of the internal heating is investigated experimentally by employing the microwave heating as well.     

Transient Behavior of Gas Entrainment Caused by Surface Vortex

A compact sodium-cooled reactor is an important candidate as a fast breeder reactor (FBR) and has been investigated in the feasibility study of FBR cycle. Due to the compact sizing of the reactor vessel, gas entrainment at the free surface of sodium coolant becomes one of the significant issues for reactor design, and it is required to clarify the criterion of gas entrainment at free surface and the tolerance. In the present study, some visualization experiments were performed in a water-air system focusing on the gas entrainment due to surface vortex and its transient phenomena. Influences of horizontal velocity were clarified by the visualization. The gas entrainment due to the surface vortex occurs intermittently. Time trends of circulation and length of gas core for the intermittent surface vortices were measured by the particle image velocimetry and visualization. It was found that the gas core length extends with time delay to the increase of circulation around the vortex.     

Treatment capability of an up-flow anammox column reactor using polyethylene sponge strips as biomass carrier

The feasibility of applying a polyethylene (PE) sponge as a biomass carrier in an anaerobic ammonium oxidation (anammox) reactor and its nitrogen removal performance were also investigated. Experiments were carried out in an up-flow column reactor with synthetic inorganic wastewater. Experimental results indicate that reactor containing PE sponge biomass carriers showed a high nitrogen removal capability and exhibited stable performance. In addition, the reactor with 8 strips PE sponge as biomass carrier exhibited greater adaptation capacity compared to that with 6 strips and could achieve a high TN removal rate within a very short period. The ratio of NO₂-N removal and NO₃-N production to NH₄-N removal for the reactor was 1.26:0.21. Furthermore, to investigate the bacterial composition of the mature community, 16S rRNA sequences were amplified by PCR and analyses were conducted using DNA databases. Results showed that a new kind of anammox microorganism (Kumadai-1) was the dominant species in the reactor when using PE sponge as a biomass carrier.     

Switched capacitor circuit for 1/f noise generation using a power-law circuit

A simple switched capacitor (SC) circuit is given which realizes a one-dimensional discrete dynamical system X_n+1 = X_n + ux^z generating intermittent chaos with a 1/f-shaped power spectrum. an integrated circuit with a pair of logarithmic and antilogarithmic amplifiters can easily realize the power-law term x^z which plays a key role in generating l/fnoise. an inevitable small offset voltage ? in the SC circuit governs the cut-off frequency of 1/f noise, and when the dynamical system is modified by introducing an effective ?, the ‘indirect method’ enables us to get the precise power spectrum in the modified system.     

Novel method for driving the ultrasonic motor

This paper reports a novel driving method for an annular plate-type ultrasonic motor. Instead of the direct current/alternating current (DC/AC) converter type driver using a conventional electromagnetic transformer, a compact disc-type piezoelectric transformer is used to obtain a high voltage output for driving the ultrasonic motor. The piezoelectric transformer is operated in the radial vibration mode at resonance frequency close to the resonance frequency of the ultrasonic motor. Later, it was found that the piezoelectric transformer could drive the ultrasonic motor, even if their resonance frequencies are not exactly the same by incorporating a matching network in the circuit. The maximum speed of the ultrasonic motor obtained by using this driving method is over 300 rpm. It is believed that the results of this study will have impact on the integration and miniaturization of the ultrasonic motor and its driving circuit.