Recent Advances in Hierarchical Reinforcement Learning

Reinforcement learning is bedeviled by the curse of dimensionality: the number of parameters to be learned grows exponentially with the size of any compact encoding of a state. Recent attempts to combat the curse of dimensionality have turned to principled ways of exploiting temporal abstraction, where decisions are not required at each step, but rather invoke the execution of temporally-extended activities which follow their own policies until termination. This leads naturally to hierarchical control architectures and associated learning algorithms. We review several approaches to temporal abstraction and hierarchical organization that machine learning researchers have recently developed. Common to these approaches is a reliance on the theory of semi-Markov decision processes, which we emphasize in our review. We then discuss extensions of these ideas to concurrent activities, multiagent coordination, and hierarchical memory for addressing partial observability. Concluding remarks address open challenges facing the further development of reinforcement learning in a hierarchical setting.     

Chemical toughening of epoxies. I. Structural modification of epoxy resins by reaction with hydroxy-terminated poly(butadiene-co-acrylonitrile)

In an attempt to toughen the epoxy resin matrix for fiber-reinforced composite applications, a chemical modification procedure of a commercially available bisphenol-A-based epoxy resin using reactive liquid rubber HTBN [hydroxy-terminated poly(butadiene-co-acrylonitrile)] and TDI (tolylene diisocyanate) is described. The progress of the reaction and the structural changes during modification process are studied using IR spectroscopy, viscosity data, and chemical analysis (epoxy value determination). The studies support the proposition that TDI acts as a coupling agent between the epoxy and HTBN, forming a urethane linkage with the former and an oxazolidone ring with the latter. The chemical reactions that possibly take place during the modification are discussed.     

Short‐beam three‐point bend test study in syntactic foam. Part III: Effects of interface modification on strength and fractographic features

Syntactic foam slabs having uncoated microballoons and paraffin oil surface‐treated microballoons were fabricated and tested for short‐beam three‐point bend test. The work points to the role of paraffin oil coating first weakening the interface between the microballoons and the matrix and hence lowering the efficiency of load transfer from matrix to the fillers (i.e., microballoons). This led to an overall decrement of 71% in the experimentally measured strength value compared to the deduced value for uncoated microballoons' specimens. The large strengths for uncoated microballoons specimens can be traced to the presence of the curvilinear marks in the matrix that, incidentally, are absent in the case of paraffin oil coated specimens. These observations are revealed distinctly in the microscopy of test‐failed specimens. © 2005 Wiley Periodicals, Inc. J Appl Polym Sci 98: 687–693, 2005     

Variability-aware architecture level optimization techniques for robust nanoscale chip design

The design space for nanoscale CMOS circuits is vast, with multiple dimensions corresponding to process variability, leakage, power, thermal, reliability, security, and yield considerations. These design issues in the form of either objectives or constraints can be handled at various levels of digital design abstraction, such as architectural, logic and transistor. At the architectural level (a.k.a. Register-Transfer Level, RTL), there is a balanced degree of freedom for fast design exploration by exploring various values of design parameters. Correct design decisions at an early phase of the design cycle ensure that design errors are not propagated to lower levels of circuit abstraction, where it is costly to correct them. Moreover, design optimization at higher levels of abstraction provides a convenient way to deal with design complexity, facilitates design verification, and increases design reuse through intellectual property (IP) cores.     

Mullitization of Diphasic Aluminosilicate Gels

Recent studies have shown that the mullitization of diphasic aluminosilicate matrices comprising transitional alumina and amorphous silica occurs via a nucleation and growth process. Nucleation is preceded by a temperature-dependent incubation period. Following this incubation period, rapid nucleation of mullite occurs, producing about 1.8 × 10¹¹ nuclei/cm³, which remains constant throughout the rest of the transformation. Both incubation and mullite growth are thermally activated processes with apparent activation energies of 987 ± 63 and 1070 ± 200 kJ/mol, respectively. The growth rate of mullite grains under isothermal conditions is time dependent. An interpretation of these results is proposed on the basis of the nucleation and growth concepts of LaMer and Dinegar which supports the concept that the growth rate of mullite grains is controlled by the dissolution of transitional alumina into the amorphous matrix.     

Brassica campestris var. Span: I. Fractionation of Rapeseed oil by molecular distillation and adsorption chromatography

Procedures for the large scale isolation of pure triglycerides and fractions rich in nontriglyceride components from Span rapeseed oil are described. Fractionation ofBrassica campestris var. Span rapeseed oil by molecular distillation yielded 4 triglyceride fractions, all of which contained traces of sterol esters. An additional triglyceride fraction rich in free and esterified sterols and other volatile components was obtained from the oil. Separation by adsorption chromatography of Span rapeseed oil yielded three fractions: A) a pure triglyceride fraction; B) a triglyceride fraction rich in sterol esters; and C) another fraction containing free sterols and other polar components. Contribution no. 559 Animal Research Institute.     

Short beam three point bend tests in syntactic foams. Part I: Microscopic characterization of the failure zones

Scanning Electron Microscopic (SEM) studies were carried out on the failure surface of syntactic foam material tested in a short beam three point bend test (SBT) by employing 21 × 15 × 3 mm³ dimension bearing specimens. The syntactic foams were fabricated using glass microballoons in epoxy binder. The failure of the tensile, compression, and shear dominated regions were studied by SEM at different magnifications. The tensile region had characteristic features, such as partial debonding of the microballoons from the matrix and cracking of glass microballoons, apart from matrix cracking and some river pattern features. The compression side was characterized by crushing and collapsing of microballoons, resulting in accumulation of debris with no apparent river pattern for matrix‐rich regions. The midway positions of the SBT failed surface comprised of deformation bands in the matrix and occasional debonding of microballoons. The morphology recorded in the tensile and compression regions corroborated well with the results obtained on these foam samples in those specimens that were subjected to pure uniaxial tension and compression, respectively. © 2005 Wiley Periodicals, Inc. J Appl Polym Sci 98: 673–679, 2005     

Atomic and micro-structure features of nanoporous aluminosilicate glasses from reactive molecular dynamics simulations

Long-term chemical durability of borosilicate glasses that makes them a widely accepted form of nuclear waste disposal is achieved through the formation of a porous aluminosilicate gel layer that provides passivity and limits the transport of water to the reaction front. Detailed understanding of the porous silicate gel layer is thus critical in elucidating the corrosion mechanism of these glasses and to design of new glass composition for waste immobilization and other applications. In this paper, we use the diffuse charge reactive potential to generate porous aluminosilicate glass structures with compositions equivalent to the gel layers formed at the glass-water interface with an aim to understand the processing condition on the microstructure and atomic structure of these systems. We demonstrate the use of the charge scaling techniques is an effective approach to generate these porous structures with controllable pore mophologies. After initial validation of the potentials and calcium aluminosilicate glass structures using neutron diffraction, we created gel structures with compositions similar to well-known model nuclear waste borosilicate glasses. The porosities and the pore size distribution bear a strong correlation to the processing temperature, as well as to the local atomic structure. Thus, by controlling the processing parameters, the generated porous structures can be customized to closely resemble gel structures due to borosilicate glass corrosion. These results provide insights of the micro- and atomic structure features of the porous aluminosilicate glasses and on the optimal procedure to generate porous structures that can be comparable to experimentally observed gel layer structures thus to elaborate on the correlations between the structure and phenomena in glass-water interactions.