Variance-penalized Markov decision processes: dynamic programming and reinforcement learning techniques

In control systems theory, the Markov decision process (MDP) is a widely used optimization model involving selection of the optimal action in each state visited by a discrete-event system driven by Markov chains. The classical MDP model is suitable for an agent/decision-maker interested in maximizing expected revenues, but does not account for minimizing variability in the revenues. An MDP model in which the agent can maximize the revenues while simultaneously controlling the variance in the revenues is proposed. This work is rooted in machine learning/neural network concepts, where updating is based on system feedback and step sizes. First, a Bellman equation for the problem is proposed. Thereafter, convergent dynamic programming and reinforcement learning techniques for solving the MDP are provided along with encouraging numerical results on a small MDP and a preventive maintenance problem.     

Small amphiphilic organics, coordination extended solids, and constant curvature structures

In this Account, we explore the role constant curvature structures play in amphiphilic small molecule crystals and extended coordination solids. A constant curvature structure is one in which there is a surface or interface that has the same curvature throughout its surface. Simple examples of such structures contain spheres (micelles), columns, or layers. Yet another family are cubic as is found in the gyroid topology. For amphiphilic systems, there are two domains, one generally hydrophobic and the other hydrophilic. We find that the interfaces between these two domains in small molecule structures resemble those in larger scale systems and adopt topologies related to constant curvature structures. The hydrophobic-to-total volume ratio, a molecular parameter, can be used to predict which type of constant curvature structure is adopted. In the case of three coordinate extended solids, constant curvature plays a role both in the extended net topology and in the hydrophobic-to-hydrophilic interface.     

Adhesion studies of tyre cords with rubber. I. Synthesis and characterization of natural rubber graft copolymers

Glycidyl methacrylate and N-vinyl pyrrolidone have been grafted onto natural rubber at 32°C using the simultaneous cobalt-60 irradiation technique. The natural rubber samples were swollen in the monomers for 24 h and thereafter subjected to gamma irradiation. The homopolymers formed in the graft copolymerization reactions and unreacted monomer were removed by solvent extraction using acetone and methanol for glycidyl methacrylate and N-vinyl pyrrolidone monomer, respectively. The influence of total dose and monomer concentration on the graft parameters was investigated. The dependence of the rate of grafting on the monomer concentration was found to be 0.93 and 0.80 for glycidyl methacrylate and N-vinyl pyrrolidone, respectively. DSC and TGA studies of the polymers were undertaken. Grafted copolymers based on glycidyl methacrylate were relatively less thermally stable compared with ungrafted natural rubber.     

Preparation and characterization of low-halogen and nonhalgoen fire-resistant low-smoke (FRLS) cable sheathing compound from blends of functionalized polyolefins and PVC

An attempt is made to develop a novel series of cable sheathing compounds with variation in chlorine content and sufficient fire retardance and unique low-smoke characteristics. These are prepared by blending PVC and functionalized polyolefins in different compositions. PE and EPDM have been functionalized by grafting dibutyl maleate (DBM) using DCP as initiator. FRLS compounds made from PVC-functionalized polyolefin blends possess the special characteristics of low-smoke, low-acid-gas generation, increased fire retardance, and improved volume resistivity, which are much better in comparison with a typical PVC sheathing compound. Thermoplastic elastomer (TPE) based nonhalogen FRLS compounds are also reported. The mechanisms for grafting, polymer-polymer and polymer-filler interactions have been presented.     

Anisotropy of X-ray structural parameters of the three phase model in cellulose I

Anisotropy of X-ray structural parameters of the three phase model proposed by Mitra and Mukherjee (ref.2), has been investigated in the case of cellulose fibres of ramie, hemp and jute. The structural parameters have been determined separately from each of the observable equatorial reflections. It has been found that the parameter degree of crystallinity as defined in the model is in fact anisotropic at least in the samples studied. The parameter degree of paracrystallinity introduced by Mitra and Mukherjee was also found to be anisotropic. Several important correlations between the structural parameters have been drawn. It has been observed, in agreement with the recent findings of Hosemann and Balta Calleja (ref.10) that the greater the paracrystalline distortion, the smaller the ‘paracrystallite size’.     

Energy Efficiency of Two‐Phase Mixing in a Modified Bubble Column

Energy efficiency for gas liquid mixing in a modified downflow bubble column reactor has been analyzed in this paper. Efficiencies of the different parts of the bubble column have been assessed on the basis of energy dissipation. Prediction of the energy dissipation coefficient as well as energy utilization efficiency due to gas‐liquid mixing as a function of different physical, geometric and dynamic variables of the system has been done by correlation method. The distribution of energy utilization in the different zones of the column has also been analyzed. Experiments were carried out with air‐water and air‐aqueous solutions of carboxy methyl cellulose with different concentrations.     

Pixel rearrangement based statistical restoration scheme reducing embedding noise

In this paper, a block based steganographic algorithm has been proposed where a sequence of secret bits are embedded into a set of pixels by rearranging the pixel locations. This algorithm has been devised as an improvement over existing statistical restoration based algorithms in order to reduce the additive noise which occurs due to embedding. It is shown that the proposed scheme substantially reduces the additive noise compared to existing statistical restoration based schemes.     

Synthesis and densification of magnesium aluminate spinel: effect of MgO reactivity

Stoichiometric magnesium aluminate spinel was synthesized by reaction sintering of alumina with caustic and sintered magnesia. The volume expansion of 5–7% during MgAl₂O₄ formation was utilized to identify the starting temperature of spinel formation and densification by high temperature dilatometry. The magnesia reactivity was determined by measurement of crystallite size and specific surface area. Caustic magnesia and sintered magnesia behave differently vis-à-vis phase formation and densification of spinel. Densification of stoichiometric Mag-Al spinel was carried out between 1650 and 1750 °C. Attempts were made to correlate the MgO reactivity with microstructure and densification of spinel.     

Prediction of flank wear by using back propagation neural network modeling when cutting hardened H-13 steel with chamfered and honed CBN tools

Productivity and quality in the finish turning of hardened steels can be improved by utilizing predicted performance of the cutting tools. This paper combines predictive machining approach with neural network modeling of tool flank wear in order to estimate performance of chamfered and honed Cubic Boron Nitride (CBN) tools for a variety of cutting conditions. Experimental work has been performed in orthogonal cutting of hardened H-13 type tool steel using CBN tools. At the selected cutting conditions the forces have been measured using a piezoelectric dynamometer and data acquisition system. Simultaneously flank wear at the cutting edge has been monitored by using a tool makers microscope. The experimental force and wear data were utilized to train the developed simulation environment based on back propagation neural network modeling. A trained neural network system was used in predicting flank wear for various different cutting conditions. The developed prediction system was found to be capable of accurate tool wear classification for the range it had been trained.