Integrating Guidance into Relational Reinforcement Learning

Reinforcement learning, and Q-learning in particular, encounter two major problems when dealing with large state spaces. First, learning the Q-function in tabular form may be infeasible because of the excessive amount of memory needed to store the table, and because the Q-function only converges after each state has been visited multiple times. Second, rewards in the state space may be so sparse that with random exploration they will only be discovered extremely slowly. The first problem is often solved by learning a generalization of the encountered examples (e.g., using a neural net or decision tree). Relational reinforcement learning (RRL) is such an approach; it makes Q-learning feasible in structural domains by incorporating a relational learner into Q-learning. The problem of sparse rewards has not been addressed for RRL. This paper presents a solution based on the use of reasonable policies to provide guidance. Different types of policies and different strategies to supply guidance through these policies are discussed and evaluated experimentally in several relational domains to show the merits of the approach.     

Elastodynamic near-tip fields for a crack propagating along the interface of two orthotropic solids

The elastodynamic stress field near a crack tip rapidly propagating along the interface between two dissimilar orthotropic elastic solids is solved numerically, for in-plane motion. The cartesian displacements are sought in the separated forms, $\text{r}{}^{\mathrm{p}}\text{U(θ)}$ and $\text{r}{}^{\mathrm{p}}\text{V(θ)}$, $\text{r}$ and θ being polar coordinates centered at the moving tip. This reduces the mathematical statement of the problem to two complex second-order linear ordinary differential equations for complex functions $\text{U(θ)}$ and $\text{V(θ)}$. By means of the finite difference method, a matrix eigenvalue problem of the type $\text{ΣA}{}_{\mathrm{ij}}\text{(p)X}{}_{\mathrm{j}}\text{= 0}$, is obtained where $\text{A}{}_{\mathrm{ij}}\text{(p)}$ are polynomials of the complex variable $\text{p}$ and $\text{X}{}_{\mathrm{j}}$, are complex unknowns. An iterative numerical scheme for determining $\text{Im(p)}$ is developed and the roots $\text{p}$ as well as angular stress and displacement distributions are calculated and plotted for various material combinations. Comparison with exact solutions for the case of dissimilar Isotropie solids indicates good accuracy of the numerical solution. The orthotropic nature of the materials is shown to have a significant effect on stress maximums.     

Finite element program for moisture and heat transfer in heated concrete

A new axisymmetric finite element program for the analysis of pore pressure, moisture content and temperature in heated concrete is described. The program is based on the diffusion equations for coupled heat and moisture transfer and uses a step-by-step time integration. The finite element scheme is based on Galerkin method. For time integration a step-by-step solution with iterations is used. The numerical analysis is complicated by the fact that the sorption isotherms exhibit a steep jump at saturation-nonsaturation transitions, and that the permeability dependence on temperature exhibits a jump of two orders of magnitude at 100°C. The mathematical model takes into account the release of chemically bound water due to dehydration and the associated changes in the pore space. The program may also be used at normal temperatures. Predictions of the program are compared with tests by HEDL as well as two other existing programs.     

Model-Reference Fuzzy Adaptive Control as a Framework for Nonlinear System Control

The paper presents a general methodology of adaptive control based on fuzzy model to deal with unknown plants. The problem of parameter estimation is solved using a direct approach, i.e. the controller parameters are adapted without explicitly estimating plant parameters. Thus, very simple adaptive and control laws are obtained using Lyapunov stability criterion. The generality of the approach is substantiated by Stone-Weierstrass theorem, which indicates that any continuous function can be approximated by fuzzy basis function expansion. In the sense of adaptive control, this implies the adaptive law with fuzzified adaptive control parameters. The proposed control algorithm may be viewed as an extension of classical adaptive control for linear plants, but compared to the latter it provides higher adaptation ability and consequently better performance if the plant is nonlinear. The global stability of the control system is assured and the tracking error converges to the residual set that depends on fuzzification properties. The main advantage of the approach is simplicity that suits control engineers since wide range of industrial processes can be controlled by the proposed method. In the paper, the control of heat exchanger is performed.     

Electrical properties and chemical compatibility of PZT thick film on Ni substrates

The characteristics of Pb(Zr,Ti)O₃(PZT) thick films that were printed and fired on Ni substrates were studied. The dielectric characteristics of samples sintered at 850°C on Ni substrates could not be measured due to the formation of a NiO layer at the Ni/PZT interface. The scanning electron microscope and energy dispersive X-ray analysis of cross-sections of the PZT thick films on Ni substrates, and of a mixture of PZT and NiO powders, fired at 850°C, did not indicate the formation of secondary phases. However, the transmission electron microscopy showed around 8% solid solubility of NiO in the PZT. A new structure with a prefired, Au thick-film layer was designed in order to prevent the diffusion of the NiO to the PZT layer during sintering. The dielectric properties of the PZT layers printed and fired on the Ni substrates with the prefired Au electrode were significantly better than those of the layers on the uncoated substrate, the dielectric losses decreased from 0.23 to 0.05.     

Modelling Highly Inflected Slovenian Language

This paper concerns the development of statistical language models of the Slovenian language for use in an automatic speech recognition system. The proposed techniques are language-independent and can be applied to other highly inflected Slavic languages. The large number of unique words in inflected languages is identified as the primary reason for performance degradation. This article discusses the concept of word-formation in the Slovenian language, which is also common to all Slavic languages. The main problems are outlined for word-based language models. A novel variation on the N-gram modelling theme is examined where, instead of using words, modelling units are chosen to be stems and endings. Only data-driven algorithms are employed, which decompose words automatically. A significant reduction in the OOV rate results when using stems and endings for modelling the Slovenian language. The final part of this article focuses on building a speech recogniser. Two different decoding strategies have been employed: one-pass and two-pass search decoders. Language modelling experiments have been performed using the VEER newswire text corpus, and recognition experiments have been conducted using the SNABI Slovenian speech database. The new language model resulted in the reduction of the OOV rate by 64%, and the recognition accuracy was improved by 4.34%.     

A characterisation of thick film resistors for strain gauge applications

Some commercial thick film resistors with sheet resistivities from 1 kohm/sq. up to 1 Mohm/sq. were evaluated for strain gauge applications. Temperature coefficients of resistivity, noise indices and gauge factors (GFs) were measured. For the same resistor series GFs and noise indices increase with increasing sheet resistivity. However, both GFs and noise indices are different for resistors with the same nominal sheet resistivity but from different resistor series. The results indicated that the microstructure rather than the different chemical composition of the conductive phase in thick film resistors is the primary reason for the different gauge factors.