Learning long-term dependencies in NARX recurrent neural networks

It has previously been shown that gradient-descent learning algorithms for recurrent neural networks can perform poorly on tasks that involve long-term dependencies, i.e. those problems for which the desired output depends on inputs presented at times far in the past. We show that the long-term dependencies problem is lessened for a class of architectures called nonlinear autoregressive models with exogenous (NARX) recurrent neural networks, which have powerful representational capabilities. We have previously reported that gradient descent learning can be more effective in NARX networks than in recurrent neural network architectures that have "hidden states" on problems including grammatical inference and nonlinear system identification. Typically, the network converges much faster and generalizes better than other networks. The results in this paper are consistent with this phenomenon. We present some experimental results which show that NARX networks can often retain information for two to three times as long as conventional recurrent neural networks. We show that although NARX networks do not circumvent the problem of long-term dependencies, they can greatly improve performance on long-term dependency problems. We also describe in detail some of the assumptions regarding what it means to latch information robustly and suggest possible ways to loosen these assumptions.     

Designing power and performance optimal application-specific Network-on-Chip architectures

This paper presents an optimal method for topology synthesis by taking into account factors related to power, performance, and contention in an application-specific Network-on-Chip (NoC) architecture. A Tabu search based approach is used for topology generation with an automated design technique, incorporating floorplan information to attain accurate values for power consumption of the routers and physical links. The Tabu search method incorporates multiple objectives and is able to generate optimal NoC topologies which account for both power and performance. The contention analysis technique assesses performance and relieves any potential bottlenecks using virtual channel insertion after considering its effect on power consumption and performance improvement within the NoC. The contention analyzer uses a Layered Queuing Network approach to model the rendezvous interactions among system components. Several experiments are conducted using various SoC benchmark applications to compare the power and performance outcomes of the proposed technique.     

Generalized far-infrared magneto-optic ellipsometry for semiconductor layer structures: determination of free-carrier effective-mass,mobility, and concentration parameters in n-type GaAs

We report for the first time on the application of generalized ellipsometry at far-infrared wavelengths (wave numbers from 150 cm(-1) to 600 cm(-1)) for measurement of the anisotropic dielectric response of doped polar semiconductors in layered structures within an external magnetic field. Upon determination of normalized Mueller matrix elements and subsequent derivation of the normalized complex Jones reflection matrix r of an n-type doped GaAs substrate covered by a highly resistive GaAs layer, the spectral dependence of the room-temperature magneto-optic dielectric function tensor of n-type GaAs with free-electron concentration of 1.6 x 10(18) cm(-3) at the magnetic field strength of 2.3 T is obtained on a wavelength-by-wavelength basis. These data are in excellent agreement with values predicted by the Drude model. From the magneto-optic generalized ellipsometry measurements of the layered structure, the free-carrier concentration, their optical mobility, the effective-mass parameters, and the sign of the charge carriers can be determined independently, which will be demonstrated. We propose magneto-optic generalized ellipsometry as a novel approach for exploration of free-carrier parameters in complex organic or inorganic semiconducting material heterostructures, regardless of the anisotropic properties of the individual constituents.     

Extracting finite-state representations from recurrent neuralnetworks trained on chaotic symbolic sequences

Concerns neural-based modeling of symbolic chaotic time series. We investigate the knowledge induction process associated with training recurrent mural nets (RNN) on single long chaotic symbolic sequences. Even though training RNN to predict the next symbol leaves the standard performance measures such as the mean square error on the network output virtually unchanged, the nets extract a lot of knowledge. We monitor the knowledge extraction process by considering the nets stochastic sources and letting them generate sequences which are then confronted with the training sequence via information theoretic entropy and cross-entropy measures. We also study the possibility of reformulating the knowledge gained by RNN in a compact easy-to-analyze form of finite-state stochastic machines. The experiments are performed on two sequences with different complexities measured by the size and state transition structure of the induced Crutchfield's epsilon-machines (1991, 1994). The extracted machines can achieve comparable or even better entropy and cross-entropy performance. They reflect the training sequence complexity in their dynamical state representations that can be reformulated using finite-state means. The findings are confirmed by a much more detailed analysis of model generated sequences. We also introduce a visual representation of allowed block structure in the studied sequences that allows for an illustrative insight into both RNN training and finite-state stochastic machine extraction processes.     

Description and visualization of graminaceous plants with an organ-based 3D architectural model,exemplified for spring barley (<Emphasis Type="Italic">Hordeum vulgare</Emphasis> L.)

Within the framework of functional-structural plant models (FSPMs) this paper presents a structural (architectural) plant model that describes the morphology of spring barley (Hordeum vulgare L.) plants. A set of equations is introduced and implemented in self-written Matlab&reg; computer programs to calculate the surface shape of vegetative and generative plant organs. Organ surfaces are approximated as geometrical primitives, visualized as triangulated surface meshes. The output of the model is a set of triangles that can be associated with geometrical (e.g. area), topological (e.g. main stem) and physiological (e.g. chlorophyll content) attributes, depending on the measurement techniques applied. This information is a prerequisite for functional (process) models to compute, e.g., the radiation field or gas exchange of the respective canopy.     

Financial volatility trading using recurrent neural networks

We simulate daily trading of straddles on financial indexes. The straddles are traded based on predictions of daily volatility differences in the indexes. The main predictive models studied are recurrent neural nets (RNN). Such applications have often been studied in isolation. However, due to the special character of daily financial time-series, it is difficult to make full use of RNN representational power. Recurrent networks either tend to overestimate noisy data, or behave like finite-memory sources with shallow memory; they hardly beat classical fixed-order Markov models. To overcome data nonstationarity, we use a special technique that combines sophisticated models fitted on a larger data set, with a fixed set of simple-minded symbolic predictors using only recent inputs. Finally, we compare our predictors with the GARCH family of econometric models designed to capture time-dependent volatility structure in financial returns. GARCH models have been used to trade volatility. Experimental results show that while GARCH models cannot generate any significantly positive profit, by careful use of recurrent networks or Markov models, the market makers can generate a statistically significant excess profit, but then there is no reason to prefer RNN over much more simple and straightforward Markov models. We argue that any report containing RNN results on financial tasks should be accompanied by results achieved by simple finite-memory sources combined with simple techniques to fight nonstationarity in the data.