An efficient parameterization of dynamic neural networks for nonlinear system identification

Dynamic neural networks (DNNs), which are also known as recurrent neural networks, are often used for nonlinear system identification. The main contribution of this letter is the introduction of an efficient parameterization of a class of DNNs. Having to adjust less parameters simplifies the training problem and leads to more parsimonious models. The parameterization is based on approximation theory dealing with the ability of a class of DNNs to approximate finite trajectories of nonautonomous systems. The use of the proposed parameterization is illustrated through a numerical example, using data from a nonlinear model of a magnetic levitation system.     

An efficient integer programming model of the dynamic lot-sizing problem

An efficient integer programming model of the dynamic lot-sizing problem is presented. The Hillier-Lieberman model is improved by a new model that has only half the number of functional constraints. The reduction on the functional constraints, together with the imposition of a properly proposed by Wagner and Whitin, significantly simplifies the model, and thus greatly improves the computation efficiency. For the 20-period problems, the new model requires only 1/369 the computation time of the Hillier-Lieberman model.     

Energy function analysis of dynamic programming neural networks

All analytical examination of the energy function associated with a dynamic programming neural network is presented. The analysis is carried out in two steps. First, the locations and numbers of the minimum states for different components of the energy function are investigated in the extreme cases. A clearer insight into the energy function can be gained through the minimum states of different components. Secondly, the locations of the minimum states of the energy function using different parameter values are derived. It is shown that the minimum states can reside in regions which are regarded as valid solutions with certain conditions. Examples and simulation results are given to justify the validity of the theories developed.     

An efficient dynamic model for batch distillation

A novel, rigorous and efficient solution technique for multicomponent batch distillation modelling equations is proposed. Model predictions using the technique are shown to be in close agreement with experimental batch distillation data for a ten sieve tray, 15 cm diameter column separating ethanol and water. The results also show improved accuracy over commercially available programs for batch distillation. The method incorporates rigorous dynamic energy blances as well as accurate representation of both tray hydraulics and non-ideal mass transfer. The technique is based on a functional approximation for liquid enthalpy and makes a difficult-to-calculate temperature derivative implicit in other terms in the equations, eliminating the need for iterative solution techniques. The numerical efficiency of the method permits its utilization in model-based optimization and control calculations. The modelling approach is applicable to both batch and continuous dynamic distillation models.     

An efficient ordering-based ensemble pruning algorithm via dynamic programming

Although ordering-based pruning algorithms possess relatively high efficiency, there remains room for further improvement. To this end, this paper describes the combination of a dynamic programming technique with the ensemble-pruning problem. We incorporate dynamic programming into the classical ordering-based ensemble-pruning algorithm with complementariness measure (ComEP), and, with the help of two auxiliary tables, propose a reasonably efficient dynamic form, which we refer to as ComDPEP. To examine the performance of the proposed algorithm, we conduct a series of simulations on four benchmark classification datasets. The experimental results demonstrate the significantly higher efficiency of ComDPEP over the classic ComEP algorithm. The proposed ComDPEP algorithm also outperforms two other state-of-the-art ordering-based ensemble-pruning algorithms, which use uncertainty weighted accuracy and reduce-error pruning, respectively, as their measures. It is noteworthy that, the effectiveness of ComDPEP is just the same with that of the classical ComEP algorithm.     

An accelerator for neural networks with pulse-coded model neurons

The labeling of features by synchronization of spikes seems to be a very efficient encoding scheme for a visual system. Simulation of a vision system with millions of pulse-coded model neurons, however, is almost impossible on the basis of available processors including parallel processors and neurocomputers. A "one-to-one" silicon implementation of pulse-coded model neurons suffers from communication problems and low flexibility. On the other hand, acceleration of the simulation algorithm of pulse-coded leaky integrator neurons has proved to be straightforward, flexible, and very efficient. Thus we decided to develop an accelerator for a special version of the French and Stein (1970) neurons with modulatory inputs which are advantageous for simulation of synchronization mechanisms. Moreover, our accelerator also provides a Hebbian-like learning rule and supports adaptivity. Up to 128 K neurons with a total number of 16 M freely allocatable synapses are simulated within one system. The size of networks, however, is not at all limited by these numbers as the system may be arbitrarily expanded. Simulation speed obviously depends on the number of interconnections and on the average activity within the network. In the case of locally interconnected networks for simulation of vision mechanisms there is only a very low percentage of simultaneously active neurons: stimuli are not simultaneously presented in all orientations and at all positions of the visual field. In these cases our accelerator provides close to real-time behavior if one second of a biological neuron is simulated by 1000 time slots.     

An efficient approach for sub-image separation from large-scale multi-panel images using dynamic programming

Multimedia Tools and Applications - Multi-panel images are increasingly used in research and medical domains for describing complicated situations like results’ comparison in paper; or case...     

Lagrange programming neural networks for time-of-arrival-based source localization

Finding the location of a mobile source from a number of separated sensors is an important problem in global positioning systems and wireless sensor networks. This problem can be achieved by making use of the time-of-arrival (TOA) measurements. However, solving this problem is not a trivial task because the TOA measurements have nonlinear relationships with the source location. This paper adopts an analog neural network technique, namely Lagrange programming neural network, to locate a mobile source. We also investigate the stability of the proposed neural model. Simulation results demonstrate that the mean-square error performance of our devised location estimator approaches the Cramér–Rao lower bound in the presence of uncorrelated Gaussian measurement noise.     

An efficient dynamic programming algorithm for a special case of the capacitated lot-sizing problem

In this paper we consider the capacitated lot-sizing problem (CLSP) with linear costs. It is known that this problem is NP-hard, but there exist special cases that can be solved in polynomial time. We derive a new O(T²) algorithm for the CLSP with non-increasing setup costs, general holding costs, non-increasing production costs and non-decreasing capacities over time, where T is the length of the model horizon. We show that in every iteration we do not consider more candidate solutions than the O(T²) algorithm proposed by [Chung and Lin, 1988. Management Science 34, 420–6]. We also develop a variant of our algorithm that is more efficient in the case of relatively large capacities. Numerical tests show the superior performance of our algorithms compared to the algorithm of [Chung and Lin, 1988. Management Science 34, 420–6].     

The SHIFT programming language for dynamic networks of hybridautomata

SHIFT is a programming language for describing and simulating dynamic networks of hybrid automata. Such systems consist of components which can be created, interconnected, and destroyed as the system evolves. Components exhibit hybrid behavior, e.g. continuous-time phases separated by instantaneous discrete-event transitions. Components may evolve independently, or they may interact through selected state variables and events. The interaction network itself may evolve. The SHIFT model and language were motivated by our need for tools that support dynamically reconfigurable hybrid systems. Our primary application is the specification and analysis of different designs for automatic control of vehicles and highway systems. From our previous experience in modeling, analysis, and implementation, we adopted the hybrid systems approach for modeling the system components. Since spatial relationships between vehicles change as they move, our application is characterized by a dynamically changing network of interactions between system components. SHIFT has also since been used in coordinated autonomous submarines, air traffic control systems, and material handling systems. We examine other work related to the SHIFT approach. In we describe the main features of the SHIFT language-states, inputs, outputs, differential equations, and algebraic definitions, discrete states, and state transitions. We give a simplified version of the SHIFT model. We discuss the models of a type, a component, and the world and give the formal semantics of the model     

An analysis of the gamma memory in dynamic neural networks

Presents a vector space framework to study short-term memory filters in dynamic neural networks. The authors define parameters to quantify the function of feedforward and recursive linear memory filters. They show, using vector spaces, what is the optimization problem solved by the PEs of the first hidden layer of the single input focused network architecture. Due to the special properties of the gamma bases, recursion brings an extra parameter lambda (the time constant of the leaky integrator) that displaces the memory manifold towards the desired signal when the mean square error is minimized. In contrast, for the feedforward memory filter the angle between the desired signal and the memory manifold is fixed for a given memory order. The adaptation of the feedback parameter can be done using gradient descent, but the optimization is nonconvex.     

An efficient attention module for 3d convolutional neural networks in action recognition

Due to illumination changes, varying postures, and occlusion, accurately recognizing actions in videos is still a challenging task. A three-dimensional convolutional neural network (3D CNN), which can simultaneously extract spatio-temporal features from sequences, is one of the mainstream models for action recognition. However, most of the existing 3D CNN models ignore the importance of individual frames and spatial regions when recognizing actions. To address this problem, we propose an efficient attention module (EAM) that contains two sub-modules, that is, a spatial efficient attention module (EAM-S) and a temporal efficient attention module (EAM-T). Specifically, without dimensionality reduction, EAM-S concentrates on mining category-based correlation by local cross-channel interaction and assigns high weights to important image regions, while EAM-T estimates the importance score of different frames by cross-frame interaction between each frame and its neighbors. The proposed EAM module is lightweight yet effective, and it can be easily embedded into 3D CNN-based action recognition models. Extensive experiments on the challenging HMDB-51 and UCF-101 datasets showed that our proposed module achieves state-of-the-art performance and can significantly improve the recognition accuracy of 3D CNN-based action recognition methods.

     

一种高效动态LEO卫星网络流量调节路由算法

针对考虑负载均衡的LEO卫星网络路由算法存在控制网络开销偏大、路由更新不及时以及流量调节机制分配不均等问题,提出了一种基于负载均衡的动态LEO卫星网络路由算法DRLB。根据卫星节点路径记录信息以及后向Agent读取策略设计新的路由机制,获得动态卫星拓扑结构;分析前向Agent的分组格式并删除冗余字段,达到减小网络开销目的;根据数据发送时间间隔构造前向Agent选址策略,提高路由更新效率,通过考虑卫星所处纬度流量分配不均问题,改进流量调节因子,获得更好的负载均衡效果。仿真结果表明,与SDRZ-MA算法相比,DRLB算法在减缓星地之间的控制开销、平均端到端时延等方面具有较好的优势。     

Compression of recurrent neural networks for efficient language modeling

Recurrent neural networks have proved to be an effective method for statistical language modeling. However, in practice their memory and run-time complexity are usually too large to be implemented in real-time offline mobile applications. In this paper we consider several compression techniques for recurrent neural networks including Long–Short Term Memory models. We make particular attention to the high-dimensional output problem caused by the very large vocabulary size. We focus on effective compression methods in the context of their exploitation on devices: pruning, quantization, and matrix decomposition approaches (low-rank factorization and tensor train decomposition, in particular). For each model we investigate the trade-off between its size, suitability for fast inference and perplexity. We propose a general pipeline for applying the most suitable methods to compress recurrent neural networks for language modeling. It has been shown in the experimental study with the Penn Treebank (PTB) dataset that the most efficient results in terms of speed and compression–perplexity balance are obtained by matrix decomposition techniques.     

An efficient heat-based model for solid-liquid-gas phase transition and dynamic interaction

For melting simulation, solid-liquid coupling, liquid-gas interaction, bubble/foam generation, etc., many new methods have been emerging in recent years in computer graphics. To further push advance the technical frontier of the aforementioned phenomena, our novel solution is to focus on an efficient heat-based method towards faithful simulation of physical procedures pertinent to phase transitions and their dynamic interactions. On the methodology aspect, this paper details a simplified temperature-based model to animate the phase transitions and their dynamic interactions, including melting, freezing, and vaporization, by integrating the latent heat model with relevant governing physical laws. On the numerical aspect, our framework supports a new algorithm aiming at tight coupling of heat transfer and multiphase FLIP-based fluids. Specifically for liquid-gas phase transition, we take into account the dissolved gas involved in liquid which further enhances the bubble generation effects. Besides the unique feature of heat transfer, we also devise a SPH-FLIP coupled model to simulate sub-grid bubbles, which enables three-phase dynamic interactions among solid, liquid, and gas. The extensive experiments show that our hybrid approach can simultaneously handle multi-phase transition driven by physics-based heat conditions, as well as the multi-phase dynamic interactions with high fidelity and visual appeal.     

An effective and efficient approximate two-dimensional dynamic programming algorithm for supporting advanced computer vision applications

Dynamic programming is a popular optimization technique, developed in the 60’s and still widely used today in several fields for its ability to find global optimum. Dynamic Programming Algorithms (DPAs) can be developed in many dimension. However, it is known that if the DPA dimension is greater or equal to two, the algorithm is an NP complete problem. In this paper we present an approximation of the fully two-dimensional DPA (2D-DPA) with polynomial complexity. Then, we describe an implementation of the algorithm on a recent parallel device based on CUDA architecture. We show that our parallel implementation presents a speed-up of about 25 with respect to a sequential implementation on an Intel I7 CPU. In particular, our system allows a speed of about ten 2D-DPA executions per second for 85 × 85 pixels images. Experiments and case studies support our thesis.     

An efficient integer programming formulation for the assignment of base stations to controllers in cellular networks

In mobile networks, the assignment of base stations to controllers when planning the network has a strong impact on network performance. In a previous paper, the authors formulated the assignment of base stations to packet controllers in GSM-EDGE Radio Access Network (GERAN) as a graph partitioning problem, which was solved by a heuristic method. In this paper, an exact method is presented to find optimal solutions that can be used as a benchmark. The proposed method is based on an effective re-formulation of the classical integer linear programming model of the graph partitioning problem, which is solved by the branch-and-cut algorithm in a commercial optimization package. Performance assessment is based on an extensive set of problem instances built from data of a live network. Preliminary analysis shows some properties of the graphs in this problem justifying the limitations of heuristic approaches and the need for more sophisticated methods. Results show that the proposed method outperforms classical heuristic algorithms used for benchmarking, even under runtime constraints. Likewise, it improves the efficiency of exact methods previously applied to similar problems in the cellular field.     

基于动态递归神经网络模型的混沌时间序列预测

提出了一种动态递归神经网络模型进行混沌时间序列预测,以最佳延迟时间为间隔的最小嵌入维数作为递归神经网络的输入维数,并按预测相点步进动态递归的生成训练数据,利用混沌特性处理样本及优化网络结构,用递归神经网络映射混沌相空间相点演化的非线性关系,提高了预测精度和稳定性。将该模型应用于Lorenz系统数据仿真以及沪市股票综合指数预测,其结果与已有网络模型预测的结果相比较,精度有很大提高。因此,证明了该预测模型在实际混沌时间序列预测领域的有效性和实用性。     

A lagrangian propagator for artificial neural networks in constraint programming

This paper discusses a new method to perform propagation over a (two-layer, feed-forward) Neural Network embedded in a Constraint Programming model. The method is meant to be employed in Empirical Model Learning, a technique designed to enable optimal decision making over systems that cannot be modeled via conventional declarative means. The key step in Empirical Model Learning is to embed a Machine Learning model into a combinatorial model. It has been showed that Neural Networks can be embedded in a Constraint Programming model by simply encoding each neuron as a global constraint, which is then propagated individually. Unfortunately, this decomposition approach may lead to weak bounds. To overcome such limitation, we propose a new network-level propagator based on a non-linear Lagrangian relaxation that is solved with a subgradient algorithm. The method proved capable of dramatically reducing the search tree size on a thermal-aware dispatching problem on multicore CPUs. The overhead for optimizing the Lagrangian multipliers is kept within a reasonable level via a few simple techniques. This paper is an extended version of [27], featuring an improved structure, a new filtering technique for the network inputs, a set of overhead reduction techniques, and a thorough experimentation.     

Layered tile architecture for efficient hardware spiking neural networks

Spiking Neural Network (SNN) is the most recent computational model that can emulate the behaviour of biological neuron system. However, its main drawback is that it is computationally intensive, which limits the system scalability. This paper highlights and discusses the importance and significance of emulating SNNs in hardware devices. A layer-level tile architecture (LTA) is proposed for hardware-based SNNs. The LTA employs a two-level sharing mechanism of computing components at the synapse and neuron levels, and achieves a trade-off between computational complexity and hardware resource costs. The LTA is implemented on a Xilinx FPGA device. Experimental results demonstrate that this approach is capable of scaling to large hardware-based SNNs.