RLProph: a dynamic programming based reinforcement learning approach for optimal routing in opportunistic IoT networks

Wireless Networks - Routing in Opportunistic Internet of Things networks (OppIoTs) is a challenging task because of intermittent connectivity between devices and the lack of a fixed path between...     

CDMA multiuser detection: a nonlinear programming approach

The optimum receiver to detect the bits of multiple code-division multiple access (CDMA) users has an exponential complexity in the number of active users in the system. Consequently, many suboptimum receivers have been developed to achieve good performance with less complexity. We take the approach of approximating the solution of the optimum multiuser detection problem (OMUD) using nonlinear programming relaxations. First, we observe that some popular suboptimum receivers indeed correspond to relaxations of the optimal detection problem. In particular, one proposed approximation method yields to iterative solutions which correspond to previously proposed heuristic nonlinear detectors. Using a nonlinear programming approach, we identify the convergence properties of these iterative detectors. Secondly, we propose a relaxation that yields a receiver which we call the generalized minimum mean squared error detector. We give a simple iterative implementation of the detector. Its performance is evaluated and comparisons to other suboptimum detection schemes are given     

Multiobjective concurrent engineering: a goal programming approach

Concurrent engineering (CE) is a commonly used engineering/management technique for product development. Multiple and sometimes conflicting objectives can be experienced when using CE as an aid in new product development. One management science technique that commonly used to analyze multiple objectives in a decision making environment is goal programming (GP). In this paper, the authors demonstrate how goal programming can be used to enhance CE and improve decision making by considering conflicting objective criteria of cost and time. The procedure by which CE problems can be modeled using GP is presented along with an illustrative example from the literature. Results from a real-world ongoing study are also reported     

Adaptive dynamic programming

Unlike the many soft computing applications where it suffices to achieve a "good approximation most of the time," a control system must be stable all of the time. As such, if one desires to learn a control law in real-time, a fusion of soft computing techniques to learn the appropriate control law with hard computing techniques to maintain the stability constraint and guarantee convergence is required. The objective of the paper is to describe an adaptive dynamic programming algorithm (ADPA) which fuses soft computing techniques to learn the optimal cost (or return) functional for a stabilizable nonlinear system with unknown dynamics and hard computing techniques to verify the stability and convergence of the algorithm. Specifically, the algorithm is initialized with a (stabilizing) cost functional and the system is run with the corresponding control law (defined by the Hamilton-Jacobi-Bellman equation), with the resultant state trajectories used to update the cost functional in a soft computing mode. Hard computing techniques are then used to show that this process is globally convergent with stepwise stability to the optimal cost functional/control law pair for an (unknown) input affine system with an input quadratic performance measure (modulo the appropriate technical conditions). Three specific implementations of the ADPA are developed for 1) the linear case, 2) for the nonlinear case using a locally quadratic approximation to the cost functional, and 3) the nonlinear case using a radial basis function approximation of the cost functional; illustrated by applications to flight control.     

Optimal motion planning and control of a nonholonomic spherical robot using dynamic programming approach: simulation and experimental results

Optimal motion planning and control of a nonholonomic spherical mobile robot is studied. Dynamic Programming (DP) as a direct and online approach is used to navigate the robot in an environment with/without obstacles. The optimal trajectory, which corresponds to the minimum cost, is determined in the case of presence of obstacles in the environment, and the robot can move towards the target optimally, without colliding with obstacles. DP yields optimal control inputs in a closed-loop form. In fact, a traditional control system is no longer needed to track the obtained trajectory since the resulted DP table includes optimal control inputs for every state in the admissible region. The effect of different final states and alternative intervals of the allowable state and control values are also studied. Results from several simulations show that the proposed method enables the robot to find an optimal trajectory from any given state towards a predefined target. An experimental setup is designed wherein a real spherical robot is driven according to the developed algorithm. A vision system monitors the robot and outputs the location/orientation of the robot at each step via image processing. Experimental results are then compared with simulations to validate the model and evaluate the control strategy.     

Modality independent elastography (MIE): a new approach to elasticity imaging

The correlation between tissue stiffness and health is an accepted form of organ disease assessment. As a result, there has been a significant amount of interest in developing methods to image elasticity parameters (i.e., elastography). The modality independent elastography (MIE) method combines a nonlinear optimization framework, computer models of soft-tissue deformation, and standard measures of image similarity to reconstruct elastic property distributions within soft tissue. In this paper, simulation results demonstrate successful elasticity image reconstructions in breast cross-sectional images acquired from magnetic resonance (MR) imaging. Results from phantom experiments illustrate its modality independence by reconstructing elasticity images of the same phantom in both MR and computed tomographic imaging units. Additional results regarding the performance of a new multigrid strategy to MIE and the implementation of a parallel architecture are also presented.     

Mapping dynamic programming onto a linear systolic array

In the paper we show a single, efficient implementation of dynamic programming on alinear array using a new mapping methodology. In this method, we start with a known 2-D array onto which the dynamic programming algorithm has been mapped. By partitioning and stretching, this 2-D array is mapped onto a linear array. We derive a data movement scheme to simulate the data streams and the computations in the 2-D array. This scheme is implemented usingfast/slow data channels. Compared to known designs in the literature our design uses constant storage in each PE, constant number of I/O lines and continuous I/O sequence. Besides, the data and control flow in the array is unidirectional. This property makes the design suitable for implementation on the well-known fault-tolerant Wafer Scale Integration model.This research was supported in part by the National Science Foundation under grant IRI-8710836 and by AFOSR under grant AFOSR-0032.     

Constrained shortest link-disjoint paths selection: a network programming based approach

Given a graph G with each link in the graph associated with two positive weights, cost and delay, we consider the problem of selecting a set of k link-disjoint paths from a node s to another node t such that the total cost of these paths is minimum and that the total delay of these paths is not greater than a specified bound. This problem, to be called the constrained shortest link-disjoint path (CSDP(k)) problem, can be formulated as an integer linear programming (ILP) problem. Relaxing the integrality constraints results in an upper bounded linear programming problem. We first show that the integer relaxations of the CSDP(k) problem and a generalized version of this problem to be called the generalized CSDP (GCSDP (k)) problem (in which each path is required to satisfy a specified bound on its delay) both have the same optimal objective value. In view of this, we focus our work on the relaxed form of the CSDP(k) problem (RELAX-CSDP(k)). We study RELAX-CSDP(k) from the primal perspective using the revised simplex method of linear programming. We discuss different issues such as formulas to identify entering and leaving variables, anti-cycling strategy, computational time complexity etc., related to an efficient implementation of our approach. We show how to extract from an optimal solution to RELAX-CSDP(k) a set of k link-disjoint s-t paths which is an approximate solution to the original CSDP(k) problem. We also derive bounds on the quality of this solution with respect to the optimum. We present simulation results that demonstrate that our algorithm is faster than currently available approaches. Our simulation results also indicate that in most cases the individual delays of the paths produced starting from RELAX-CSDP(k) do not deviate in a significant way from the individual path delay requirements of the GCSDP(k) problem.     

Ultrasound elastography based on multiscale estimations of regularized displacement fields

Elasticity imaging is based on the measurements of local tissue deformation. The approach to ultrasound elasticity imaging presented in this paper relies on the estimation of dense displacement fields by a coarse-to-fine minimization of an energy function that combines constraints of conservation of echo amplitude and displacement field continuity. The multiscale optimization scheme presents several characteristics aimed at improving and accelerating the convergence of the minimization process. This includes the nonregularized initialization at the coarsest resolution and the use of adaptive configuration spaces. Parameters of the energy model and optimization were adjusted using data obtained from a tissue-like phantom material. Elasticity images from normal in vivo breast tissue were subsequently obtained with these parameters. Introducing a smoothness constraint into motion field estimation helped solve ambiguities due to incoherent motion, leading to elastograms less degraded by decorrelation noise than the ones obtained from correlation-based techniques.     

Systolic processing for dynamic programming problems

In this paper we investigate systolic processing for problems formulated in dynamic programming. These problems are classified as monadic-serial, polyadic-serial, monadic-nonserial, and polyadic-nonserial. Problems in serial formulations can be implemented easily in systolic arrays; however, nonserial problems may have to be transformed into a serial one before an efficient implementation can be found. monadic-serial dynamic programming problem can be solved as the search of an optimal path in a multistage graph and can be computed as a string of matrix multiplications. Three efficient systolic-array designs are presented. A polyadic-serial dynamic programming problem can be solved by either a divide- and-conquer algorithm or the search of optimal solutions in a serial AND/OR-graph. We have evaluated the asymptotically optimal architecture for divide- and-conquer algorithms and have developed efficient methods of mapping a regular AND/OR-graph into systolic arrays. Cases are studied for transforming a problem in a nonserial formulation into a serial one.This research was supported by National Science Foundation Grant DMC 85-19649 and Joint Services Electronics Program Contract N000014-84-C-0149.     

Design of practically perfect-reconstruction cosine-modulated filter banks: a second-order cone programming approach

Designing optimal perfect-reconstruction (PR) and near PR (NPR) cosine-modulated filter banks is essentially a constrained nonlinear minimization problem. This paper proposes two second-order cone-programming based algorithms for designing NPR and practically PR cosine-modulated filter banks with improved performance relative to several established design methods.     

Genetic expression programming: a new approach for QoS traffic prediction in EPONs

The Ethernet passive optical network is being regarded as the most promising for next-generation optical access solutions in the access networks. In time division multiplexing, passive optical network technology (TDM-PON) and the dynamic bandwidth allocation (DBA) play a crucial key role to achieve efficient bandwidth allocation and fairness among subscribers. Therefore, the traffic prediction in DBA during the waiting time must be put into the account. In this paper, we propose a new prediction approach with an evolutionary algorithm genetic expression programming (GEP) prediction incorporated with Limited IPACT referred as GLI-DBA to tackle the queue variation during waiting times as well as to reduce the high-priority packet delay. Simulation results show that the GEP prediction in DBA can reduce the expedited forwarding (EF) packet delay, shorten the EF queue length, enhance the quality of services and maintain the fairness among the optical network units (ONUs). We conducted and evaluated the detail simulation in two different scenarios with distinctive traffic proportion. Simulation results show that the GLI-DBA has EF packet delay improvement up to 30 % over dynamic bandwidth allocation for multiple of services (DBAM). It also shows that our proposed prediction scheme performs better than the DBAM when the number of ONUs increases.     

Progress in dynamic programming search for LVCSR

Initially introduced in the late 1960s and early 1970s, dynamic programming algorithms have become increasingly popular in automatic speech recognition. There are two reasons why this has occurred. First, the dynamic programming strategy can be combined with a very efficient and practical pruning strategy so that very large search spaces can be handled. Second, the dynamic programming strategy has turned out to be extremely flexible in adapting to new requirements. Examples of such requirements are the lexical tree organization of the pronunciation lexicon and the generation of a word graph instead of the single best sentence. We attempt to review the use of dynamic programming search strategies for large-vocabulary continuous speech recognition (LVCSR). The following methods are described in detail: searching using a lexical tree, language-model look-ahead and word-graph generation     

Chemical mechanical planarization operation via dynamic programming

In this paper, the impact on non-planarization index by the down force and rotational speed during a SiO₂ or Cu CMP process was investigated. Since the magnitudes of down force and rotational speed have limits, we choose the dynamic programming approach because of its ability to achieve constrained optimization by the down force and rotational speed. The duration and the amount of input were computed based on the chemical mechanical polishing model by Luo and Dornfeld [J. Luo, D.A. Dornfeld, IEEE Trans. Semiconduct. Manufact. 14(2) (2001) 112-132.] when the other parameters were fixed. Experiments done for blanket wafers based on dynamic programming operation and conventional constant removal rate operation was compared with each other. The non-planarization index could be improved consistently by dynamic programming operation versus constant removal rate operation. The improvement ranges from 2% to 39% improvement over the base recipe of constant removal rate in all experiments as shown in Table 3 and Table 6. The thickness removal error is consistently smaller by constant removal rate operation versus dynamic programming operation in all experiments as shown in Table 3 and Table 6. To get the best performance of both planarization and thickness removal, it is recommended that planarization step and overpolish step in SiO₂ and Cu CMP should use different mode of operation, i.e., dynamic programming operation during planarization step for minimizing non-planarization index and constant removal rate operation during overpolish step for minimizing thickness removal error. The incremental time calculation for eliminating thickness removal error during overpolish step can be done using the thickness error and removal rate derived from Luos’ removal rate model based on constant wafer pressure and platen speed at the end of planarization step.Our contribution is a new approach for CMP. Standard CMP uses constant removal rate operation in both planarization step and overpolish step. Our new approach uses dynamic programming operation during planarization step and constant removal rate operation during overpolish step.     

Formant coding of speech using dynamic programming

An algorithm is proposed which will obtain, from an input speech signal, formant parameter data to control a parallel formant speech synthesiser. By allowing some delay and employing variable-frame-rate techniques, the parameter data can be obtained at a low frame rate (typically 20 frames per second) suitable for transmission or storage.     

A dynamic programming processor for speech recognition

A dynamic programming processor with parallel and pipeline architecture is described. A 2-μm CMOS technology was applied to the DP processor, which is composed of 127309 transistors on a 7.17×8.62-mm² die and is housed in an 84-pin PLCC (plastic leaded chip carrier) or PGA (pin grid array) package. The clock frequency is 20 MHz, and the instruction cycle time is 100 ns. Precise electrical simulations permitted the safe use of nonstandard logic and area and power reduction. Implementation of a direct access to all internal registers has proven useful for chip test and software development. A system using one DP processor has given very good results on a wide variety of applications and 0.48% error rate on tests with standard NATO tapes. These results are significantly better than those published for other systems on the same tests     

A dynamic programming algorithm for nonlinear smoothing

This paper presents a smoothness optimization approach to the nonlinear smoothing problem. Linear smoothing techniques fail to provide adequate results for curves which exhibit both sharp discontinuities to be preserved and, due to measurement or processing errors, outliers and noise to be filtered out. The nonlinear algorithm is based on a criterion for the overall smoothness of the curve. The smoothness criterion is optimized by a dynamic programming strategy. The resulting algorithm turns out to be computationally attractive. The computation time grows proportionally to N² and the storage requirements are 2N locations where N is the number of samples to be smoothed. The algorithm is applied to smoothing pitch contours.