Data-driven policy iteration algorithm for continuous-time stochastic linear-quadratic optimal control problems

This paper studies a continuous-time stochastic linear-quadratic (SLQ) optimal control problem on infinite-horizon. Combining the Kronecker product theory with an existing policy iteration algorithm, a data-driven policy iteration algorithm is proposed to solve the problem. In contrast to most existing methods that need all information of system coefficients, the proposed algorithm eliminates the requirement of three system matrices by utilizing data of a stochastic system. More specifically, this algorithm uses the collected data to iteratively approximate the optimal control and a solution of the stochastic algebraic Riccati equation (SARE) corresponding to the SLQ optimal control problem. The convergence analysis of the obtained algorithm is given rigorously, and a simulation example is provided to illustrate the effectiveness and applicability of the algorithm.     

基于改进拟牛顿-K近邻的TDOA定位算法

为了提高TDOA声源定位方法室内定位的速度和准确度。本文提出一种改进拟牛顿-K近邻（IQN-IKN）方法。基于定位方程相邻三次迭代结果构造近似矩阵,并采用近似矩阵逼近雅可比矩阵,提高拟牛顿方法定位中非线性定位方程的求解速度;提出基于密集距离法的K近邻方法,进行定位估计结果的寻优,提高定位的准确性。实验结果表明,相对于传统拟牛顿算法,改进算法具有更快的定位估计速度及更高的定位准确性。     

Calculation of the permanent of a sparse positive matrix

The exact calculation of permanents of n×n matrices is a non-polynomial computational problem as a function of n. An efficient deterministic algorithm is presented that allows for the approximate calculation of permanents obtained from sparse positive matrices within controllable precision bounds. The upper and lower bounds can be made arbitrarily close to each other and the algorithm outperforms existing ones for sufficiently large matrices.     

广义离散系统多传感器信息融合Kalman滤波器

考虑了广义离散随机线性系统的多传感器信息融合状态估计问题．在广义系统无脉冲的假设条件下。通过等价变换将其转化为正常系统．应用经典Kalman滤波方法,在线性最小方差信息融合准则下,提出了按矩阵加权的广义系统多传感器信息融合稳态Kalman状态滤波器．仿真结果说明了算法的有效性。     

Subspace algorithms for identifying separable-in-denominator 2D systems with deterministic–stochastic inputs

In this paper, we present an algorithm for identifying two-dimensional (2D) causal, recursive and separable-in-denominator (CRSD) state-space models in the Roesser form with deterministic–stochastic inputs. The algorithm implements the N4SID, PO-MOESP and CCA methods, which are well known in the literature on 1D system identification, but here we do so for the 2D CRSD Roesser model. The algorithm solves the 2D system identification problem by maintaining the constraint structure imposed by the problem (i.e. Toeplitz and Hankel) and computes the horizontal and vertical system orders, system parameter matrices and covariance matrices of a 2D CRSD Roesser model. From a computational point of view, the algorithm has been presented in a unified framework, where the user can select which of the three methods to use. Furthermore, the identification task is divided into three main parts: (1) computing the deterministic horizontal model parameters, (2) computing the deterministic vertical model parameters and (3) computing the stochastic components. Specific attention has been paid to the computation of a stabilised Kalman gain matrix and a positive real solution when required. The efficiency and robustness of the unified algorithm have been demonstrated via a thorough simulation example.     

On Kalman Active Observers

The paper introduces the Active Observer (AOB) algorithm in the framework of Kalman filters. The AOB reformulates the Kalman filter to accomplish model-reference adaptive control based on: (1) A desired closed loop system. (2) An extra equation to estimate an equivalent disturbance referred to the system input. An active state is introduced to compensate unmodeled terms, providing a feedforward compensation action. (3) Stochastic design of the Kalman matrices. Stability analysis with model errors is discussed. An example of robot force control with an external and unknown nonlinear disturbance is presented (SISO system). Another example of model-matching control for steer-by-wire (SBW) vehicles with underactuated structure is discussed (MIMO system).     

Solving certain queueing problems modelled by toeplitz matrices

By using the concept of generating function associated with a Toeplitz matrix, we analyze existence conditions for the probability invariant vector π of certain stochastic semi-infinite Toeplitz-like matrices. An application to the shortest queue problem is shown. By exploiting the functional formulation given in terms of generating functions, we devise a weakly numerically stable algorithm for computing the probability invariant vector π. The algorithm is divided into three stages. At the first stage the zeros of a complex function are numerically computed by means of an extension of the Aberth method. At the second stage the first k components of π are computed by solving an interpolation problem, where k is a suitable constant associated with the matrix. Finally, at the third stage a triangular Toeplitz system is solved and its solution is refined by applying the power method or any other refinement method based on regular splittings. In the solution of the triangular Toeplitz system and at each step of the refinement method, special FFT-based techniques are applied in order to keep the arithmetic cost within the O(n log n) bound, where n is an upper bound to the number of the computed components. Numerical comparisons with the available algorithms show the effectiveness of our algorithm in a wide set of cases.     

基于小波算法的二阶线性定常分布参数系统的辨识

本文基于正交函数逼近方法,借助于小波变换,并利用其运算矩阵及其运算性质,研究了分布参数系统的辨识问题。将Haar小波正交基应用于分布参数系统的辨识中,经正交小波逼近变换,将原偏微分描述的分布参数系统转化为代数矩阵方程,并且,考虑了初始条件和边界条件,获得了算法简单、计算方便、具有较高精度的辨识算法,简化了分布参数系统辨识的求解过程,应用在分布参数系统辨识中不失为一种有效的分析方法。仿真实例表明了本文所提出的算法的有效性。     

隐式多变量广义自校正控制器

本文提出了具有在线选择加权阵的两种多变量自校正控制器.加权阵的在线选择和自校 正控制算法都采用隐式方案来实现.本文还证明了该自校正控制器具有全局收敛性.     

随机奇异系统分布式最优融合降阶卡尔曼滤波器

Based on the optimal fusion algorithm weighted by matrices in the linear minimum variance (LMV) sense, a distributed full-order optimal fusion Kalman filter (DFFKF) is given for discrete-time stochastic singular systems with multiple sensors, which involves the inverse of a high-dimension matrix to compute matrix weights. To reduce the computational burden, a distributed reduced-order fusion Kalman filter (DRFKF) is presented, which involves in parallel the inverses of two relatively low-dimension matrices to compute matrix weights. A simulation example shows the effectiveness.     

随机系统的多模型直接自适应解耦控制器

针对多变量离散时间随机系统, 提出了一种采用广义最小方差性能指标的多模型直接自适应解耦控制器. 该多模型控制器由多个固定控制器和两个自适应控制器构成. 固定控制器用以覆盖系统参数的可能变化范围, 自适应控制器用以保证系统的稳定性和提高暂态性能. 该多模型控制器利用矩阵的伪交换性和拟Diophantine方程性质, 基于广义最小方差性能指标, 将随机系统辨识算法和最优控制器设计相结合, 直接辨识出控制器的参数, 通过广义最小方差性能指标中加权多项式的选取,不但实现了多变量系统的动态解耦控制, 而且消除了稳态误差、配置了闭环极点. 文末给出了全局收敛性分析. 仿真结果表明该方法明显优于常规自适应控制器.     

Stochastic Stability and Stabilization of Singular It ‐type Markovian Jump Systems with Uncertain Transition Rates: An LMI Approach

This paper investigates the stochastic stability and stabilization for a class of singular stochastic systems of Itô‐type with Markovian switching, the transition rates (TRs) in the jumping processes are uncertain. The aims are to establish sufficient conditions to ensure the considered system to be stochastically stable in the mean square sense, which is supported by a detailed proof of existence and uniqueness of the system solution, and to propose a controller such that the system can be stabilizable. The controller is first proposed and has advantage over traditional ones, the controller gain matrices are obtained by solving a strict linear matrix inequality (LMI). Finally, a numerical example is provided to illustrate the validity of the obtained methodology.     

A Permanent Algorithm with exp[Ω ( n ^{1/3}/2 ln n )] Expected Speedup for 0-1 Matrices

Abstract. This paper outlines a permanent algorithm with mildly exponential expected speedup over Ryser's inclusion and exclusion algorithm for 0-1 matrices. The algorithm is based on a finite-difference formula that is a generalization of Ryser's formula. The matrix is augmented by a column of entries selected to produce zero-valued terms in the formula. The algorithm achieves speedup by avoiding computation of many zero-valued terms.     

Efficient and High-quality Recommendations via Momentum-incorporated Parallel Stochastic Gradient Descent-Based Learning

A recommender system (RS) relying on latent factor analysis usually adopts stochastic gradient descent (SGD) as its learning algorithm. However, owing to its serial mechanism, an SGD algorithm suffers from low efficiency and scalability when handling large-scale industrial problems. Aiming at addressing this issue, this study proposes a momentum-incorporated parallel stochastic gradient descent (MPSGD) algorithm, whose main idea is two-fold: a) implementing parallelization via a novel data-splitting strategy, and b) accelerating convergence rate by integrating momentum effects into its training process. With it, an MPSGD-based latent factor (MLF) model is achieved, which is capable of performing efficient and high-quality recommendations. Experimental results on four high-dimensional and sparse matrices generated by industrial RS indicate that owing to an MPSGD algorithm, an MLF model outperforms the existing state-of-the-art ones in both computational efficiency and scalability.      

关于矩阵指数的PADE逼近新算法

基于广义逆矩阵Pade逼近的特点是在保持逼近阶的前提下,在构造过程中不需要 用到矩阵的乘法运算.利用该结果建立矩阵指数etA的一种新的非线性逼近算法.该方法与原 Pade近似法相比具有明显的优点,即它对奇异矩阵和高阶矩阵是适用的,并且所得到的算法 适合编程上机进行计算.给出的一个计算实例说明了算法的有效性.逼近公式的存在性和唯 一性得到了证明.     

Minimax filtering in linear stochastic uncertain discrete-continuous systems

Filtering of the states of a system, whose dynamics is defined by an Ito stochastic differential equation, by discrete and discrete-continuous observations is studied under the assumption that the intensities of continuous noises and covariance matrices of discrete noises are known only within to membership of certain uncertainty sets. A minimax approach is used to solve the problem. The filter is optimized with an integral quality criterion. Minimax filtering equations are derived from the solution of the dual optimization problem. A numerical solution algorithm for the problem is designed. Results of numerical experiments are presented.     

用插值的粗粒化Box Model进行学习控制

针对一类未知动力学方程的单输入系统,采用插值的粗粒化Box Model进行学习控制. 与传统的Box Model及基于随机元胞自动机的控制模式比较,本模型设计思想自然,算法 简单.计算量小(常常小几个数量级),训练时间短,对三个较有代表性的实例的仿真结果 也是令人满意的.     

多变量系统的仿真软件

控制系统仿真的核心问题是微分方程组和代数方程组的联立求解.微分方程组的数值解 法很多,其中以变阶变步长Gear算法最适合控制系统仿真.基于Gear算法作者研制了 面向典型元件框图和传递函数矩阵的通用仿真软件,具有友好的人机界面,功能强,使用方便, 为控制系统的智能设计提供了有效工具.     

The distributed data SCF

This paper describes a distributed data parallel SCF algorithm. The distinguishing features of this algorithm are: (a) columns of density and Fock matrices are distributed evenly among processors, (b) pair-wise dynamic load balancing is developed to achieve excellent load balance, (c) network communication time is minimized via careful analysis of data flow in the SCF algorithm. The developed performance models and benchmarking results illustrate good performance of the distributed data SCF algorithm.     

An ADDHP-based Q-learning algorithm for optimal tracking control of linear discrete-time systems with unknown dynamics

This paper investigates a novel Q-learning algorithm based on action dependent dual heuristic programming (ADDHP) to solve the infinite-time domain linear quadratic tracker (LQT) for unknown linear discrete-time systems. The proposed method is conducted based on only system data without requiring the knowledge of the system matrices. After the reference system is determined, an augmented system composed of the original system and the reference system is constructed, and it is proved that the value function of LQT is quadratic concerning the state of the augmented system. Using the quadratic value function, the augmented algebraic Riccati equation (ARE) is derived to solve the LQT. Due to the difficulty of directly solving the augmented ARE, a Q-learning algorithm based on ADDHP structure is used to solve this problem. With unknown system matrices, a model neural network is developed to reconstruct system dynamics incorporating stability analysis. The estimated system matrices are employed to the proposed algorithm to calculate the optimal control by policy iteration. Moreover, the convergence of the algorithm is proved. Two simulation examples are used to validate the performance of the method, where all results have demonstrated the effectiveness of the proposed ADDHP-based Q-learning method without a priori knowledge of system matrices for LQT.