安全屏障机制下基于SAC算法的机器人导航系统

为了提高移动机器人自主导航系统的智能化水平和安全性,设计了安全屏障机制下基于SAC(Soft Actor-Critic)算法的自主导航系统,并构建了依赖于机器人与最近障碍物距离、目标点距离以及偏航角的回报函数.在Gazebo仿真平台中,搭建载有激光雷达的移动机器人以及周围环境.实验结果表明,安全屏障机制在一定程度上降低了机器人撞击障碍物的概率,提高了导航的成功率,并使得基于SAC算法的移动机器人自主导航系统具有更高的泛化能力.在更改起终点甚至将静态环境改为动态时,系统仍具有自主导航的能力.     

Adaptive variable structure control of MIMO nonlinear systems with time-varying delays and unknown dead-zones

In this paper, adaptive variable structure neural control is presented for a class of uncertain multi-input multi-output （MIMO） nonlinear systems with state time-varying delays and unknown nonlinear dead-zones. The unknown time-varying delay uncer- tainties are compensated for using appropriate Lyapunov-Krasovskii functionals in the design. The approach removes the assumption of linear function outside the deadband without necessarily constructing a dead-zone inverse as an added contribution. By utilizing the integral-type Lyapunov function and introducing an adaptive compensation term for the upper bound of the residual and optimal approximation error as well as the dead-zone disturbance, the closed-loop control system is proved to be semi-globally uniformly ultimately bounded. In addition, a modified adaptive control algorithm is given in order to avoid the high-frequency chattering phenomenon. Simulation results demonstrate the effectiveness of the approach.     

基于柔性策略-评价网络的微电网源储协同优化调度策略

近年来,微电网中的可再生能源与储能占比不断增大,给其优化调度带来了新的挑战。针对微电网源储协同调度问题中非凸非线性约束带来的求解困难,利用深度强化学习算法构建基于数据的策略函数,通过不断地与环境进行交互学习寻找最优策略,避免了对原非凸非线性问题的直接求解。考虑到训练过程中策略函数可能不满足安全约束,进一步提出了一种利用部分模型信息的微电网源储协同优化调度安全策略学习方法,得到了满足网络安全约束的优化策略。此外,针对强化学习的智能体在训练过程中与环境的交互耗时较长的问题,采用神经网络对环境进行建模以提高学习效率。     

LIDAR: learning from imperfect demonstrations with advantage rectification

In actor-critic reinforcement learning(RL)algorithms,function estimation errors are known to cause ineffective random exploration at the beginning of training,and lead to overestimated value estimates and suboptimal policies.In this paper,we address the problem by executing advantage rectification with imperfect demonstrations,thus reducing the function estimation errors.Pretraining with expert demonstrations has been widely adopted to accelerate the learning process of deep reinforcement learning when simulations are expensive to obtain.However,existing methods,such as behavior cloning,often assume the demonstrations contain other information or labels with regard to performances,such as optimal assumption,which is usually incorrect and useless in the real world.In this paper,we explicitly handle imperfect demonstrations within the actor-critic RL frameworks,and propose a new method called learning from imperfect demonstrations with advantage rectification(LIDAR).LIDAR utilizes a rectified loss function to merely learn from selective demonstrations,which is derived from a minimal assumption that the demonstrating policies have better performances than our current policy.LIDAR learns from contradictions caused by estimation errors,and in turn reduces estimation errors.We apply LIDAR to three popular actor-critic algorithms,DDPG,TD3 and SAC,and experiments show that our method can observably reduce the function estimation errors,effectively leverage demonstrations far from the optimal,and outperform state-of-the-art baselines consistently in all the scenarios.     

时变时滞随机非线性系统的自适应神经网络跟踪控制

针对一类具有时变时滞的不确定随机非线性严格反馈系统的自适应跟踪问题,利用Razumikhin引理和backstepping方法,提出一种新的自适应神经网络跟踪控制器.该控制器可保证闭环系统的所有误差变量皆四阶矩半全局一致最终有界,并且跟踪误差可以稳定在原点附近的邻域内.仿真例子表明所提出控制方案的有效性.     

Neural network language models for off-line handwriting recognition

Unconstrained off-line continuous handwritten text recognition is a very challenging task which has been recently addressed by different promising techniques. This work presents our latest contribution to this task, integrating neural network language models in the decoding process of three state-of-the-art systems: one based on bidirectional recurrent neural networks, another based on hybrid hidden Markov models and, finally, a combination of both. Experimental results obtained on the IAM off-line database demonstrate that consistent word error rate reductions can be achieved with neural network language models when compared with statistical N-gram language models on the three tested systems. The best word error rate, 16.1%, reported with ROVER combination of systems using neural network language models significantly outperforms current benchmark results for the IAM database.     

An ADDHP-based Q-learning algorithm for optimal tracking control of linear discrete-time systems with unknown dynamics

This paper investigates a novel Q-learning algorithm based on action dependent dual heuristic programming (ADDHP) to solve the infinite-time domain linear quadratic tracker (LQT) for unknown linear discrete-time systems. The proposed method is conducted based on only system data without requiring the knowledge of the system matrices. After the reference system is determined, an augmented system composed of the original system and the reference system is constructed, and it is proved that the value function of LQT is quadratic concerning the state of the augmented system. Using the quadratic value function, the augmented algebraic Riccati equation (ARE) is derived to solve the LQT. Due to the difficulty of directly solving the augmented ARE, a Q-learning algorithm based on ADDHP structure is used to solve this problem. With unknown system matrices, a model neural network is developed to reconstruct system dynamics incorporating stability analysis. The estimated system matrices are employed to the proposed algorithm to calculate the optimal control by policy iteration. Moreover, the convergence of the algorithm is proved. Two simulation examples are used to validate the performance of the method, where all results have demonstrated the effectiveness of the proposed ADDHP-based Q-learning method without a priori knowledge of system matrices for LQT.     

矿山信息物理融合系统多节点智联策略

针对当前矿山信息物理融合系统(CPS)的通信节点无法与基于不同无线通信协议的感知节点实现智能连接的问题,在通信节点上集成多种通信模块构成多模态通信节点,提出了一种基于渐进式神经网络的矿山CPS多节点智联策略。采用渐进式神经网络控制多模态通信节点准确切换工作模态,实现异构无线通信网络自主建立;利用异步优势动作评价算法对渐进式神经网络进行深度训练,提高渐进式神经网络的收敛速度和训练精度。实验结果表明,该策略实现了多模态通信节点与多类感知节点之间的准确、可靠通信。     

基于深度强化学习的智能灯个性化调节方法

提出一种基于深度强化学习的智能灯亮度个性化调节方法,综合考虑自然光亮度及用户位置对用户实际感受亮度的影响,动态计算并设置灯光亮度,以满足用户个性化使用习惯.在每次完成灯光亮度自动调节后,根据用户是否再次进行手动调节设定正、负反馈,训练强化学习模型逐渐拟合用户使用习惯.实验分别实现了DQN、DDQN和A3C三种算法,在基...     

Design of K-means clustering-based polynomial radial basis function neural networks (pRBF NNs) realized with the aid of particle swarm optimization and differential evolution

In this paper, we introduce an advanced architecture of K-means clustering-based polynomial Radial Basis Function Neural Networks (p-RBF NNs) designed with the aid of Particle Swarm Optimization (PSO) and Differential Evolution (DE) and develop a comprehensive design methodology supporting their construction. The architecture of the p-RBF NNs comes as a result of a synergistic usage of the evolutionary optimization-driven hybrid tools. The connections (weights) of the proposed p-RBF NNs being of a certain functional character and are realized by considering four types of polynomials. In order to design the optimized p-RBF NNs, a prototype (center value) of each receptive field is determined by running the K-means clustering algorithm and then a prototype and a spread of the corresponding receptive field are further optimized through running Particle Swarm Optimization (PSO) and Differential Evolution (DE). The Weighted Least Square Estimation (WLSE) is used to estimate the coefficients of the polynomials (which serve as functional connections of the network). The performance of the proposed model and the comparative analysis involving models designed with the aid of PSO and DE are presented in case of a nonlinear function and two Machine Learning (ML) datasets