Parallel Reinforcement Learning: A Framework and Case Study

In this paper, a new machine learning framework is developed for complex system control, called parallel reinforcement learning. To overcome data deficiency of current data-driven algorithms, a parallel system is built to improve complex learning system by self-guidance. Based on the Markov chain (MC) theory, we combine the transfer learning, predictive learning, deep learning and reinforcement learning to tackle the data and action processes and to express the knowledge. Parallel reinforcement learning framework is formulated and several case studies for real-world problems are finally introduced.      

Octopus: Combining Learning and Parallel Search

This paper presents Octopus, an automated theorem-proving system that combines learning and parallel search. The learning technique involves proving a simpler version of a given theorem and then using what it has learned to prove the given theorem. As of January 2004 Octopus had successfully proved 43 of the 1.0-rated theorems of the TPTP Problem Library.     

Learning with Genetic Algorithms: An Overview

Genetic algorithms represent a class of adaptive search techniques that have been intensively studied in recent years. Much of the interest in genetic algorithms is due to the fact that they provide a set of efficient domain-independent search heuristics which are a significant improvement over traditional weak methods without the need for incorporating highly domain-specific knowledge. There is now considerable evidence that genetic algorithms are useful for global function optimization and NP-hard problems. Recently, there has been a good deal of interest in using genetic algorithms for machine learning problems. This paper provides a brief overview of how one might use genetic algorithms as a key element in learning systems.     

Exact Learning of Formulas in Parallel

We investigate the parallel complexity of learning formulas from membership and equivalence queries. We show that many restricted classes of boolean functions cannot be efficiently learned in parallel with a polynomial number of processors.     

计算系统生物学中并行随机仿真方法研究进展

随机仿真是计算系统生物学中对随机离散模型进行仿真研究的一类重要方法。本文对随机仿真方法的原理及并行化研究的进展进行了论述,指出了并行化是解决随机仿真性能开销问题的重要途径,并依照细粒度并行和粗粒度并行分类,阐述了当前并行随机仿真方法的研究现状,重点针对空间并行性,介绍了反应-扩散系统随机仿真的方法和相关工具。最后,对并行随机仿真方法研究未来发展进行了展望。     

2-DOF并联机构动力学建模与迭代学习控制

针对一种直线电机驱动的2-DOF并联机构,结合直线电机的动力学特性,采用Lagrange方法对其进行动力学建模。考虑该机构重复性动作及其不确定性和非线性特点,提出一种自适应神经网络迭代学习控制方法。在该控制算法的作用下,系统输出能较好地跟踪给定输入。严格证明及仿真结果验证了该算法的有效性。     

并行学习神经网络集成方法

该文分析了神经网络集成中成员神经网络的泛化误差、成员神经网络之间的差异度对神经网络集成泛化误差的影响,提出了一种并行学习神经网络集成方法;对参与集成的成员神经网络,给出了一种并行训练方法,不仅满足了成员网络本身的精度要求,还满足了它与其余成员网络的差异性要求;另外,给出了一种并行确定集成成员神经网络权重方法．实验结果表明,使用该文的成员神经网络训练方法、成员神经网络集成方法能够构建有效的神经网络集成系统．     

Parallel Driving in CPSS:A Unified Approach for Transport Automation and Vehicle Intelligence

The emerging development of connected and automated vehicles imposes a significant challenge on current vehicle control and transportation systems. This paper proposes a novel unified approach, Parallel Driving, a cloud-based cyberphysical-social systems (CPSS) framework aiming at synergizing connected automated driving. This study first introduces the CPSS and ACP-based intelligent machine systems. Then the parallel driving is proposed in the cyber-physical-social space, considering interactions among vehicles, human drivers, and information. Within the framework, parallel testing, parallel learning and parallel reinforcement learning are developed and concisely reviewed. Development on intelligent horizon (iHorizon) and its applications are also presented towards parallel horizon. The proposed parallel driving offers an ample solution for achieving a smooth, safe and efficient cooperation among connected automated vehicles with different levels of automation in future road transportation systems.      

一种基于P学习的分布式并行多任务分配算法

并行多任务分配是多agent系统中极具挑战性的课题, 主要面向资源分配、灾害应急管理等应用需求, 研究如何把一组待求解任务分配给相应的agent联盟去执行. 本文提出了一种基于自组织、自学习agent的分布式并行多任务分配算法, 该算法引入P学习设计了单agent寻找任务的学习模型, 并给出了agent之间通信和协商策略. 对比实验说明该算法不仅能快速寻找到每个任务的求解联盟, 而且能明确给出联盟中各agent成员的实际资源承担量, 从而可以为实际的控制和决策任务提供有价值的参考依据.     

可扩展机器学习的并行与分布式优化算法综述

机器学习问题通常会转换成一个目标函数去求解,优化算法是求解目标函数中参数的重要工具.在大数据环境下,需要设计并行与分布式的优化算法,通过多核计算和分布式计算技术来加速训练过程.近年来,该领域涌现了大量研究工作,部分算法也在各机器学习平台得到广泛应用.本文针对梯度下降算法、二阶优化算法、邻近梯度算法、坐标下降算法、交替方向乘子算法五类最常见的优化方法展开研究,每一类算法分别从单机并行和分布式并行来分析相关研究成果,并从模型特性、输入数据特性、算法评价、并行计算模型等角度对每个算法进行详细对比.随后对有代表性的可扩展机器学习平台中优化算法的实现和应用情况进行对比分析.同时对本文中介绍的所有优化算法进行多层次分类,方便用户根据目标函数类型选择合适的优化算法,也可以通过该多层次分类图交叉探索如何将优化算法应用到新的目标函数类型.最后分析了现有优化算法存在的问题,提出可能的解决思路,并对未来研究方向进行展望.     

Parallel Control for Continuous-Time Linear Systems: A Case Study

In this paper, a new parallel controller is developed for continuous-time linear systems. The main contribution of the method is to establish a new parallel control law, where both state and control are considered as the input. The structure of the parallel control is provided, and the relationship between the parallel control and traditional feedback controls is presented. Considering the situations that the systems are controllable and incompletely controllable, the properties of the parallel control law are analyzed. The parallel controller design algorithms are given under the conditions that the systems are controllable and incompletely controllable. Finally, numerical simulations are carried out to demonstrate the effectiveness and applicability of the present method.      

Reinforcement Learning in Process Industries: Review and Perspective

This survey paper provides a review and perspective on intermediate and advanced reinforcement learning (RL) techniques in process industries. It offers a holistic approach by covering all levels of the process control hierarchy. The survey paper presents a comprehensive overview of RL algorithms, including fundamental concepts like Markov decision processes and different approaches to RL, such as value-based, policy-based, and actor-critic methods, while also discussing the relationship between classical control and RL. It further reviews the wide-ranging applications of RL in process industries, such as soft sensors, low-level control, high-level control, distributed process control, fault detection and fault tolerant control, optimization, planning, scheduling, and supply chain. The survey paper discusses the limitations and advantages, trends and new applications, and opportunities and future prospects for RL in process industries. Moreover, it highlights the need for a holistic approach in complex systems due to the growing importance of digitalization in the process industries.

     

计算几何与并行算法

本文首先介绍了计算几何的基本概念,论述了计算几何的四个基本问题,即几何搜索问题、相交问题、邻接问题及凸壳问题。然后重点分析了凸壳构造问题,介绍了其最佳串行算法、及相应的并行算法。接着对一些计算几何的串行及并行算法进行了分析比较。最后提出了笔者对新一代并行计算机系统上设计计算几何并行算法的看法。     

ALECSYS and the AutonoMouse: Learning to Control a Real Robot by Distributed Classifier Systems

In this article we investigate the feasibility of using learning classifier systems as a tool for building adaptive control systems for real robots. Their use on real robots imposes efficiency constraints which are addressed by three main tools: parallelism, distributed architecture, and training. Parallelism is useful to speed up computation and to increase the flexibility of the learning system design. Distributed architecture helps in making it possible to decompose the overall task into a set of simpler learning tasks. Finally, training provides guidance to the system while learning, shortening the number of cycles required to learn. These tools and the issues they raise are first studied in simulation, and then the experience gained with simulations is used to implement the learning system on the real robot. Results have shown that with this approach it is possible to let the AutonoMouse, a small real robot, learn to approach a light source under a number of different noise and lesion conditions.This work was partially written while the author was at International Computer Science Institute, 1947 Center Street, Suite 600, Berkeley, 94704-1198 California, USA.     

可持续自主学习的micROS机器人操作系统平行学习架构

作为机器人平台最重要的基础软件,机器人操作系统是提高机器人自主性与智能化水平的核心和关键.围绕实现适应环境的智能机器人系统这一目标,基于已有的micROS研究,提出了可持续自主学习的群体智能机器人操作系统平行学习架构,描述了架构设计、核心概念、实现途径和应用验证.在micROS可扩展分布式层次架构的基础上,提出了支持可持续自主学习的平行学习架构,设计并实现了机器人操作系统的两大核心概念——基于“角色”的控制抽象和基于“语义情境图”的数据抽象,突破了群体智能行为操控、自组织无线网络等群体机器人自主智能协同急需解决的关键技术问题,在此基础上开展了面向多种场景的应用验证.     

面向深度学习图像分类的GPU并行方法研究

针对深度学习图像分类场景中多GPU并行后传输效率低的问题,提出一种低时间复杂度的Ring All Reduce改进算法。通过分节点间隔配对原则优化数据传输流程,缓解传统参数服务器并行结构的带宽损耗。基于数据并行难以支撑大规模网络参数及加速延缓的问题,根据深度学习主干网络所包含的权重参数低于全连接层权重参数、同步开销小、全连接层权重大与梯度传输开销过高等特点,提出GPU混合并行优化算法,将主干网络进行数据并行,全连接层进行模型并行,并通过改进的Ring All Reduce算法实现各节点之间的并行后数据通信,用于基于深度学习模型的图像分类。在Cifar10和mini ImageNet两个公共数据集上的实验结果表明,该算法在保持分类精度不变的情况下可以获得更好的加速效果,相比数据并行方法,可达到近45%的提升效果。     

强化学习算法与应用综述

强化学习是机器学习领域的研究热点,是考察智能体与环境的相互作用,做出序列决策、优化策略并最大化累积回报的过程.强化学习具有巨大的研究价值和应用潜力,是实现通用人工智能的关键步骤.本文综述了强化学习算法与应用的研究进展和发展动态,首先介绍强化学习的基本原理,包括马尔可夫决策过程、价值函数、探索-利用问题.其次,回顾强化学习经典算法,包括基于价值函数的强化学习算法、基于策略搜索的强化学习算法、结合价值函数和策略搜索的强化学习算法,以及综述强化学习前沿研究,主要介绍多智能体强化学习和元强化学习方向.最后综述强化学习在游戏对抗、机器人控制、城市交通和商业等领域的成功应用,以及总结与展望.     

A Survey and Systematic Categorization of Parallel K-Means and Fuzzy-C-Means Algorithms

Parallel processing has turned into one of the emerging fields of machine learning due to providing consistent work by performing several tasks simultaneously,enhancing reliability (the presence of more than one device ensures the workflow even if some devices disrupted), saving processing time and introducinglow cost and high-performance computation units. This research study presents a survey of parallel K-means and Fuzzy-c-means clustering algorithmsbased on their implementations in parallel environments such as Hadoop, MapReduce, Graphical Processing Units, and multi-core systems. Additionally,the enhancement in parallel clustering algorithms is investigated as hybrid approaches in which K-means and Fuzzy-c-means clustering algorithms areintegrated with metaheuristic and other traditional algorithms.