Reinforcement learning based resource allocation in business process management

Efficient resource allocation is a complex and dynamic task in business process management. Although a wide variety of mechanisms are emerging to support resource allocation in business process execution, these approaches do not consider performance optimization. This paper introduces a mechanism in which the resource allocation optimization problem is modeled as Markov decision processes and solved using reinforcement learning. The proposed mechanism observes its environment to learn appropriate policies which optimize resource allocation in business process execution. The experimental results indicate that the proposed approach outperforms well known heuristic or hand-coded strategies, and may improve the current state of business process management.     

Fuzzy Q-learning in continuous state and action space

An adaptive fuzzy Q-learning(AFQL)based on fuzzy inference systems(FIS)is proposed.The FIS realized by a normalized radial basis function(NRBF)neural network is used to approach Q-value function,whose input is composed of state and action.The rules of FIS are created incrementally according to the novelty of each element of the pair of state-action.Moreover the premise part and consequent part of the FIS are updated using extended Kalman filter(EKF).The action that impacts on environment is the one with maximum output of FIS in the current state and generated through optimization method.Simulation results in the wall-following task of mobile robots and the inverted pendulum balancing problem demonstrate that the superiority and applicability of the proposed AFQL method.     

Intelligent AGV driving toward an autonomous decentralized manufacturing system

This paper proposes two methods that give intelligence to automatically guided vehicles (AGVs). In order to drive AGVs autonomously, two types of problems need to be overcome. They are the AGV navigation problem and collision avoidance problem. The first problem has been well known since 1980s. A new method based on the feature scene recognition and acquisition is proposed. The sparse distributed memory neural network (SDM) is employed for the scene recognition and acquisition. The navigation route for the AGV is learnt by use of Q-learning depending on the recognized and acquired scenes. The second problem is described as mutual understanding of behaviors between AGVs. The method of mutual understanding is proposed by the use of Q-learning. Those two methods are combined together for driving plural AGVs autonomously to deliver raw materials between machine tools in a factory. They are incorporated into the AGVs as the machine intelligence. In experimental simulations, it is verified that the first proposed method can guide the AGV to the suitable navigation and that the second method can acquire knowledge of mutual understanding of the AGVs’ behaviors.     

基于模糊Q-学习的多智能体协作策略研究

研究了对机器人足球中的非控球队员同控球队员的协作问题．机器人根据当前状态下机器人和球的位置信息决定目标位置．对目标位置的确定进行了Q学习,对于场地的划分和角色的分配,采用了模糊理论,并对学习结果进行了仿真试验．     

基于Q学习蚁群算法的热网优化方法

以热网最小年费用作为目标函数,引入基于Q学习规则的蚁群算法,建立了热网优化算法。结合算例,比较了比摩阻算法、模拟退火算法、基于Q学习规则蚁群算法,基于Q学习规则蚁群算法的热网最小年费用最低。     

基于强化学习的交叉口交通低排放信号控制研究

交叉口车辆排放较为复杂,尤其是在考虑初始排队长度的情况下,更是难以建立明确的数学模型。Q学习是一种无模型的强化学习算法,通过与环境的试错交互学习最优控制策略。本文提出了一种基于Q学习的交通排放信号控制方案。利用仿真平台USTCMTS2.0,通过不断地试错学习找到在不同相位排队长度下最优配时。在Q学习中添加了模糊初始化Q函数的方法以改进Q学习的收敛速度,加速了学习过程。仿真实验结果表明:强化学习算法取得较好的效果。相比较Hideki的方法,在车流量较高时,车辆平均排放量减少了13.9%,并且对Q函数值的模糊初始化大大加速了Q函数收敛的过程。     

Supplier's optimal bidding strategy in electricity pay-as-bid auction: Comparison of the Q-learning and a model-based approach

In this paper, the bidding decision making problem in electricity pay-as-bid auction is studied from a supplier's point of view. The bidding problem is a complicated task, because of suppliers’ uncertain behaviors and demand fluctuation. In a specific case, in which, the market clearing price (MCP) is considered as a continuous random variable with a known probability distribution function (PDF), an analytic solution is proposed. The suggested solution is generalized to consider the effect of supplier market power due to transmission congestion. As a result, an algebraic equation is developed to compute optimal offering price. The basic assumption in this approach is to take the known probabilistic model for the MCP.