Data-Driven Human-Robot Interaction Without Velocity Measurement Using Off-Policy Reinforcement Learning

In this paper,we present a novel data-driven design method for the human-robot interaction(HRI)system,where a given task is achieved by cooperation between the human and the robot.The presented HRI controller design is a two-level control design approach consisting of a task-oriented performance optimization design and a plant-oriented impedance controller design.The task-oriented design minimizes the human effort and guarantees the perfect task tracking in the outer-loop,while the plant-oriented achieves the desired impedance from the human to the robot manipulator end-effector in the inner-loop.Data-driven reinforcement learning techniques are used for performance optimization in the outer-loop to assign the optimal impedance parameters.In the inner-loop,a velocity-free filter is designed to avoid the requirement of end-effector velocity measurement.On this basis,an adaptive controller is designed to achieve the desired impedance of the robot manipulator in the task space.The simulation and experiment of a robot manipulator are conducted to verify the efficacy of the presented HRI design framework.     

An experimental study on the optimization of controller gains for an electro-hydraulic servo system using evolution strategies

This paper deals with an experimental optimization problem of the controller gains for an electro-hydraulic position control system through evolution strategies (ESs)-based method. The optimal controller gains for the control system are obtained by maximizing fitness function designed specially to evaluate the system performance. In this paper, for an electro-hydraulic position control system which would represent a hydraulic mill stand for the roll-gap control in plate hot-rollings, the time delay controller (TDC) is designed, and three control parameters of this controller are directly optimized through a series of experiments using this method. It is shown that the near-optimal value of the controller gains is obtained in about 5th generation, which corresponds to approximately 150 experiments. The optimal controller gains are experimentally confirmed by inspecting the fitness function topologies that represent system performance in the gain spaces. It is found that there are some local optimums on a fitness function topology so that the optimization of the three control parameters of a TDC by manual tuning could be a task of great difficulty. The optimized results via the ES coincide with the maximum peak point in topologies. It is also shown that the proposed method is an efficient scheme giving economy of time and labor in optimizing the controller gains of fluid power systems experimentally.     

Near Optimal PID Controllers for the Biped Robot While Walking on Uneven Terrains

The execution of the gaits generated with the help of a gait planner is a crucial task in biped locomotion. This task is to be achieved with the help of a suitable torque based controller to ensure smooth walk of the biped robot. It is important to note that the success of the developed proportion integration differentiation (PID) controller depends on the selected gains of the controller. In the present study, an attempt is made to tune the gains of the PID controller for the biped robot ascending and descending the stair case and sloping surface with the help of two non-traditional optimization algorithms, namely modified chaotic invasive weed optimization (MCIWO) and particle swarm optimization (PSO) algorithms. Once the optimal PID controllers are developed, a simulation study has been conducted in computer for obtaining the optimal tuning parameters of the controller of the biped robot. Finally, the optimal gait angles obtained by using the best controller are fed to the real biped robot and found that the biped robot has successfully negotiated the said terrains.     

Adaptive control for uncertain nonlinear systems based on multiple neural networks.

A new adaptive multiple neural network controller (AMNNC) with a supervisory controller for a class of uncertain nonlinear dynamic systems was developed in this paper. The AMNNC is a kind of adaptive feedback linearizing controller where nonlinearity terms are approximated with multiple neural networks. The weighted sum of the multiple neural networks was used to approximate system nonlinearity for the given task. Each neural network represents the system dynamics for each task. For a job where some tasks are repeated but information on the load is not defined and unknown or varying, the proposed controller is effective because of its capability to memorize control skill for each task with each neural network. For a new task, most similar existing control skills may be used as a starting point of adaptation. With the help of a supervisory controller, the resulting closed-loop system is globally stable in the sense that all signals involved are uniformly bounded. Simulation results on a cartpole system for the changing mass of the pole were illustrated to show the effectiveness of the proposed control scheme for the comparison with the conventional adaptive neural network controller (ANNC).     

Bridging the gap between the linear and nonlinear predictive control: Adaptations for efficient building climate control

The linear model predictive control which is frequently used for building climate control benefits from the fact that the resulting optimization task is convex (thus easily and quickly solvable). On the other hand, the nonlinear model predictive control enables the use of a more detailed nonlinear model and it takes advantage of the fact that it addresses the optimization task more directly, however, it requires a more computationally complex algorithm for solving the non-convex optimization problem. In this paper, the gap between the linear and the nonlinear one is bridged by introducing a predictive controller with linear time-dependent model. Making use of linear time-dependent model of the building, the newly proposed controller obtains predictions which are closer to reality than those of linear time invariant model, however, the computational complexity is still kept low since the optimization task remains convex. The concept of linear time-dependent predictive controller is verified on a set of numerical experiments performed using a high fidelity model created in a building simulation environment and compared to the previously mentioned alternatives. Furthermore, the model for the nonlinear variant is identified using an adaptation of the existing model predictive control relevant identification method and the optimization algorithm for the nonlinear predictive controller is adapted such that it can handle also restrictions on discrete-valued nature of the manipulated variables. The presented comparisons show that the current adaptations lead to more efficient building climate control.     

Adaptive fuzzy model based inverse controller design using BB-BC optimization algorithm

The use of inverse system model as a controller might be an efficient way in controlling non-linear systems. It is also a known fact that fuzzy logic modeling is a powerful tool in representing nonlinear systems. Therefore, inverse fuzzy model can be used as a controller for controlling nonlinear plants. In this context, firstly, a new fuzzy model based inverse controller design methodology is presented in this study. The design methodology introduced here is based on a recursive optimization procedure that searches for an optimal inverse model control signal at every sampling time. Since the task of optimization should be accomplished in between two sampling periods the use of a fast optimization algorithm becomes essential. For this reason, Big Bang-Big Crunch (BB-BC) optimization algorithm is used due to its low computational time and high global convergence properties. Even though, inverse model controllers may produce perfect control while operating in an open loop fashion, this open loop control would not be sufficient in the case of modeling mismatches or disturbances that might occur over the system. In order to overcome this problem, secondly, an on-line adaptation mechanism via BB-BC optimization algorithm is introduced in addition to BB-BC optimization based fuzzy model inverse controller. The adaptation mechanism is used to update the related parameters of the model while minimizing the absolute value of the instantaneous error between the system and model outputs. In this manner, the system output is somehow fed back, the overall control form can be considered as a closed-loop system. The new fuzzy model based inverse control scheme with the new online adaptation mechanism has been implemented and tested on the two real time processes; namely, heat transfer and pH processes and very satisfactory results has been reported.     

Chaotic Evolution: fusion of chaotic ergodicity and evolutionary iteration for optimization

We propose a novel population-based optimization algorithm, Chaotic Evolution (CE), which uses ergodic property of chaos to implement exploration and exploitation functions of an evolutionary algorithm. CE introduces a mathematical mechanism into an iterative process of evolution and simulates ergodic motion in a search space with a simple principle. A control parameter, direction factor rate, is proposed to guide search direction in CE. It is easy to extend its search capability by using different chaotic system in CE algorithm framework. The scalability of CE is higher than that of some other evolutionary computation algorithms. A series of comparative evaluations and investigations is conducted to analyse characteristics of the proposal. Our proposal can obtain better optimization performance by comparing with differential evolution and some of its variants. We point out that the chaos theory is used not only to describe and explain a non-linear system, but also to implement a variety of optimization algorithms based on its ergodic property.     

基于改进QPSO算法的电动汽车模糊控制器参数优化

目前电动汽车常以无刷直流电机（BLDCM）作为驱动器,但BLDCM调速控制系统中模糊控制器的量化因子和比例因子采用传统方法,自调节能力弱,针对该问题提出一种改进QPSO算法（AMF-QPSO）实现对量化因子和比例因子的自适应调节。AMF-QPSO算法以收缩—扩张系数（contraction expansion,CE）控制方式为研究重点,提出粒子活性概念,并以其作为反馈量实现动态自适应调节CE系数; 同时,为防止种群高度聚集,采用精英群体随机交叉学习机制,对部分活性低的精英粒子进行扰动,增强种群后期多样性。最后,通过LabVIEW实验平台,以具体案例验证AMF-QPSO算法性能。实验结果表明,AMF-QPSO优化的模糊PID控制器具有比标准模糊PID控制器和QPSO优化的模糊PID控制器更好的控制性和自适应性。     

Optimal interval controller design for uncertain systems

The interval controller design is a hot issue for uncertain systems, whereas how to design an optimal interval controller under the premise of ensuring system stability is a difficult problem that needs further study. This paper mainly aims at the single input single output uncertain system to propose an optimal interval controller based on the Kharitonov theorem and an interval optimization algorithm, which can guarantee the stability and optimization of a closed-loop interval system. According to the Kharitonov theorem, the optimal interval controller design can be transformed into an optimal controller synthesis issue of multiple vertex objects. An interval particle swarm optimization (IPSO) algorithm is then used to optimize the quadratic performance index with interval variables for each vertex object to obtain the solution domains of the controller parameters, and the vertex method is utilized to prevent interval width expansion or divergence in the iteration. Finally, the intersections of the solution domains for all vertex objects are obtained as the optimal interval solution of interval controller parameters. In addition, the stability verification approach of the closed-loop system and the empirical rule to select the interval particle width are given. Simulation results for typical examples demonstrate that the designed interval controller not only performs optimally but also can robustly stabilize the interval system.     

Performance analysis of fractional order fuzzy PID controllers applied to a robotic manipulator

A two-link robotic manipulator is a Multi-Input Multi-Output (MIMO), highly nonlinear and coupled system. Therefore, designing an efficient controller for this system is a challenging task for the control engineers. In this paper, the Fractional Order Fuzzy Proportional-Integral-Derivative (FOFPID) controller for a two-link planar rigid robotic manipulator for trajectory tracking problem is investigated. Robustness testing of FOFPID controller for model uncertainties, disturbance rejection and noise suppression is also investigated. To study the effectiveness of FOFPID controller, its performance is compared with other three controllers namely Fuzzy PID (FPID), Fractional Order PID (FOPID) and conventional PID. For tuning of parameters of all the controllers, Cuckoo Search Algorithm (CSA) optimization technique was used. Two performance indices namely Integral of Absolute Error (IAE) and Integral of Absolute Change in Controller Output (IACCO) having equal weightage for both the links are considered for minimization. Numerical simulation results clearly indicate the superiority of FOFPID controller over the other controllers for trajectory tracking, model uncertainties, disturbance rejection and noise suppression.     

PI controller design using ABC algorithm for MPPT of PV system supplying DC motor pump load

Maximum power point tracking (MPPT) is used in photovoltaic (PV) systems to maximize its output power. A new MPPT system has been suggested for PV–DC motor pump system by designing two PI controllers. The first one is used to reach MPPT by monitoring the voltage and current of the PV array and adjusting the duty cycle of the DC/DC converter. The second PI controller is designed for speed control of DC series motor by setting the voltage fed to the DC series motor through another DC/DC converter. The suggested design problem of MPPT and speed controller is formulated as an optimization task which is solved by artificial bee colony (ABC) to search for optimal parameters of PI controllers. Simulation results have shown the validity of the developed technique in delivering MPPT to DC series motor pump system under atmospheric conditions and tracking the reference speed of motor. Moreover, the performance of the ABC algorithm is compared with genetic algorithm for various disturbances to prove its robustness.

     

Robust SVC controller design and analysis for uncertain power systems

A Static Var Compensator (SVC) installed in a power transmission network can be effectively used to enhance the damping of electromechanical oscillations [Schweickardt, H. E., Romegialli, G., & Reichert, K. (1978). Closed loop control of static VAR sources (SVS) on EHV transmission lines. IEEE Pes winter power meeting, (paper no A78, pp. 135–136), New York, Jan. 29–Feb. 3]. An adequately designed robust controller, which takes into account variations in the operating conditions, can help to achieve the desired damping control. The proposed approach described in this paper is aimed to achieve damping of electromechanical oscillations by considering a systematic approach, based on interval systems theory and Kharitonov's Theorem. The method presented allows for the design of a fixed-parameter, low-order controller, given a supposed stability degree of the system. The synthesis of a robust SVC controller is divided into two tasks. The first is the determination of the region of stability in the controller parameter plane by plotting the stability boundary locus. The second task is the optimization of the selected controller parameters from the obtained solutions to the first task. Examples of eigenvalue analysis and time simulation demonstrate the effectiveness and robustness of the designed controller.     

A novel fused neural network controller for lateral control of autonomous vehicles

We present a novel fused feed-forward neural network controller inspired by the notion of task decomposition principle. The controller is structurally simple and can be applied to a class of control systems that their control requires manipulation of two input variables. The benchmark problem of inverted pendulum is such example that its control requires availability of the angle as well as the displacement. We demonstrate that the lateral control of autonomous vehicles belongs to this class of systems and successfully apply the proposed controller to this problem. The parameters of the controller are encoded into real value chromosomes for genetic algorithm (GA) optimization. The neural network controller contains three neurons and six connection weights implying a small search space implying faster optimization time due to few controller parameters. The controller is also tested on two benchmark control problems of inverted pendulum and the ball-and-beam system. In particular, we apply the controller to lateral control of a prototype semi-autonomous vehicle. Simulation results suggest a good performance for all the tested systems. To demonstrate the robustness of the controller, we conduct Monte-Carlo evaluations when the system is subjected to random parameter uncertainty. Finally experimental studies on the lateral control of a prototype autonomous vehicle with different speed of operation are included. The simulation and experimental studies suggest the feasibility of this controller for numerous applications.     

基于交叉熵算法的PID 控制器设计

交叉熵优化方法是一种新型高效的随机优化算法,算法控制参数简单,鲁棒性强.将交叉熵优化算法用于PID控制器的参数设计,并与基于遗传算法的PID控制器设计进行对比,结果表明,交叉熵优化算法不仅所获结果较优,而且计算复杂度也明显小于遗传算法.     

Evolving an interval type-2 fuzzy PID controller for the redundant robotic manipulator

Robotic manipulators are a multi-input multi-output, dynamically coupled, highly time-varying, complex and highly nonlinear systems wherein the external disturbances, parameter variations, and random noise adversely affects the performance of the robotic system. Therefore, in order to deal with such complexities, however, an intriguing task for control researchers, these systems require an efficient and robust controller. In this paper, a novel application of genetic algorithms (GA) optimization approach to optimize the scaling factors of interval type-2 fuzzy proportional derivative plus integral (IT2FPD+I) controllers is proposed for 5-DOF redundant robot manipulator for trajectory tracking task. All five controllers' parameters are optimized simultaneously. Further, a procedure for selecting appropriate initial search space is also demonstrated. In order to make a fair comparison between different controllers, the tuning of each of the controllers' parameters is done with GA. This optimization technique uses the time domain optimal tuning while minimizing the fitness function as the sum of integral of multiplication of time with square error (ITSE) for each joint. To ascertain the effectiveness of IT2FPID controller, it is compared against type-1 fuzzy PID (T1FPID) and conventional PID controllers. Furthermore, robustness testing of developed IT2FPID controller for external disturbances, parameter variations, and random noise rejection is also investigated. Finally, the experimental study leads us to claim that our proposed controller can not only assure best trajectory tracking in joint and Cartesian space, but also improves the robustness of the systems for external disturbances, parameter variations, and random noise.     

Incremental Evolutionary Design of TSK Fuzzy Controllers

This paper proposes a novel method for the incremental design and optimization of first order Tagaki-Sugeno-Kang (TSK) fuzzy controllers by means of an evolutionary algorithm. Starting with a single linear control law, the controller structure is gradually refined during the evolution. Structural augmentation is intertwined with evolutionary adaptation of the additional parameters with the objective not only to improve the control performance but also to maximize the stability region of the nonlinear system. From the viewpoint of optimization the proposed method follows a divide-and-conquer approach. Additional rules and their parameters are introduced into the controller structure in a neutral fashion, such that the adaptations of the less complex controller in the previous stage are initially preserved. The proposed scheme is evaluated at the task of TSK fuzzy controller design for the upswing and stabilization of a rotational inverted pendulum. In the first case, the objective is a time optimal controller that upswings the pendulum in to the upper equilibrium point in shortest time. The stabilizing controller is designed as a state optimal controller. In a second application the optimization method is applied to the design of a fuzzy controller for vision-based mobile robot navigation. The results demonstrate that the incremental scheme generates solutions that are similar in control performance to pure parameter optimization of only the gains of a TSK system. Even more important, whereas direct optimization of control systems with more than 35 rules fails to identify a stabilizing control law, the incremental scheme optimizes fuzzy state-space partitions and gains for hundreds of rules.     

Optimum design of fractional order PID controller for AVR system using chaotic ant swarm

Fractional-order PID (FOPID) controller is a generalization of standard PID controller using fractional calculus. Compared to PID controller, the tuning of FOPID is more complex and remains a challenge problem. This paper focuses on the design of FOPID controller using chaotic ant swarm (CAS) optimization method. The tuning of FOPID controller is formulated as a nonlinear optimization problem, in which the objective function is composed of overshoot, steady-state error, raising time and settling time. CAS algorithm, a newly developed evolutionary algorithm inspired by the chaotic behavior of individual ant and the self-organization of ant swarm, is used as the optimizer to search the best parameters of FOPID controller. The designed CAS-FOPID controller is applied to an automatic regulator voltage (AVR) system. Numerous numerical simulations and comparisons with other FOPID/PID controllers show that the CAS-FOPID controller can not only ensure good control performance with respect to reference input but also improve the system robustness with respect to model uncertainties.     

基于GSPN和人工免疫算法的并行测试任务调度研究

针对ATS中并行测试任务调度复杂、难以优化的问题,提出了一种广义随机Petri网和人工免疫算法相结合的任务调度优化算法.首先对并行测试系统建立广义随机Petri网(GSPN)模型,然后将激发的变迁序列集作为并行测试任务调度路径;将免疫克隆选择算法(ICSA)应用到并行测试系统任务调度问题中,并提出一种自适应克隆选择算子,搜索最优任务调度路径,得到以测试时间最短为目标的最优任务调度方案.用某型雷达接收机并行测试系统对该算法进行仿真验证,结果表明,与改进的混合遗传算法(IHGA)相比,该算法能够便捷地得到任务调度最优序列,且测试效率更高.     

基于Windows CE3.0的嵌入式控制器研究

嵌入式控制器硬件平台采用PC/104微处理器模块，PC/104微处理器和外围设备构成堆栈式结构。Windows CE3.0为嵌入式控制器提供了功能强大的实时操作系统。文章详细分析了Windows CE3.0的特点，根据工业控制对控制系统在网络通信、组态软件、开放性和可靠性方面的要求，设计出适合工业环境的基于Windows CE3.0的嵌入式控制器。     

Differential evolution of fuzzy controller for environmentally-powered wireless sensors

Environmentally-powered wireless sensors use ambient energy from their environment to support their own energy needs. As such, they must operate without significant maintenance or user supervision. Due to the stochastic availability of ambient energy, its harvesting, storage and consumption must be managed by an efficient and robust controller that maintains data collection and transmission rates at desired levels, while maximizing the useful operational time of the system. To accomplish this task, the control system must observe the state of charge of an internal energy storage device, and consider the amount of energy available for harvest in the future. At the same time, the complexity of the controller must be limited so that it can be implemented on the simple embedded system of the sensor hardware. This paper presents a comprehensive synthesis of desired behavior of such controllers, and describes procedures for their design and optimization through an evolutionary fuzzy approach. The main contribution is the formalization of design objectives and development of the fitness function that drives the optimization process. Additional contributions include a comprehensive evaluation of several soft computing optimization approaches, thorough analysis of the optimized controller, its comparison to baseline control strategies, and validation of its operation with real energy availability forecasts.