基于内在动机的智能机器人自主发育算法

针对两轮自平衡机器人在学习过程中主动性差的问题,受心理学内在动机理论启发,提出一种基于内在动机的智能机器人自主发育算法。该算法在强化学习的理论框架中,引入模拟人类好奇心的内在动机理论作为内部驱动力,与外部奖赏信号一起作用于整个学习过程。采用双层内部回归神经网络存储知识的学习与积累,使机器人逐步学会自主平衡技能。最后针对测量噪声污染对机器人平衡控制中两轮角速度的影响,进一步采用卡尔曼滤波方法进行补偿,以提高算法收敛速度,降低系统误差。仿真实验表明,该算法能够使两轮机器人通过与环境的交互获得认知,成功地学会运动平衡控制技能。     

Autonomous and fast robot learning through motivation

Research on robot techniques that are fast, user-friendly, and require little application-specific knowledge by the user, is more and more encouraged in a society where the demand of home-care or domestic-service robots is increasing continuously. In this context we propose a methodology which combines reinforcement learning and genetic algorithms to teach a robot how to perform a task when only the specification of the main restrictions of the desired behaviour is provided. Through this combination, both paradigms must be merged in such a way that they influence each other to achieve a fast convergence towards a good robot-control policy, and reduce the random explorations the robot needs to carry out in order to find a solution.Another advantage of our proposal is that it is able to easily incorporate any kind of domain-dependent knowledge about the task. This is very useful for improving a robot controller, for applying a robot-controller to move a different robot-platform, or when we have certain “feelings” about how the task should be solved.The performance of our proposal is shown through its application to solve a common problem in mobile robotics.     

基于内发动机机制的移动机器人自主路径规划方法

基于内发动机机制,为移动机器人建立一种新的路径规划方法.将已有内发动机机制中基于状态的好奇心函数扩展为基于动作的好奇心函数,并建立相应的动作选择机制,更符合生物可解释性.设计障碍物分布环境下的移动机器人状态能量函数,用于决定学习的方向.实验结果表明,所建立的方法能够有效地帮助机器人学习环境知识,实现不同初始状态下的避障导航任务.同时,能量函数的设计不依赖于具体环境,即使目标点发生改变,机器人也能通过重新学习到达目标,体现出方法的高度自主性和非任务性.     

内部动机驱动的机器人未知环境在线自主学习

针对目前智能移动机器人在未知环境中学习遇到的如学习主动性、实时性差,无法在线积累学习的知识和经验等问题,受心理学中内部动机的启发,提出一种内部动机驱动的移动机器人未知环境在线自主学习方法,在一定程度上弥补目前该领域存在的一些问题。该方法通过在移动机器人Q学习的框架下,将奖励机制用基于心理学启发的内部动机取代,提高其对于未知环境的学习主动性,同时,采用增量自组织神经网络代替经典Q学习中的查找表,实现输入输出空间的映射,使得机器人能够在线增量地学习未知环境。实验结果表明,通过内部动机驱动的方法,移动机器人对于未知环境的学习主动性得到了提高,智能程度有了明显改进。     

Acceptance of Internet-based learning medium: the role of extrinsic and intrinsic motivation

This study is one of the few attempts to investigate students’ acceptance of an Internet-based learning medium (ILM). By integrating a motivational perspective into the technology acceptance model, our model captured both extrinsic (perceived usefulness and ease of use) and intrinsic (perceived enjoyment) motivators for explaining students’ intention to use the new learning medium. Data collected from 544 undergraduate students were examined through the LISREL VIII framework. The results showed that both perceived usefulness and perceived enjoyment significantly and directly impacted their intention to use ILM. Surprisingly, perceive ease of use did not posit a significant impact on student attitude or intention towards ILM usage. Implications of this study are important for both researchers and practitioners.     

Visual approach skill for a mobile robot using learning and fusion of simple skills

This paper presents a reinforcement learning algorithm which allows a robot, with a single camera mounted on a pan tilt platform, to learn simple skills such as watch and orientation and to obtain the complex skill called approach combining the previously learned ones. The reinforcement signal the robot receives is a real continuous value so it is not necessary to estimate an expected reward. Skills are implemented with a generic structure which permits complex skill creation from sequencing, output addition and data flow of available simple skills.     

Specificity of motor learning in simulator training of endoscopic-surgery skills

The introduction of simulators for the practice of endoscopic-surgery sensori-motor skills opens a wide range of design options. An obvious one is augmented visual information early in practice, in particular a direct view of the site instead of the endoscopic view. We studied the effects of such augmented visual information on the simulated ablation of tissue with straight, horizontal and parallel cuts. Direct view had an immediate beneficial effect on performance as compared with endoscopic-view practice. However, in subsequent tests with endoscopic view the benefits disappeared and turned into costs for some aspects of performance, e.g., duration. This finding highlights for a simulated surgical task that optimisation of practice by a performance criterion may not result in optimisation by a transfer criterion.

Practitioner Summary: Endoscopic surgery represents a challenge for human sensori-motor skills, but new simulator-based training methods give leeway for optimisation. A candidate is augmented visual feedback, in particular a direct rather than endoscopic view of the site. However, performance becomes dependent on the augmented feedback so that the costs outweigh the benefits.     

The effects of peer intrinsic and extrinsic motivation on MMOG game-based collaborative learning

We combined Massively Multiplayer Online Game and technology-based collaborative learning methods to examine peer motivational factors influencing intention to learn; these have seldom been jointly examined. We proposed two new constructs, peer intrinsic motivation and peer extrinsic motivation, and investigated their effect on a player's intention to learn individually and collaboratively. Our survey and interview findings showed that an individual player's peer intrinsic and extrinsic motivations had significantly positive influence on his or her intention to learn collaboratively and individually. Implications for academics, educators, game developers, and players are discussed.     

Level of intrinsic motivation of distance education students in e‐learning environments

According to researches, motivation that initiates and sustains behaviour is one of the most significant components of learning in any environment. Accordingly, level of intrinsic motivation triggers and sustains the interest of the open and distance education students when it comes to learning on their own in e‐learning environments. Despite a comprehensive literature regarding the motivation of those learning in traditional learning environments, the number of studies addressing the motivation of open and distance education students in e‐learning environments is not sufficient. In this context, this study aims at determining the level of intrinsic motivation of open and distance education students. Thus, data were collected from 1,639 distance education students in 22 programmes, through Intrinsic Motivation in e‐Learning Questionnaire developed and validated to that end. Analyses carried out indicate that the level of intrinsic motivation of open and distance education students is high in e‐learning environments, but there is not a statistically significant difference by gender, programme structure (graduate/undergraduate), instruction type (distance–blended), and academic disciplines.     

The effects of social messaging on students' learning and intrinsic motivation in peer assessment

This study aims to gain a better understanding of how the newly arisen social messaging may impact the practice of peer assessment. Seventy-nine ESL (English as second language) students reviewed each other's English essays in three peer assessment groups: a three-member group using wiki (wiki group), a three-member group using social messaging (small messaging group), and a six-member group using social messaging (big messaging group). Data analysis suggested that peer assessment facilitated by social messaging can be at least of the same effectiveness as wiki-facilitated peer assessment on ESL students' writing skills and intrinsic motivation. In addition, the findings indicated that students in the small messaging group outperformed students in the big messaging group on essay writing and reported a significantly higher rating on perceived competence, a positive indicator of the behavioural measures of intrinsic motivation, than students in the big messaging group.     

基于学习自动机的具有内发动机的感知运动系统的建立

以学习自动机为数学模型, 结合斯金纳操作条件反射, 建立一种人工感知运动系统, 称为感知运动自动机(SMA). 该系统包括感知状态集合、动作集合、感知运动取向性映射集合等9 部分. 系统引入好奇心和取向性概念, 设计具有主动学习环境的内发动机机制, 定义并分析了取向性学习过程, 证明了系统熵的收敛性. 通过模拟斯金纳鸽子实验表明了系统的可行性和有效性, 仿真结果表明系统具有较好的自学习和自组织特性, 同时稳定性较高.

     

Reinforcement learning for robot soccer

Batch reinforcement learning methods provide a powerful framework for learning efficiently and effectively in autonomous robots. The paper reviews some recent work of the authors aiming at the successful application of reinforcement learning in a challenging and complex domain. It discusses several variants of the general batch learning framework, particularly tailored to the use of multilayer perceptrons to approximate value functions over continuous state spaces. The batch learning framework is successfully used to learn crucial skills in our soccer-playing robots participating in the RoboCup competitions. This is demonstrated on three different case studies.

Instructional animations can be superior to statics when learning human motor skills

Based on the assumption of a working memory processor devoted to human movement, cognitive load theory is used to explore some conditions under which animated instructions are hypothesised to be more effective for learning than equivalent static graphics. Using paper-folding tasks dealing with human movement, results from three experiments confirmed our hypothesis, indicating a superiority of animation over static graphics. These results are discussed in terms of a working memory processor that may be facilitated by our mirror-neuron system and may explain why animated instructional animations are superior to static graphics for cognitively based tasks that involve human movement.     

Automatic recognition vision system guided for apple harvesting robot

In apple harvesting robot, the first key part is the machine vision system, which is used to recognize and locate the apples. In this paper, the procedure on how to develop an automatic recognition vision system guided for apple harvesting robot, is proposed. We first use a color charge coupled device camera to capture apple images, and then utilize an industrial computer to process images for recognising fruit. Meanwhile, the vector median filter is applied to remove the color images noise of apple, and images segmentation method based on region growing and color feature is investigated. After that the color feature and shape feature of image are extract, a new classification algorithm based on support vector machine for apple recognition is introduced to improve recognition accuracy and efficiency. Finally, these procedures proposed have been tested on apple harvesting robot under natural conditions in September 2009, and showed a recognition success rate of approximately 89% and average recognition time of 352 ms.     

Effects of conformity and learning anxiety on intrinsic and extrinsic motivation: the case of Facebook course groups

Universal Access in the Information Society - The objective of this study is to enhance our understanding of whether learners’ conformity behaviors and learning anxiety can affect their...     

一种结合内在动机理论的移动机器人环境认知模型

为了探讨动机在环境认知中所起作用,模仿生物环境认知行为,以学习自动机为框架,提出一种结合内在动机理论的移动机器人环境认知模型(CM-LA).该模型以访问次数及访问时间间隔共同定义内在动机,并以操作条件反射原理描述外在动机的影响.同时,所设计的动作选择机制可以使机器人根据已有的环境认知对新增障碍物进行有效避让.实验结果表...     

迭代学习神经网络控制在机器人示教学习中的应用

示教学习是机器人运动技能获取的一种高效手段.当采用摄像机作为示教轨迹记录部件时,示教学习涉及如何通过反复尝试获得未知机器人摄像机模型问题.本文力图针对非线性系统重复作业中的可重复不确定性学习,提出一个迭代学习神经网络控制方案,该控制器将保证系统最大跟踪误差维持在神经网络有效近似域内.为此提出了一个适合于重复作业应用的分布式神经网络结构.该神经网络由沿期望轨线分布的一系列局部神经网络构成,每一局部神经网络对对应期望轨迹点邻域进行近似并通过重复作业完成网络训练.由于所设计的局部神经网络相互独立,因此一个全程轨迹可以通过分段训练完成,由起始段到结束段,逐段实现期望轨迹的准确跟踪.该方法在具有未知机器人摄像机模型的轨迹示教模仿中得到验证,显示了它是一种高效的训练方法,同时具有一致的误差限界能力.     

Supervised fuzzy reinforcement learning for robot navigation

This paper addresses a new method for combination of supervised learning and reinforcement learning (RL). Applying supervised learning in robot navigation encounters serious challenges such as inconsistent and noisy data, difficulty for gathering training data, and high error in training data. RL capabilities such as training only by one evaluation scalar signal, and high degree of exploration have encouraged researchers to use RL in robot navigation problem. However, RL algorithms are time consuming as well as suffer from high failure rate in the training phase. Here, we propose Supervised Fuzzy Sarsa Learning (SFSL) as a novel idea for utilizing advantages of both supervised and reinforcement learning algorithms. A zero order Takagi–Sugeno fuzzy controller with some candidate actions for each rule is considered as the main module of robot's controller. The aim of training is to find the best action for each fuzzy rule. In the first step, a human supervisor drives an E-puck robot within the environment and the training data are gathered. In the second step as a hard tuning, the training data are used for initializing the value (worth) of each candidate action in the fuzzy rules. Afterwards, the fuzzy Sarsa learning module, as a critic-only based fuzzy reinforcement learner, fine tunes the parameters of conclusion parts of the fuzzy controller online. The proposed algorithm is used for driving E-puck robot in the environment with obstacles. The experiment results show that the proposed approach decreases the learning time and the number of failures; also it improves the quality of the robot's motion in the testing environments.     

Indirect robot model learning for tracking control

Learning task-space tracking control on redundant robot manipulators is an important but difficult problem. A main difficulty is the non-uniqueness of the solution: a task-space trajectory has multiple joint-space trajectories associated, therefore averaging over non-convex solution space needs to be done if treated as a regression problem. A second class of difficulties arise for those robots when the physical model is either too complex or even not available. In this situation machine learning methods may be a suitable alternative to classical approaches. We propose a learning framework for tracking control that is applicable for underactuated or non-rigid robots where an analytical physical model of the robot is unavailable. The proposed framework builds on the insight that tracking problems are well defined in the joint task- and joint-space coordinates and consequently predictions can be obtained via local optimization. Physical experiments show that state-of-the art accuracy can be achieved in both online and offline tracking control learning. Furthermore, we show that the presented method is capable of controlling underactuated robot architectures as well.     

Fuzzy learning control for a flexible-link robot

There are two main drawbacks in fuzzy control: 1) the design of fuzzy controllers is usually performed in an ad hoc manner where it is often difficult to choose some of the controller parameters; and 2) the fuzzy controller constructed for the nominal plant may later perform inadequately if significant and unpredictable plant parameter variations occur. In this paper we illustrate these two problems on a two-link flexible robot testbed by: 1) developing, implementing, and evaluating a fuzzy controller for the robotic mechanism, and 2) illustrating that payload variations can have negative effects on the performance of a well designed fuzzy control system. Next, we show how to develop and implement a fuzzy model reference learning controller for the flexible robot and illustrate that it can automatically synthesize a rule-base for a fuzzy controller that will achieve comparable performance to the case where it was manually constructed, and automatically tune the fuzzy controller so that it can adapt to variations in the payload