Target Reaching by Using Visual Information and Q-learning Controllers

This paper presents a solution to the problem of manipulation control: target identification and grasping. The proposed controller is designed for a real platform in combination with a monocular vision system. The objective of the controller is to learn an optimal policy to reach and to grasp a spherical object of known size, randomly placed in the environment. In order to accomplish this, the task has been treated as a reinforcement problem, in which the controller learns by a trial and error approach the situation-action mapping. The optimal policy is found by using the Q-Learning algorithm, a model free reinforcement learning technique, that rewards actions that move the arm closer to the target.The vision system uses geometrical computation to simplify the segmentation of the moving target (a spherical object) and determines an estimate of the target parameters. To speed-up the learning time, the simulated knowledge has been ported on the real platform, an industrial robot manipulator PUMA 560. Experimental results demonstrate the effectiveness of the adaptive controller that does not require an explicit global target position using direct perception of the environment.     

Mobile PointMe-based spatial haptic interaction with annotated media for learning purposes

In this paper, we propose a framework to facilitate handheld device-based PointMe interaction with annotated media content, where the user points his/her handheld device to the media presentation screen in order to access services/information from the annotated media. Each annotation is encoded into customized learning object metadata (LOM) format that provides access point for available information about a specific media on the screen. By incorporating a fusion of infra-red (IR) motion points and accelerometer data of the handheld device, the system determines the learner’s PointMe location on the media screen. By using the pointed location spatial geometrical query is performed over the displayed annotated media in order to deliver learner’s level-specific interactive learning content to the user’s handheld device. The level customization is achieved by incorporating learner’s age, progress, etc., parameters in the media content delivery process. We design intuitive learning scenarios by leveraging the PointMe interaction scheme, where learners are challenged to answer quiz questions in learning-based games. This real-world interaction technique with the annotated media and seamless virtual learning information acquisition process makes the system transparent to young learners and encourage them to engage and concentrate on their learning activities. We develop a prototype and perform experiments in a technology-augmented learning space to show the suitability of the proposed approach.     

A practical use of learning system using user preference in ubiquitous computing environment

U-learning, or ubiquitous learning, has become the state-of-the-art educational trend in information technology. In ubiquitous learning environment, students now have the ability to learn anytime and anywhere. In many computer-assisted learning systems (such as e-learning, m-learning and u-learning), students must follow the lesson plans and learning environments established by the teacher. To overcome this limitation and increase effective learning, new techniques that reflect alternative learning styles, such as self-directed learning, adaptive learning and personalized learning, have been developed. These techniques have been researched extensively, but they still do not consistently reflect the specific needs of all students. In this article, a u-learning system is proposed that considers the learning level and preferences of each learner. The system provides information about the student’s preferred learning section and difficulty level of learning contents and indicates the areas that may require additional study (based on the educational history of the student), thus allowing students to set up an optimized learning environment. A topic preference vector was applied to calculate the student’s preferences, and the learning section and the difficulty level were used as each vector value in the system. To verify the effectiveness of the proposed system, an English-learning system was implemented using content from the reading comprehension section of the Test of English for International Communication (TOEIC). An experiment was conducted using a control group and a test group. The results demonstrated that the system proposed in this paper is useful for improving learning efficiency.     

Robotic grasping and manipulation through human visuomotor learning

A major goal of robotics research is to develop techniques that allow non-experts to teach robots dexterous skills. In this paper, we report our progress on the development of a framework which exploits human sensorimotor learning capability to address this aim. The idea is to place the human operator in the robot control loop where he/she can intuitively control the robot, and by practice, learn to perform the target task with the robot. Subsequently, by analyzing the robot control obtained by the human, it is possible to design a controller that allows the robot to autonomously perform the task. First, we introduce this framework with the ball-swapping task where a robot hand has to swap the position of the balls without dropping them, and present new analyses investigating the intrinsic dimension of the ball-swapping skill obtained through this framework. Then, we present new experiments toward obtaining an autonomous grasp controller on an anthropomorphic robot. In the experiments, the operator directly controls the (simulated) robot using visual feedback to achieve robust grasping with the robot. The data collected is then analyzed for inferring the grasping strategy discovered by the human operator. Finally, a method to generalize grasping actions using the collected data is presented, which allows the robot to autonomously generate grasping actions for different orientations of the target object.     

Bringing chatbots into education: Towards natural language negotiation of open learner models

There is an extensive body of work on Intelligent Tutoring Systems: computer environments for education, teaching and training that adapt to the needs of the individual learner. Work on personalisation and adaptivity has included research into allowing the student user to enhance the system’s adaptivity by improving the accuracy of the underlying learner model. Open Learner Modelling, where the system’s model of the user’s knowledge is revealed to the user, has been proposed to support student reflection on their learning. Increased accuracy of the learner model can be obtained by the student and system jointly negotiating the learner model. We present the initial investigations into a system to allow people to negotiate the model of their understanding of a topic in natural language. This paper discusses the development and capabilities of both conversational agents (or chatbots) and Intelligent Tutoring Systems, in particular Open Learner Modelling. We describe a Wizard-of-Oz experiment to investigate the feasibility of using a chatbot to support negotiation, and conclude that a fusion of the two fields can lead to developing negotiation techniques for chatbots and the enhancement of the Open Learner Model. This technology, if successful, could have widespread application in schools, universities and other training scenarios.     

Learning cooperative grasping with the graph representation of a state-action space

In this paper we present a method for two robot manipulators to learn cooperative tasks. If a single robot is unable to grasp an object in a certain orientation, it can only continue with the help of other robots. The grasping can be realized by a sequence of cooperative operations that re-orient the object. Several sequences are needed to handle the different situations in which an object is not graspable for the robot. It is shown that a distributed learning method based on a Markov decision process is able to learn the sequences for the involved robots, a master robot that needs to grasp and a helping robot that supports him with the re-orientation. A novel state-action graph is used to store the reinforcement values of the learning process. Further an example of aggregate assembly shows the generality of this approach.     

机器人多指操作的递阶控制1)

This paper presents a hierarchical control system for robot multifingered coordinate manipulation. Given a manipulation,the task planner generates a sequence of object's motion velocities at first,and then generates for coordinate motion the desired velocities of finger's motion and desired orientation change of the grasped object according to the desired velocities of object's motion.At the same time,the force planner generates the grasp forces on the fingers in order to resist the external forces on the object,according to the grasp posture.Finally,the system generates a result compliance velocity from both the desired finger's velocities and desired grasp forces,and transfers it into joint velocites through the finger's inverse Jacobian.Then the controller of joint motion implements the control of both forces and velocities for the fingers.The approach has been applied to the development of control system HKUST dexterous hand successfully.Experiment results show that it is not only possible to trail and control the object's track,but also possible to realize force control and the hybrid control of both forces and velocities through this method.     

目标不同视角下观察信息的迁移和再利用

基于视觉信息的目标检测和识别模型在训练时往往依赖于来自于训练样本的视角信息,然而附带了视角信息的训练样本通常只有很少的数据库可以提供。当此类信息缺失时,传统的通用目标检测系统通常通过一些非监督学习方法来对样本的视角信息进行粗略估计。本文改进并引入了一种选择性迁移学习方法即TransferBoost方法来解决目标视角信息缺失的问题。本文TransferBoost方法基于GentleBoost框架实现,该方法通过重新利用其它类别样本中的先验信息来提升当前类别样本的学习质量。当给定一个标定完善的样本集作为源数据库时,TransferBoost通过同时调整每个样本的权值和每个源任务的权值实现样本级和任务级的两级知识迁移。这种双层迁移学习更有效地从混合了相关源数据和不相关源数据的数据集中提取了有用的信息。实验结果表明,和直接使用传统的机器学习方法相比较,迁移学习方法所需要的训练样本数大大减少,从而降低了目标检测与识别系统的训练代价,扩展了现有系统的应用范围。     

Personalized curriculum sequencing utilizing modified item response theory for web-based instruction

Curriculum sequencing is an important research issue for Web-based instruction systems because no fixed learning pathway will be appropriate for all learners. Therefore, many researchers focused on developing e-learning systems with personalized learning mechanism to assist on-line Web-based learning and adaptively provide learning pathways. However, although most personalized systems consider learner preferences, interests and browsing behavior in providing personalized curriculum sequencing services, these systems usually neglect to consider whether learner ability and the difficulty level of the recommended courseware are matched to each other or not. Generally, inappropriate courseware leads to learner cognitive overload or disorientation during learning, thus reducing learning effect. Besides, the problem of concept continuity of learning pathways also needs to be considered while implementing personalized curriculum sequencing. Smoother learning pathways increase learning effect, avoiding unnecessarily difficult concepts. This paper presents a prototype of personalized Web-based instruction system (PWIS) based on the proposed modified Item Response Theory (IRT) to perform personalized curriculum sequencing through simultaneously considering courseware difficulty level, learner's ability and the concept continuity of learning pathways during learning. In the proposed modified IRT, the information function is revised to consider the concept continuity of learning pathway as well as considering the difficulty level of courseware and individual learner ability. Experiment results indicate that applying the proposed modified IRT for Web-based learning can construct suitable learning pathway to learners for personalized learning, and help them to learn more effectively.     

机器人多指操作的递阶控制

为机器人多指协调操作建立一递阶控制系统.给定一操作任务,任务规划器首先生 成一系列物体的运动速度;然后,协调运动规划器根据期望的物体运动速度生成期望的手指 运动速度和期望的抓取姿态变化;同时,抓取力规划器为平衡作用在物体上的外力,根据当前 的抓取姿态,生成各手指所需的抓取力;最后,系统将手指的期望运动速度与为实现期望抓取 力而生成的顺应速度合并,并通过手指的逆雅可比转化为手指关节运动速度后,由手指的关 节级运动控制器实现手指的运动和抓取力的控制.该控制方法已成功应用于香港科技大学 (HKUST)灵巧手控制系统的开发.实验证明该方法不仅能完成物体轨迹的跟踪控制任务, 而且能完成物体对环境的力控制和力与速度的混合控制.     

Adaptive neural controller for cooperative multiple robot manipulator system manipulating a single rigid object

In this article, an adaptive neural controller is developed for cooperative multiple robot manipulator system carrying and manipulating a common rigid object. In coordinated manipulation of a single object using multiple robot manipulators simultaneous control of the object motion and the internal force exerted by manipulators on the object is required. Firstly, an integrated dynamic model of the manipulators and the object is derived in terms of object position and orientation as the states of the derived model. Based on this model, a controller is proposed that achieves required trajectory tracking of the object as well as tracking of the desired internal forces arising in the system. A feedforward neural network is employed to learn the unknown dynamics of robot manipulators and the object. It is shown that the neural network can cope with the unknown nonlinearities through the adaptive learning process and requires no preliminary offline learning. The adaptive learning algorithm is derived from Lyapunov stability analysis so that both error convergence and tracking stability are guaranteed in the closed loop system. Finally, simulation studies and analysis are carried out for two three-link planar manipulators moving a circular disc on specified trajectory.     

Static grip selection for robot-based automated assembly systems

In this article we present an algorithmic approach to determine the suitable grasp of an object in an automated assembly environment. The algorithm is based on the available object surfaces and the initial and final task constraints and gripper characteristics. If the imposed task and gripper constraints do not allow a possible grasp, intermediate motions may need to be made to reorient the part. Once a set of possible grasps which statisfy task and gripper constraints are found, the stability of each grasp is analyzed using screw theory. An optimal grasp is one which minimizes the grasping forces over the possible set of grasps. Results utilizing our methodology are presented. Our method can be interfaced with CAD database such as a solid modelling system based on boundary representation for automatic selection of grasping configurations.     

Coupled dynamical system based arm–hand grasping model for learning fast adaptation strategies

Performing manipulation tasks interactively in real environments requires a high degree of accuracy and stability. At the same time, when one cannot assume a fully deterministic and static environment, one must endow the robot with the ability to react rapidly to sudden changes in the environment. These considerations make the task of reach and grasp difficult to deal with. We follow a Programming by Demonstration (PbD) approach to the problem and take inspiration from the way humans adapt their reach and grasp motions when perturbed. This is in sharp contrast to previous work in PbD that uses unperturbed motions for training the system and then applies perturbation solely during the testing phase. In this work, we record the kinematics of arm and fingers of human subjects during unperturbed and perturbed reach and grasp motions. In the perturbed demonstrations, the target’s location is changed suddenly after the onset of the motion. Data show a strong coupling between the hand transport and finger motions. We hypothesize that this coupling enables the subject to seamlessly and rapidly adapt the finger motion in coordination with the hand posture. To endow our robot with this competence, we develop a coupled dynamical system based controller, whereby two dynamical systems driving the hand and finger motions are coupled. This offers a compact encoding for reach-to-grasp motions that ensures fast adaptation with zero latency for re-planning. We show in simulation and on the real iCub robot that this coupling ensures smooth and “human-like” motions. We demonstrate the performance of our model under spatial, temporal and grasp type perturbations which show that reaching the target with coordinated hand–arm motion is necessary for the success of the task.     

Personalized web-based tutoring system based on fuzzy item response theory

With the rapid growth of computer and Internet technologies, e-learning has become a major trend in the computer assisted teaching and learning field. Previously, many researchers put effort into e-learning systems with personalized learning mechanism to aid on-line learning. However, most systems focus on using learner’s behaviors, interests, and habits to provide personalized e-learning services. These systems commonly neglect to consider if learner ability and the difficulty level of the recommended courseware are matched to each other. Frequently, unsuitable courseware causes learner’s cognitive overload or disorientation during learning. To promote learning effectiveness, our previous study proposed a personalized e-learning system based on Item response theory (PEL-IRT), which can consider both course material difficulty and learner ability evaluated by learner’s crisp feedback responses (i.e. completely understanding or not understanding answer) to provide personalized learning paths for individual learners. The PEL-IRT cannot estimate learner ability for personalized learning services according to learner’s non-crisp responses (i.e. uncertain/fuzzy responses). The main problem is that learner’s response is not usually belonging to completely understanding or not understanding case for the content of learned courseware. Therefore, this study developed a personalized intelligent tutoring system based on the proposed fuzzy item response theory (FIRT), which could be capable of recommending courseware with suitable difficulty levels for learners according to learner’s uncertain/fuzzy feedback responses. The proposed FIRT can correctly estimate learner ability via the fuzzy inference mechanism and revise estimating function of learner ability while the learner responds to the difficulty level and comprehension percentage for the learned courseware. Moreover, a courseware modeling process developed in this study is based on a statistical technique to establish the difficulty parameters of courseware for the proposed personalized intelligent tutoring system. Experiment results indicate that applying the proposed FIRT to web-based learning can provide better learning services for individual learners than our previous study, thus helping learners to learn more effectively.     

Recurrent neural robot controllers: feedback mechanisms for identifying environmental motion dynamics

In this paper a series of recurrent controllers for mobile robots have been developed. The system combines the iterative learning capability of neural controllers and the optimisation ability of particle swarms. In particular, three controllers have been developed: an Exo-sensing, an Ego-sensing and a Composite controller which is the hybrid of the latter two. The task for each controller is to learn to follow a moving target and identify its trajectory using only local information. We show how the learned behaviours of each architecture rely on different sensory representations, although good results are obtained in all cases.     

A self-adjusting active compliance controller for multiple robots handling an object

This paper presents the concept and experimental validation of a self-adjusting active compliance controller for n robots handling its compliant behaviour concerning partly unknown flexible object. The control strategy is based on the decomposition of the 6n-dimensional position/force space and includes a feedforward and feedback level. The feedforward level contains motion coordination, force distribution of external forces, creation of internal forces, and an additional loop adding the elastic displacements due to the applied forces to the planned robot positions. The feedback level is organized in the form of an active compliance control law. For adjusting the controller to the, in general, unknown flexible behaviour, which in practice is the main problem of the controller design, a quasi-static model of the system is derived for different contact cases of the object and a procedure is presented, which by use of this model is capable of determining the compliance of the considered system and therefore of adjusting the controller. Experiments with two puma-type robots have been conducted to show the applicability of the self-adjusting control strategy. The task has been to grasp and move an unconstrained object. It is shown, that the system can adjust the control parameters to the unknown system compliance and that the control performance is improved considerably.     

Learning how to grasp based on neural network retraining

Humans have an incredible capacity to manipulate objects using dextrous hands. A large number of studies indicate that robot learning by demonstration is a promising strategy to improve robotic manipulation and grasping performance. Concerning this subject we can ask: How does a robot learn how to grasp? This work presents a method that allows a robot to learn new grasps. The method is based on neural network retraining. With this approach we aim to enable a robot to learn new grasps through a supervisor. The proposed method can be applied for 2D and 3D cases. Extensive object databases were generated to evaluate the method performance in both 2D and 3D cases. A total of 8100 abstract shapes were generated for 2D cases and 11700 abstract shapes for 3D cases. Simulation results with a computational supervisor show that a robotic system can learn new grasps and improve its performance through the proposed HRH (Hopfield-RBF-Hopfield) grasp learning approach.     

Deep learning-based method for vision-guided robotic grasping of unknown objects

Nowadays, robots are heavily used in factories for different tasks, most of them including grasping and manipulation of generic objects in unstructured scenarios. In order to better mimic a human operator involved in a grasping action, where he/she needs to identify the object and detect an optimal grasp by means of visual information, a widely adopted sensing solution is Artificial Vision. Nonetheless, state-of-art applications need long training and fine-tuning for manually build the object’s model that is used at run-time during the normal operations, which reduce the overall operational throughput of the robotic system. To overcome such limits, the paper presents a framework based on Deep Convolutional Neural Networks (DCNN) to predict both single and multiple grasp poses for multiple objects all at once, using a single RGB image as input. Thanks to a novel loss function, our framework is trained in an end-to-end fashion and matches state-of-art accuracy with a substantially smaller architecture, which gives unprecedented real-time performances during experimental tests, and makes the application reliable for working on real robots. The system has been implemented using the ROS framework and tested on a Baxter collaborative robot.