The role of memory and concepts in learning

The extent to which concepts, memory, and planning are necessary to the simulation of intelligent behavior is a fundamental philosophical issue in Artificial Intelligence. An active and productive segement of the AI community has taken the position that multiple low-level agents, properly organized, can account for high-level behavior. Empirical research on these questions with fully operational systems has been restricted to mobile robots that do simple tasks. This paper recounts experiments with Hoyle, a system in a cerebral, rather than a physical, domain. The program learns to perform well and quickly, often outpacing its human creators at two-person, perfect information board games. Hoyle demonstrates that a surprising amount of intelligent behavior can be treated as if it were situation-determined, that often planning is unnecessary, and that the memory required to support this learning is minimal. Concepts, however, are crucial to this reactive program's ability to learn and perform.     

A Study on Acquiring Underlying Behavioral Criteria for Manipulator Motion by Focusing on Learning Efficiency

Conventional humanoid robotic behaviors are directly programmed depending on the programmer's personal experience. With this method, the behaviors usually appear unnatural. It is believed that a humanoid robot can acquire new adaptive behaviors from a human, if the robot has the criteria underlying such behaviors. The aim of this paper is to establish a method of acquiring human behavioral criteria. The advantage of acquiring behavioral criteria is that the humanoid robots can then autonomously produce behaviors for similar tasks with the same behavioral criteria but without transforming data obtained from morphologically different humans every time for every task. In this paper, a manipulator robot learns a model behavior, and another robot is created to perform the model behavior instead of being performed by a person. The model robot is presented some behavioral criteria, but the learning manipulator robot does not know them and tries to infer them. In addition, because of the difference between human and robot bodies, the body sizes of the learning robot and the model robot are also made different. The method of obtaining behavioral criteria is realized by comparing the efficiencies with which the learning robot learns the model behaviors. Results from the simulation have demonstrated that the proposed method is effective for obtaining behavioral criteria. The proposed method, the details regarding the simulation, and the results are presented in this paper.     

A cognitive robot architecture based on tactile and visual information

In this paper, we propose an architecture for a cognitive robot based on tactile and visual information. Visual information contains various features such as location and area of various colored regions. Most of these features are irrelevant for object recognition to achieve the given task. In the architecture, tactile information plays a key role in the selection of visual features and discritization of selected features. In order to find appropriate visual features we use a correlation coefficient between the values of the features and action series. Then the ChiMerge algorithm is employed to discritize the value of the selected feature into a small number of intervals. Consequently, quantization of a state space to accomplish the given task is achieved. An appropriate behavior to the given task is acquired by using this state space with reinforcement learning algorithm. We give experimental results of computer simulation to show the validity of our method.     

Modeling human behavior in manual control Rendezvous and Docking task

Manual control rendezvous and docking (RVD) with human participation can be used when autonomous RVD is invalid under uncertain environment. Because of the particularity and complexity of the RVD task, it is necessary to understand human cognitive processes when evaluating human performance. A modeling approach, focusing on the information processing underlying the decisions process, is proposed to achieve real-time visualization of information processing and to generate human-like behavior of manual control RVD in this paper. It is implemented by combining the symbolic knowledge representations with queuing network mechanism. This computational model here can be used for describing and explaining how human cognition works. Furthermore, a quantitative validation of the model is conducted by comparing the performance results of the model with the results of people doing the same tasks, which reflects that this model can be applied as “replacements” for human participants to evaluate their cognition and performance in manual control RVD task.     

基于Java 3D的虚拟人仿真方法

提出一种将3DS MAX、MS3D与Java 3D编程技术相结合的虚拟人仿真方法,可使虚拟人达到相对逼真且交互性强的效果。该方法首先采用3DS MAX角色动画技术进行人体静态建模和动作建模;然后通过将底层基本动作片段转化为MS3D格式,供Java 3D的骨骼动画模型接口调用;最后利用Java 3D编程来控制虚拟人的高层行为活动。该方法有利于角色建模、运动仿真和行为控制等设计工作的分工协作,适合于网络环境下多角色、复杂动作的虚拟人仿真。     

An online pruning strategy for supervised ARTMAP-based neural networks

Identifying an appropriate architecture of an artificial neural network (ANN) for a given task is important because learning and generalisation of an ANN is affected by its structure. In this paper, an online pruning strategy is proposed to participate in the learning process of two constructive networks, i.e. fuzzy ARTMAP (FAM) and fuzzy ARTMAP with dynamic decay adjustment (FAMDDA), and the resulting hybrid networks are called FAM/FAMDDA with temporary nodes (i.e. FAM-T and FAMDDA-T, respectively). FAM-T and FAMDDA-T possess a capability of reducing the network complexity online by removing unrepresentative neurons. The performances of FAM-T and FAMDDA-T are evaluated and compared with those of FAM and FAMDDA using a total of 13 benchmark data sets. To demonstrate the applicability of FAM-T and FAMDDA-T, a real fault detection and diagnosis task in a power plant is tested. The results from both benchmark studies and real-world application show that FAMDDA-T and FAM-T are able to yield satisfactory classification performances, with the advantage of having parsimonious network structures.     

一种分层强化学习的知识推理方法

针对知识推理过程中,随着推理路径长度的增加,节点的动作空间急剧增长,使得推理难度不断提升的问题,提出一种分层强化学习的知识推理方法(knowledge reasoning method of hierarchical reinforcement learning, MutiAg-HRL),降低推理过程中的动作空间大小。MutiAg-HRL调用高级智能体对知识图谱中的关系进行粗略推理,通过计算下一步关系及给定查询关系之间的相似度,确定目标实体大致位置,依据高级智能体给出的关系,指导低级智能体进行细致推理,选择下一步动作;模型还构造交互奖励机制,对两个智能体的关系和动作选择及时给予奖励,防止模型出现奖励稀疏问题。为验证该方法的有效性,在FB15K-237和NELL-995数据集上进行实验,将实验结果与TransE、MINERVA、HRL等11种主流方法进行对比分析,MutiAg-HRL方法在链接预测任务上的hits@k平均提升了1.85%,MRR平均提升了2%。     

The SUN Attribute Database: Beyond Categories for Deeper Scene Understanding

In this paper we present the first large-scale scene attribute database. First, we perform crowdsourced human studies to find a taxonomy of 102 discriminative attributes. We discover attributes related to materials, surface properties, lighting, affordances, and spatial layout. Next, we build the “SUN attribute database” on top of the diverse SUN categorical database. We use crowdsourcing to annotate attributes for 14,340 images from 707 scene categories. We perform numerous experiments to study the interplay between scene attributes and scene categories. We train and evaluate attribute classifiers and then study the feasibility of attributes as an intermediate scene representation for scene classification, zero shot learning, automatic image captioning, semantic image search, and parsing natural images. We show that when used as features for these tasks, low dimensional scene attributes can compete with or improve on the state of the art performance. The experiments suggest that scene attributes are an effective low-dimensional feature for capturing high-level context and semantics in scenes.     

基于视觉词袋模型的人耳识别

人耳识别技术是生物特征识别和人工智能领域的一个重要分支.针对人耳图像特有的纹理特征,首先采用空间金字塔视觉词袋模型进行人耳特征提取,该模型将人耳图像中相对低级的局部描述子特征转化为具有高级语义含义的全局特征.最后采用支持向量机对样本向量进行训练与判别.实验表明,本文所采用的模型能取得较高的识别率,可作为人耳识别方法的一种扩展与探索.     

Efficient behavior learning in human–robot collaboration

We present a novel method for a robot to interactively learn, while executing, a joint human–robot task. We consider collaborative tasks realized by a team of a human operator and a robot helper that adapts to the human’s task execution preferences. Different human operators can have different abilities, experiences, and personal preferences so that a particular allocation of activities in the team is preferred over another. Our main goal is to have the robot learn the task and the preferences of the user to provide a more efficient and acceptable joint task execution. We cast concurrent multi-agent collaboration as a semi-Markov decision process and show how to model the team behavior and learn the expected robot behavior. We further propose an interactive learning framework and we evaluate it both in simulation and on a real robotic setup to show the system can effectively learn and adapt to human expectations.     

Learning Atomic Human Actions Using Variable-Length Markov Models

Visual analysis of human behavior has generated considerable interest in the field of computer vision because of its wide spectrum of potential applications. Human behavior can be segmented into atomic actions, each of which indicates a basic and complete movement. Learning and recognizing atomic human actions are essential to human behavior analysis. In this paper, we propose a framework for handling this task using variable-length Markov models (VLMMs). The framework is comprised of the following two modules: a posture labeling module and a VLMM atomic action learning and recognition module. First, a posture template selection algorithm, based on a modified shape context matching technique, is developed. The selected posture templates form a codebook that is used to convert input posture sequences into discrete symbol sequences for subsequent processing. Then, the VLMM technique is applied to learn the training symbol sequences of atomic actions. Finally, the constructed VLMMs are transformed into hidden Markov models (HMMs) for recognizing input atomic actions. This approach combines the advantages of the excellent learning function of a VLMM and the fault-tolerant recognition ability of an HMM. Experiments on realistic data demonstrate the efficacy of the proposed system.     

采用自然语言的移动机器人任务编程

本文在机器人具备基本运动技能的基础上［1］，采用基于指令教导的学习方法．通 过自然语言教会机器人完成抽象化任务，并以程序体方式保存所学知识，也即通过自然语言 对话自动生成程序流．通过让机器人完成导航等任务，验证所提自然语言编程方法的可行性 ．     

GA-based fuzzy reinforcement learning for control of a magneticbearing system

This paper proposes a TD (temporal difference) and GA (genetic algorithm)-based reinforcement (TDGAR) learning method and applies it to the control of a real magnetic bearing system. The TDGAR learning scheme is a new hybrid GA, which integrates the TD prediction method and the GA to perform the reinforcement learning task. The TDGAR learning system is composed of two integrated feedforward networks. One neural network acts as a critic network to guide the learning of the other network (the action network) which determines the outputs (actions) of the TDGAR learning system. The action network can be a normal neural network or a neural fuzzy network. Using the TD prediction method, the critic network can predict the external reinforcement signal and provide a more informative internal reinforcement signal to the action network. The action network uses the GA to adapt itself according to the internal reinforcement signal. The key concept of the TDGAR learning scheme is to formulate the internal reinforcement signal as the fitness function for the GA such that the GA can evaluate the candidate solutions (chromosomes) regularly, even during periods without external feedback from the environment. This enables the GA to proceed to new generations regularly without waiting for the arrival of the external reinforcement signal. This can usually accelerate the GA learning since a reinforcement signal may only be available at a time long after a sequence of actions has occurred in the reinforcement learning problem. The proposed TDGAR learning system has been used to control an active magnetic bearing (AMB) system in practice. A systematic design procedure is developed to achieve successful integration of all the subsystems including magnetic suspension, mechanical structure, and controller training. The results show that the TDGAR learning scheme can successfully find a neural controller or a neural fuzzy controller for a self-designed magnetic bearing system.     

Human action recognition using two-stream attention based LSTM networks

It is well known that different frames play different roles in feature learning in video based human action recognition task. However, most existing deep learning models put the same weights on different visual and temporal cues in the parameter training stage, which severely affects the feature distinction determination. To address this problem, this paper utilizes the visual attention mechanism and proposes an end-to-end two-stream attention based LSTM network. It can selectively focus on the effective features for the original input images and pay different levels of attentions to the outputs of each deep feature maps. Moreover, considering the correlation between two deep feature streams, a deep feature correlation layer is proposed to adjust the deep learning network parameter based on the correlation judgement. In the end, we evaluate our approach on three different datasets, and the experiments results show that our proposal can achieve the state-of-the-art performance in the common scenarios.     

EM-based policy hyper parameter exploration: application to standing and balancing of a two-wheeled smartphone robot

This paper proposes a novel policy search algorithm called EM-based Policy Hyper Parameter Exploration (EPHE) which integrates two reinforcement learning algorithms: Policy Gradient with Parameter Exploration (PGPE) and EM-based Reward-Weighted Regression. Like PGPE, EPHE evaluates a deterministic policy in each episode with the policy parameters sampled from a prior distribution given by the policy hyper parameters (mean and variance). Based on EM-based Reward-Weighted Regression, the policy hyper parameters are updated by reward-weighted averaging so that gradient calculation and tuning of the learning rate are not required. The proposed method is tested in the benchmarks of pendulum swing-up task, cart-pole balancing task and simulation of standing and balancing of a two-wheeled smartphone robot. Experimental results show that EPHE can achieve efficient learning without learning rate tuning even for a task with discontinuities.     

Reduction of state space in reinforcement learning by sensor selection

Much research has been conducted on the application of reinforcement learning to robots. Learning time is a matter of concern in reinforcement learning. In reinforcement learning, information from sensors is projected on to a state space. A robot learns the correspondence between each state and action in state space and determines the best correspondence. When the state space is expanded according to the number of sensors, the number of correspondences learnt by the robot is increased. Therefore, learning the best correspondence becomes time consuming. In this study, we focus on the importance of sensors for a robot to perform a particular task. The sensors that are applicable to a task differ for different tasks. A robot does not need to use all installed sensors to perform a task. The state space should consist of only those sensors that are essential to a task. Using such a state space consisting of only important sensors, a robot can learn correspondences faster than in the case of a state space consisting of all installed sensors. Therefore, in this paper, we propose a relatively fast learning system in which a robot can autonomously select those sensors that are essential to a task and a state space for only such important sensors is constructed. We define the measure of importance of a sensor for a task. The measure is the coefficient of correlation between the value of each sensor and reward in reinforcement learning. A robot determines the importance of sensors based on this correlation. Consequently, the state space is reduced based on the importance of sensors. Thus, the robot can efficiently learn correspondences owing to the reduced state space. We confirm the effectiveness of our proposed system through a simulation.     

Human action segmentation and classification based on the Isomap algorithm

Visual analysis of human behavior has attracted a great deal of attention in the field of computer vision because of the wide variety of potential applications. Human behavior can be segmented into atomic actions, each of which indicates a single, basic movement. To reduce human intervention in the analysis of human behavior, unsupervised learning may be more suitable than supervised learning. However, the complex nature of human behavior analysis makes unsupervised learning a challenging task. In this paper, we propose a framework for the unsupervised analysis of human behavior based on manifold learning. First, a pairwise human posture distance matrix is derived from a training action sequence. Then, the isometric feature mapping (Isomap) algorithm is applied to construct a low-dimensional structure from the distance matrix. Consequently, the training action sequence is mapped into a manifold trajectory in the Isomap space. To identify the break points between the trajectories of any two successive atomic actions, we represent the manifold trajectory in the Isomap space as a time series of low-dimensional points. A temporal segmentation technique is then applied to segment the time series into sub series, each of which corresponds to an atomic action. Next, the dynamic time warping (DTW) approach is used to cluster atomic action sequences. Finally, we use the clustering results to learn and classify atomic actions according to the nearest neighbor rule. If the distance between the input sequence and the nearest mean sequence is greater than a given threshold, it is regarded as an unknown atomic action. Experiments conducted on real data demonstrate the effectiveness of the proposed method.