Robotic execution of everyday tasks by means of external vision/force control

In this article, we present an integrated manipulation framework for a service robot, that allows to interact with articulated objects at home environments through the coupling of vision and force modalities. We consider a robot which is observing simultaneously his hand and the object to manipulate, by using an external camera (i.e. robot head). Task-oriented grasping algorithms (Proc of IEEE Int Conf on robotics and automation, pp 1794–1799, 2007) are used in order to plan a suitable grasp on the object according to the task to perform. A new vision/force coupling approach (Int Conf on advanced robotics, 2007), based on external control, is used in order to, first, guide the robot hand towards the grasp position and, second, perform the task taking into account external forces. The coupling between these two complementary sensor modalities provides the robot with robustness against uncertainties in models and positioning. A position-based visual servoing control law has been designed in order to continuously align the robot hand with respect to the object that is being manipulated, independently of camera position. This allows to freely move the camera while the task is being executed and makes this approach amenable to be integrated in current humanoid robots without the need of hand-eye calibration. Experimental results on a real robot interacting with different kind of doors are presented.     

A NEW APPROACH TO FILTERING AND ADAPTIVE CONTROL: OPTIMALITY RESULTS

Abstract

In a previous paper (Tesfatsion, in press) a new method was proposed for adaptive control. The key distinguishing feature is the direct consistent estimation and updating of the criterion (expected returns) function without recourse to prior state space specification, updated state probabilities, and Bayes's rule. The stability of a simple linear criterion function filtering scheme designed for control-dependent states was investigated in detail. In particular, it was shown that control variable sequences selected in accordance with the directly updated criterion function estimates converge under plausible restrictions to a local maximum of the true criterion function.

The principal purpose of the present paper is to establish sufficient conditions for control variables selected in accordance with the linear scheme to converge to a global maximum of the true criterion function. As will be clarified, when states are nontrivially dependent on control variable selection, the decision maker determines the trade-off between rate of convergence and asymptotic global optimality of control variable selections by his choice of greatest lower bound for the prior (initial period) criterion function. For trivial state-control dependence the asymptotic global optimality of control variable selections holds under weak restrictions.     

Vision-Based Task-Level Control of a Flexible-Link Manipulator

This paper discusses a vision-based approach to implement task-level control in flexible-link manipulators. The proposed approach emphasizes the advantage of using vision in the control of flexible manipulators. It is pointed out that taking advantage of the inherent robustness, implementation of an image-based visual servo can be regarded as a synthetic solution to precise task-level control of flexible manipulators. This approach is implemented in a three-dimensional flexible-link manipulator. The implementation makes good use of filters in decoupling task-level control and vibration suppression control. Moreover, we point out that although the robustness of the approach can help to overcome the difficulty in control resulting from the complex measurement of the link's elastic deformation, it lacks in capability of tip trajectory specification. This problem is analyzed in this research and it leads to the proposal for the integration of the image interpolation technique. This technique makes the proposed approach adequate for tasks involved with complex tip trajectories. For flexible-link manipulators, the proposed approach with the remedy is the first vision-based synthetic solution that attempts to make a flexible manipulator usable for a practical task.     

Continuous visual servoing despite the changes of visibility in image features

In the recent past, the visibility problem in vision-based control has been widely investigated. The proposed solutions generally have a common goal: to always keep the object in the camera's field of view during the visual servoing. Contrary to this solution, we propose a new approach based on the concept of allowing the changes of visibility in image features during the control task. To this aim, the camera invariant visual-servoing approach has been redefined in order to take into account the changes of visibility in image features. A new smooth task function using weighted features is presented, and a continuous control law is obtained starting from it by imposing its exponential decrease to zero. Furthermore, the local stability analysis of the invariant visual-servoing approach with weighted features is presented. Finally, this promising way of dealing with the visibility issue has been successfully tested with an eye-in-hand robotic system.     

Recognizing hidden emotions from difference image using mean local mapped pattern

Recent progress in computer vision has pushed the limit of facial recognition from human identification to micro-expressions (MEs). However, the visual analysis of MEs is still a very challenging task because of the short occurrence and insignificant intensity of the underlying signals. To date, the accuracy of recognizing hidden emotions from frames using conventional methods is still far from reaching saturation. To address this, we have proposed a new ME recognition approach based on Mean Local Mapped Pattern (M-LMP) as a texture feature, which outperforms other state-of-the art features in terms of accuracy due to its capability of capturing small pixel transitions. Inspired by previous work, we applied M-LMP to the difference image computed from an onset frame and an apex frame, where the former represents the frame with neutral emotion and the latter consists of the frame with the largest ME intensity. The extracted local features were classified using support vector machine (SVM) and K nearest neighbourhood (KNN) classifiers. The validation of the proposed approach was performed on the CASME II and CAS(ME)² datasets, and the results were compared with other similar state-of-the-art approaches. Comprehensive experiments were conducted using various parameters to show the robustness of our approach in the imbalanced and small dataset.

     

Exploiting label semantic relatedness for unsupervised image annotation with large free vocabularies

Automatic Image Annotation (AIA) is the task of assigning keywords to images, with the aim to describe their visual content. Recently, an unsupervised approach has been used to tackle this task. Unsupervised AIA (UAIA) methods use reference collections that consist of the textual documents containing images. The aim of the UAIA methods is to extract words from the reference collection to be assigned to images. In this regard, by using an unsupervised approach it is possible to include large vocabularies because any word could be extracted from the reference collection. However, having a greater diversity of words for labeling entails to deal with a larger number of wrong annotations, due to the increasing difficulty for assigning a correct relevance to the labels. With this problem in mind, this paper presents a general strategy for UAIA methods that reranks assigned labels. The proposed method exploits the semantic-relatedness information among labels in order to assign them an appropriate relevance for describing images. Experimental results in different benchmark datasets show the flexibility of our method to deal with assignments from free-vocabularies, and its effectiveness to improve the initial annotation performance for different UAIA methods. Moreover, we found that (1) when considering the semantic-relatedness information among the assigned labels, the initial ranking provided by a UAIA method is improved in most of the cases; and (2) the robustness of the proposed method to be applied on different UAIA methods, will allow extending capabilities of state-of-the-art UAIA methods.

     

Learning robustly stable open-loop motions for robotic manipulation

Robotic arms have been shown to be able to perform cyclic tasks with an open-loop stable controller. However, model errors make it hard to predict in simulation what cycle the real arm will perform. This makes it difficult to accurately perform pick and place tasks using an open-loop stable controller. This paper presents an approach to make open-loop controllers follow the desired cycles more accurately. First, we check if the desired cycle is robustly open-loop stable, meaning that it is stable even when the model is not accurate. A novel robustness test using linear matrix inequalities is introduced for this purpose. Second, using repetitive control we learn the open loop controller that tracks the desired cycle. Hardware experiments show that using this method, the accuracy of the task execution is improved to a precision of 2.5 cm, which suffices for many pick and place tasks.     

Simultaneous learning of perception and action in mobile robots

One of the main problems of robots is the lack of adaptability and the need for adjustment every time the robot changes its working place. To solve this, we propose a learning approach for mobile robots using a reinforcement-based strategy and a dynamic sensor-state mapping. This strategy, practically parameterless, minimises the adjustments needed when the robot operates in a different environment or performs a different task.Our system will simultaneously learn the state space and the action to execute on each state. The learning algorithm will attempt to maximise the time before a robot failure in order to obtain a control policy suited to the desired behaviour, thus providing a more interpretable learning process. The state representation will be created dynamically, starting with an empty state space and adding new states as the robot finds new situations that has not seen before. A dynamic creation of the state representation will avoid the classic, error-prone and cyclic process of designing and testing an ad hoc representation. We performed an exhaustive study of our approach, comparing it with other classic strategies. Unexpectedly, learning both perception and action does not increase the learning time.     

People counting via multiple views using a fast information fusion approach

Real-time estimates of a crowd size is a central task in civilian surveillance. In this paper we present a novel system counting people in a crowd scene with overlapping cameras. This system fuses all single view foreground information to localize each person present on the scene. The purpose of our fusion strategy is to use the foreground pixels of each single views to improve real-time objects association between each camera of the network. The foreground pixels are obtained by using an algorithm based on codebook. In this work, we aggregate the resulting silhouettes over cameras network, and compute a planar homography projection of each camera’s visual hull into ground plane. The visual hull is obtained by finding the convex hull of the foreground pixels. After the projection into the ground plane, we fuse the obtained polygons by using the geometric properties of the scene and on the quality of each camera detection. We also suggest a region-based approach tracking strategy which keeps track of people movements and of their identities along time, also enabling tolerance to occasional misdetections. This tracking strategy is implemented on the result of the views fusion and allows to estimate the crowd size dependently on each frame. Assessment of experiments using public datasets proposed for the evaluation of counting people system demonstrates the performance of our fusion approach. These results prove that the fusion strategy can run in real-time and is efficient for making data association. We also prove that the combination of our fusion approach and the proposed tracking improve the people counting.

     

Audio watermarking technique in frequency domain: comparative study MDCT Vs DCT

Digital watermarking is the art of hiding information in a digital document in order to protect it. The inserted mark and the marked document can be an image, an audio or a video. In this article, we will present a comparative study between two variants of a digital audio watermarking technique operating in the frequency domain. In the first variant, the time-frequency mapping is performed by Modified Discrete Cosine Transform (MDCT). For the second variant, the time-frequency mapping is performed by the Discrete Cosine Transform (DCT). We will study the contribution of each transformation, point of view robustness against different types of attacks delivered by Stirmark audio, imperceptibility by using a statistical approach by calculating the SNR and an objective approach by calculating the ODG notes given by PEAQ and capacity of insertion. Finally, to highlight our results, we will compare the two variants of the proposed technique with some other existing techniques.

     

Clustering by scale-space filtering

In pattern recognition and image processing, the major application areas of cluster analysis, human eyes seem to possess a singular aptitude to group objects and find important structures in an efficient and effective way. Thus, a clustering algorithm simulating a visual system may solve some basic problems in these areas of research. From this point of view, we propose a new approach to data clustering by modeling the blurring effect of lateral retinal interconnections based on scale space theory. In this approach, a data set is considered as an image with each light point located at a datum position. As we blur this image, smaller light blobs merge into larger ones until the whole image becomes one light blob at a low enough level of resolution. By identifying each blob with a cluster, the blurring process generates a family of clustering along the hierarchy. The advantages of the proposed approach are: 1) The derived algorithms are computationally stable and insensitive to initialization and they are totally free from solving difficult global optimization problems. 2) It facilitates the construction of new checks on cluster validity and provides the final clustering a significant degree of robustness to noise in data and change in scale. 3) It is more robust in cases where hyperellipsoidal partitions may not be assumed. 4) it is suitable for the task of preserving the structure and integrity of the outliers in the clustering process. 5) The clustering is highly consistent with that perceived by human eyes. 6) The new approach provides a unified framework for scale-related clustering algorithms derived from many different fields such as estimation theory, recurrent signal processing on self-organization feature maps, information theory and statistical mechanics, and radial basis function neural networks     

A new adaptive backstepping Coulomb friction compensator for servo control systems

A new Coulomb friction compensator is proposed for servo control systems in this paper. The novelty of the new approach lies in its capability of assigning the eigenvalues of the resulting closed loop system while attacking the problem. First, based on the standard backstepping methodology, an implicit Lyapunov function, with part of the components being only symbolically constructed at the very beginning, is utilized. To increase the robustness of the system against disturbance and model inaccuracy, an integral term is employed in the design. Using part of the variable gradient method, we are able to turn the implicit Lyapunov function into an explicit one, which is positive definite, and whose time‐derivative is negative definite. Second, it will be shown that the resulting closed loop error system is a switched linear system with two possible active modes that share the same set of eigenvalues, which is at our disposal. Unlike the common adaptive control design methods, such as the Control Lyapunov Function approach, in which the gains are typically positive but otherwise arbitrary, and are hence difficult to choose and have a lack of connection with the system's performance, our new scheme imposes two further constraints on the gains. It turns out that we can then match these gains with the coefficients of the desired characteristic equation of the closed loop system. In this respect, the gains are linked to the system's overall performance, which is a new and very appealing feature for such a scheme. Finally, a procedure of constructing a common Lyapunov function is provided to prove exponential stability of the aforementioned switched linear system. In addition, using the invariance principle, we will show the convergence of the estimated Coulomb friction coefficient to its real value. Numerical simulations are given to validate the effectiveness of the design and its robustness against friction time‐variations. Compared to existing results, the proposed scheme is much simpler, hence, much more advantageous computationally. Copyright © 2009 John Wiley and Sons Asia Pte Ltd and Chinese Automatic Control Society     

Effects of task characteristics on unimanual and bimanual movement times

Fitts' law cannot be used to predict movement times (MTs) of bimanual tasks since no empirical relationships associating task difficulty and bimanual MT have been demonstrated yet. Development of a ‘bimanual task difficulty index’ has been challenged by the complex patterns of coordination involved in simultaneously performing two tasks, one with each hand, under a control system with limited visual and attentional resources. To address this fundamental issue in human motor performance, bimanual object transfers with the left and right hands to targets of various precision requirements and separated by different distances were studied in six healthy subjects. Visual resource allocation during task performance was used to identify ‘primary’ and ‘secondary’ hand movements in bimanual tasks. While the primary movement was similar to a unimanual movement, the secondary MT varied with its own, as well as the contralateral hand's task constraints. These results, which were stable and consistent across six subjects, provide preliminary evidence that visual behaviour, indicating closed-loop control, can be used to systematically derive bimanual MTs.

Practitioner summary: A simple extension of Fitts' law cannot be used to predict movement times (MTs) of bimanual tasks since there is no consensus on the definition of a ‘bimanual task difficulty index’ in the literature. In this paper, we have approached this problem by using visual resource allocation patterns to systematically derive bimanual MTs.     

Novel hands-free interaction techniques based on the software switch approach for computer access with head movements

Head-operated computer accessibility tools (CATs) are useful solutions for the ones with complete head control; but when it comes to people with only reduced head control, computer access becomes a very challenging task since the users depend on a single head-gesture like a head nod or a head tilt to interact with a computer. It is obvious that any new interaction technique based on a single head-gesture will play an important role to develop better CATs to enhance the users’ self-sufficiency and the quality of life. Therefore, we proposed two novel interaction techniques namely HeadCam and HeadGyro within this study. In a nutshell, both interaction techniques are based on our software switch approach and can serve like traditional switches by recognizing head movements via a standard camera or a gyroscope sensor of a smartphone to translate them into virtual switch presses. A usability study with 36 participants (18 motor-impaired, 18 able-bodied) was also conducted to collect both objective and subjective evaluation data in this study. While HeadGyro software switch exhibited slightly higher performance than HeadCam for each objective evaluation metrics, HeadCam was rated better in subjective evaluation. All participants agreed that the proposed interaction techniques are promising solutions for computer access task.

     

考虑重力影响的空间机械臂轨迹跟踪滑模控制

针对空间机械臂由地面装调到空间应用过程中重力环境发生变化的问题,使用滑模控制器对空间机械臂进行控制,通过将配置特殊的非线性结构-fal函数引入趋近律的设计中,提出一种新的基于趋近律的滑模控制方法,并基于李亚普诺夫理论证明了闭环系统的渐近稳定性。仿真结果表明该方法能够很好地完成不同重力环境下机械臂的轨迹跟踪控制任务,并具有较强的鲁棒性。     

TWO SOLUTION TECHNIQUES FOR ADAPTIVE REINVESTMENT: A SMALL SAMPLE COMPARISON

Abstract

Two adaptive control techniques are compared in the context of an adaptive reinvestment two-armed bandit problem. The first solution technique is the Bayesian-dynamic programming approach. The distinctive feature of the recently proposed second solution technique is the direct estimation and updating of the criterion function without recourse to explicit probability distribution estimation via Bayes's rule. The control selections generated by the two solution techniques are shown to closely approximate each other. An explanation for this close correspondence is provided by means of an equivalence theorem for control objective specification.     

A fuzzy based ROI selection for encryption and watermarking in medical image using DWT and SVD

Nowadays secure medical image watermarking had become a stringent task in telemedicine. This paper presents a novel medical image watermarking method by fuzzy based Region of Interest (ROI) selection and wavelet transformation approach to embed encrypted watermark. First, the source image will undergo fuzzification to determine the critical points through central and final intensity along the radial line for selecting region of interest (ROI). Second, watermark image is altered to time-frequency domain through wavelet decomposition where the sub-bands are swapped based on the magnitude value obtained through logistic mapping. In the each sub-band all the pixels get swapped, results in fully encrypted image which guarantees the watermark to a secure, reliable and an unbreakable form. In order to provide more robustness to watermark image, singular values are obtained for encrypted watermark image and key component is calculated for avoiding false positive error. Singular values of the source and watermark image are modified through key component. Experimental results reveal that the proposed algorithm attains high robustness and improved security to the watermarked image against various kinds of attacks.

     

Group online adaptive learning

Sharing information among multiple learning agents can accelerate learning. It could be particularly useful if learners operate in continuously changing environments, because a learner could benefit from previous experience of another learner to adapt to their new environment. Such group-adaptive learning has numerous applications, from predicting financial time-series, through content recommendation systems, to visual understanding for adaptive autonomous agents. Here we address the problem in the context of online adaptive learning. We formally define the learning settings of Group Online Adaptive Learning and derive an algorithm named Shared Online Adaptive Learning (SOAL) to address it. SOAL avoids explicitly modeling changes or their dynamics, and instead shares information continuously. The key idea is that learners share a common small pool of experts, which they can use in a weighted adaptive way. We define group adaptive regret and prove that SOAL maintains known bounds on the adaptive regret obtained for single adaptive learners. Furthermore, it quickly adapts when learning tasks are related to each other. We demonstrate the benefits of the approach for two domains: vision and text. First, in the visual domain, we study a visual navigation task where a robot learns to navigate based on outdoor video scenes. We show how navigation can improve when knowledge from other robots in related scenes is available. Second, in the text domain, we create a new dataset for the task of assigning submitted papers to relevant editors. This is, inherently, an adaptive learning task due to the dynamic nature of research fields evolving in time. We show how learning to assign editors improves when knowledge from other editors is available. Together, these results demonstrate the benefits for sharing information across learners in concurrently changing environments.