Learning Motion of Dexterous Manipulation For A Pair of Multi‐DOF Fingers With Soft‐TIPS

This paper shows that a pair of dual multi‐DOF fingers with soft‐tips can learn iteratively a desired periodic motion of manipulation of an object if sensory feedback signals are designed adequately. It is shown that dynamics of the overall fingers and object system satisfy passivity but residual error dynamics for a given periodic posture of the object and a fixed value of contact force satisfy output‐dissipativity only in an approximate sense. Numerical simulation results are presented which show that the pair of fingers manipulating an object is capable of learning iteratively a variety of dexterous motions with a good performance.     

Augmenting Tactile 3D Data Navigation With Pressure Sensing

We present a pressure‐augmented tactile 3D data navigation technique, specifically designed for small devices, motivated by the need to support the interactive visualization beyond traditional workstations. While touch input has been studied extensively on large screens, current techniques do not scale to small and portable devices. We use phone‐based pressure sensing with a binary mapping to separate interaction degrees of freedom (DOF) and thus allow users to easily select different manipulation schemes (e. g., users first perform only rotation and then with a simple pressure input to switch to translation). We compare our technique to traditional 3D‐RST (rotation, scaling, translation) using a docking task in a controlled experiment. The results show that our technique increases the accuracy of interaction, with limited impact on speed. We discuss the implications for 3D interaction design and verify that our results extend to older devices with pseudo pressure and are valid in realistic phone usage scenarios.     

Dexterous manipulation of an object by means of multi‐DOF robotic fingers with soft tips

This article analyzes the dynamics of motion of various setups of two multiple degree‐of‐freedom (DOF) fingers that have soft tips, in fine manipulation of an object, and shows performances of their motions via computer simulation. A mathematical model of these dynamics is described as a system of nonlinear differential equations expressing motion of the overall fingers‐object system together with algebraic constraints due to tight area contacts between the finger‐tips and surfaces of the object. First, problems of (1) dynamic, stable grasping and (2) regulation of the object rotational angle by means of a setup of dual two‐DOF fingers, are treated. Second, the problem of regulating the position of the object mass center by means of a pair of two‐DOF and three‐DOF fingers is considered. Third, a set of dual three‐DOF fingers is treated, in order to let it perform a sophisticated task, which is specified by a periodic pattern of the object posture and a constant internal force. In any case, there exist sensory‐motor coordinations, which are described by analytic feedback connections from sensing to actions at finger joints. In the cases of setpoint control problems, convergences of motion to secure grasping together with the specified object rotational angle and/or the specified object mass center position, are proved theoretically. A constraint stabilization method (CSM) is used for solving numerically the differential algebraic equations to show performances of the proposed sensory‐feedback schemes. © 2002 Wiley Periodicals, Inc.     

Kinematics of grasping and manipulation of a B‐spline surface object by a multifingered robot hand

Kinematics of grasping and manipulation by a multifingered robotic hand where multifinger surfaces are in contact with an object is solved. The surface of the object was represented by B‐spline surfaces to model objects of various shapes. The fingers were modeled by cylindrical links and a half ellipsoid fingertip. Geometric contact equations have been solved for all possible contact combinations between the finger surface and the object. The simulation system calculated joint displacements and contact locations for a given trajectory of the object. Since there are no closed form solutions for contact or intersection between these surfaces, kinematics of grasping was solved by recursive numerical calculation. The initial estimate of the contact point was obtained by approximating the B‐spline surface by a polyhedron. As for the simulation of manipulation, exact contact locations were updated by solving the contact equations according to the given contact conditions such as pure rolling, twist‐rolling, or slide‐twist‐rolling. Several examples of simulation of grasping and manipulation are presented. ©1999 John Wiley & Sons, Inc.     

MTi: A method for user identification for multitouch displays

This paper describes MTi, a biometric method for user identification on multitouch displays. The method is based on features obtained only from the coordinates of the 5 touchpoints of one of the user's hands. This makes MTi applicable to all multitouch displays large enough to accommodate a human hand and detect 5 or more touchpoints without requiring additional hardware and regardless of the display's underlying sensing technology. MTi only requests that the user places his hand on the display with the fingers comfortably stretched apart. A dataset of 34 users was created on which our method reported 94.69% identification accuracy. The method's scalability was tested on a subset of the Bosphorus hand database (100 users, 94.33% identification accuracy) and a usability study was performed.     

Extracting data from human manipulation of objects towards improving autonomous robotic grasping

Humans excel in manipulation tasks, a basic skill for our survival and a key feature in our manmade world of artefacts and devices. In this work, we study how humans manipulate simple daily objects, and construct a probabilistic representation model for the tasks and objects useful for autonomous grasping and manipulation by robotic hands. Human demonstrations of predefined object manipulation tasks are recorded from both the human hand and object points of view. The multimodal data acquisition system records human gaze, hand and fingers 6D pose, finger flexure, tactile forces distributed on the inside of the hand, colour images and stereo depth map, and also object 6D pose and object tactile forces using instrumented objects. From the acquired data, relevant features are detected concerning motion patterns, tactile forces and hand-object states. This will enable modelling a class of tasks from sets of repeated demonstrations of the same task, so that a generalised probabilistic representation is derived to be used for task planning in artificial systems. An object centred probabilistic volumetric model is proposed to fuse the multimodal data and map contact regions, gaze, and tactile forces during stable grasps. This model is refined by segmenting the volume into components approximated by superquadrics, and overlaying the contact points used taking into account the task context. Results show that the features extracted are sufficient to distinguish key patterns that characterise each stage of the manipulation tasks, ranging from simple object displacement, where the same grasp is employed during manipulation (homogeneous manipulation) to more complex interactions such as object reorientation, fine positioning, and sequential in-hand rotation (dexterous manipulation). The framework presented retains the relevant data from human demonstrations, concerning both the manipulation and object characteristics, to be used by future grasp planning in artificial systems performing autonomous grasping.     

Whole-finger manipulation with a two-fingered robot hand

This paper describes a method for whole-finger rolling manipulation using a two-fingered robot hand. 'Whole-finger' refers to the use of the complete phalangeal surface during the manipulation. An example of whole-finger manipulation by the human hand is the rolling of a pen between two fingers. The proposed method is based on a two-dimensional model for modelling an object manipulation and is derived from a study of the movement of the contact line between both fingers. Also, the method uses tactile sensor information to estimate the contact point position together with the local curvature of the object. This whole-finger dexterous manipulation is demonstrated on a prototype two-fingered hand. This 5 d.o.f. hand consists of a tendon driven index and thumb, and is equipped with force and tactile sensors. The dimensions and performance of this device are 'human-sized'. A hybrid force-position control scheme is used. The hierarchical control structure is implemented on a dual transputer system. This paper first describes the kinematic model used for whole-finger manipulation. In the second part, the main emphasis is put on the mechanical design and on the transputer-based control system.     

Modeling of Viscoelastic Contacts and Evolution of Limit Surface for Robotic Contact Interface

Viscoelastic contact is a type of contact which includes, in addition to linear or nonlinear elastic response, time-dependent response due to relaxation or creep phenomena that govern the contact behavior. The characteristics of the time-dependent relaxation of such a viscoelastic contact are typically exponentially decaying functions, and exponentially growing functions for creep, respectively. Such contacts can be found in anthropomorphic robotic fingers, soft materials, viscoelastic skin with rigid core, and human fingers and feet. In this paper, the nature of viscoelastic contacts is investigated, and the evolution of their friction limit surfaces and of the pressure distributions at the contact interface are studied. Two cases commonly found in robotic grasping and manipulation are discussed. Based on the modeling formulation, it is found that the two important parameters of analysis and modeling for such contacts, i.e., the radius of contact area and the profile of pressure distribution, can be chosen using proposed coupling equations as the viscoelastic contact interface evolves with time. The new contribution of this paper includes a proposal of coupling equations between the two important parameters to describe the viscoelastic contact interface, and a study of the evolution of limit surfaces for viscoelastic contact interface due to temporal dependency, and the implication on grasp stability. It is found from the evolution of limit surfaces that when normal force is applied with typical viscoelastic contacts, grasp becomes more stable as time elapses. The modeling can be applied to the design of fingertips and the analysis of robotic grasping and manipulation involving viscoelastic fingers     

The design and empirical evaluations of 3D positioning techniques for pressure-based touch control on mobile devices

The previous three degrees of freedom (DOF) 3D touch translations require more than one finger (usually two hands) to be performed, which limits their usability on mobile devices that need one hand to be held in most occasions. Given that the pressure-sensitive touch screen will become ubiquitous in the near future, we presented a pressure-based 3DOF 3D positioning technique that only uses one finger in operating. Our technique collects the normal force of the touch pressure and uses it to represent the depth value in 3D translating. Then we conducted several groups of tightly controlled user studies to conclude (1) how different strategies of pressure recognition will affect 3D translating and (2) how is the performance of the pressure-based manipulation compared to the previous two-fingered technique. Finally, we discussed some guidelines to help developers in the design of the pressure-sensing technique in 3D manipulations.     

A study on the relationships among hand muscles and form factors of large-screen curved mobile devices

The aim of this study is to determine the effect of form factors (curvature rate, depth) of mobile devices on hand comfort when using large-screen curved mobile devices. To that end, four muscles (APB, APL, FDI, EDC) were selected and mockups for experiment were developed with four curvature rate levels (flat/100/200/300R) and five depth levels (3/5/7/9/11 mm). User experiment was conducted using three interactions (tapping, dragging, and texting) that usually take place on smartphones. EMG signal of each participant was measured. Combination of curvature and depth could maximize the physical comfort of hands and fingers through the experiment so that the interaction effect between form factors and tasks could be identified. Our results revealed that only APL and FDI had statistically significant muscle activity amounts across the curvature rates. With respect to effect of depth, all four muscles showed statistically significant difference in muscle activity. Moreover, results from the interaction effect among task levels of two tasks (target location of tapping and direction of dragging) and two form factors showed no interaction effect in either tapping or texting. The results of this study could be used to determine the optimal range for form factors of curved screen mobile devices in terms of physical comfort of the hand while using large-screen devices.     

多指手协调操作中内力的计算

机器人多指手协调操作物体时,合理地确定手指对被操作物体的作用力是必要的.本文将手指尖与被操作物体之间接触模拟为具有摩擦的点接触,对由多指手与被操作物体组成的这样一个速度较低的系统作了静力分析,并对多指手操作物体时的操作力作了合理的分配,提出基于力矩最小的内力的最优计算方法,在计算中,充分考虑到手指只能推而不能拉物体这一实事.最后,以4个手指操作一个圆柱形物体为例.对操作过程作了图形仿真.     

Evaluation of direct manipulation using finger tracking for complex tasks in an immersive cube

A solution for interaction using finger tracking in a cubic immersive virtual reality system (or immersive cube) is presented. Rather than using a traditional wand device, users can manipulate objects with fingers of both hands in a close-to-natural manner for moderately complex, general purpose tasks. Our solution couples finger tracking with a real-time physics engine, combined with a heuristic approach for hand manipulation, which is robust to tracker noise and simulation instabilities. A first study has been performed to evaluate our interface, with tasks involving complex manipulations, such as balancing objects while walking in the cube. The user’s finger-tracked manipulation was compared to manipulation with a 6 degree-of-freedom wand (or flystick), as well as with carrying out the same task in the real world. Users were also asked to perform a free task, allowing us to observe their perceived level of presence in the scene. Our results show that our approach provides a feasible interface for immersive cube environments and is perceived by users as being closer to the real experience compared to the wand. However, the wand outperforms direct manipulation in terms of speed and precision. We conclude with a discussion of the results and implications for further research.     

Integrality and separability of multitouch interaction techniques in 3D manipulation tasks

Multitouch displays represent a promising technology for the display and manipulation of data. While the manipulation of 2D data has been widely explored, 3D manipulation with multitouch displays remains largely unexplored. Based on an analysis of the integration and separation of degrees of freedom, we propose a taxonomy for 3D manipulation techniques with multitouch displays. Using that taxonomy, we introduce Depth-Separated Screen-Space (DS3), a new 3D manipulation technique based on the separation of translation and rotation. In a controlled experiment, we compared DS3 with Sticky Tools and Screen-Space. Results show that separating the control of translation and rotation significantly affects performance for 3D manipulation, with DS3 performing faster than the two other techniques.     

Computational Simulation of Alternative Photographic Processes

We present a novel computational framework for physically and chemically‐based simulations of analog alternative photographic processes. In the real world, these processes allow the creation of very personal and unique depictions due to the combination of the chemicals used, the physical interaction with liquid solutions, and the individual craftsmanship of the artist. Our work focuses not only on achieving similar compelling results, but on the manual process as well, introducing a novel exploratory approach for interactive digital image creation and manipulation. With such an emphasis on the user interaction, our simulations are devised to run on tablet devices; thus we propose the combination of a lightweight data‐driven model to simulate the chemical reactions involved, with efficient fluids simulations that modulate them. This combination allows realistic gestures‐based user interaction with constant visual feedback in real‐time. Using the proposed framework, we have built two prototypes with different tradeoffs between realism and flexibility, showing its potential to build novel image editing tools.     

Effects of Switchable DOF for Mid-Air Manipulation in Immersive Virtual Environments

In the context of mid-air manipulation, this paper presents the effects of allowing the user to dynamically switch between 1DOF, 2DOF, and 3DOF operations. Such “manipulation with switchable DOF” has been widely used in commercial graphics packages and is increasingly being adopted for virtual reality editing, where the 3D scene is constructed through mid-air interaction in immersive virtual environments. However, its effectiveness and advantages/disadvantages have not been investigated. This paper compares “manipulation with switchable DOF” with “manipulations with DOF separation,” which allows only 1DOF operations, and “manipulation without DOF separation,” which provides 3DOF operations. Using translation, rotation, and scaling, three methods were evaluated in terms of completion time and precision. The experiment results showed that “manipulation with switchable DOF” outperformed “manipulation with DOF separation” in terms of completion time whereas three methods were comparable in terms of precision. “Manipulation with switchable DOF” was further analyzed, and the results showed that the more 3DOF operations led to the shorter completion time and the more 1DOF operations led to the higher precision.     

多指手操作:运动学算法和实验

本文探讨了多指手的运动学操作问题，即给出被抓物体所要运动的轨迹，如何用多指手进行操作以实现物体的运动，也即怎样根据物体的运动来确定各手指的运动．我们提出和考察了三种运动学操作算法：广义逆算法、接触轨迹控制算法和抓持优化算法．每种算法各有其特点，可根据具体的操作任务加以采用，甚至可综合用于复杂操作任务的不同阶段．在实验系统ＨＫＵＳＴＨＡＮＤ上以两指手操作蓝球的实验实现了这些算法，考察了其可行性和有效性．     

Local open‐ and closed‐loop manipulation of multiagent networks

The manipulation of networked cyberphysical devices via local external actuation or feedback control is explored, in the context of a canonical multiagent dynamical system that is engaged in a consensus or synchronization task. One main focus is to understand whether or not, and how easily, a stakeholder can manipulate the full network's dynamics by hijacking only one agent's actuation signal. Explicit spectral characterizations are given of the energy (effort) required to manipulate the dynamics. These characterizations are used to (1) gain structural insights into ease of manipulation, (2) show that manipulation along the consensus manifold is easy, and (3) address network design to enable or prevent manipulation. Additionally, it is shown that the multiagent system can be manipulated effectively along the consensus manifold using local feedback controls, which do not require model knowledge or wide‐area measurements.     

Control of twisting manipulation using a multi-fingered robotic hand with a common rotational axis

The main purpose of the present study is to prove the usability of a mechanism with a common rotational axis during twisting manipulation using a multi-fingered robotic hand where two fingers and two other fingers can independently rotate in inner and outer circles with a dual turning mechanism. Although various types of conventional multi-fingered hands have potential capability to achieve twisting manipulations such as opening a bottle cap from within a hand, it is well-known that such tasks are difficult to execute quickly due to limited working space of the fingers and complexity of control. The proposed hand with a common rotational axis is effective in rotational manipulation around a particular axis, where each joint role assignment is completely decoupled into internal force control for grasping an object and velocity control around the axis for rotating the object. We prove the usability of this mechanism with a common rotational axis through the use of a control scheme, and show experimental results involving manipulation tasks where twisting manipulation is dominant.

     

Neural-Network-Based Contact Force Observers for Haptic Applications

In the majority of robotic and haptic applications, including manipulation and human-robot interaction, contact force needs to be monitored and controlled. Transparent implementation of bilateral teleoperation or haptic controllers necessitates the exchange of operator and environment contact forces. This requires the use of expensive commercially available force/torque sensors, which are rather bulky, are vulnerable to impact forces, and increase system inertia and compliance. An alternative solution is the use of dynamic force observers, which estimate external forces using system dynamic model. However, due to the uncertainties in system dynamic structure and parameters, these model-based observers do not produce accurate force estimates, and often create a dynamic lag that may cause bandwidth limitation and instability. This paper proposes two neural-network-based force/torque observers that do not require a system dynamic model. The observers can estimate human hand force and environment contact force with up to 98.3% accuracy in the sense of mean-square error, and with negligible dynamic lag. The performance of the proposed observers are extensively analyzed in separate human-robot and robot-environment experimental settings, and in a two-channel bilateral teleoperation control loop with multiple runs with two Planar Twin-Pantograph haptic devices