Machine vision based path planning for a robotic golf club head welding system

A path planning method based on machine vision techniques is constructed for a golf-club head robotic welding system. This system uses 3D machine vision techniques to recognize the weldseam and generates a welding path for the robot. The location of the weldseam is discovered by applying a Sobel mask to the captured data. A Laplace mask is also useful to filter out the noise points due to the scatter light refraction of tack-welding spots. The weldseam is then replenished and smoothed out by a B-spline curve fitting. The task frame of the weldseam is computed by finding the tangent, normal, and bi-normal of the curve. The robotic welding path is obtained by further rotations and translation along the axes of the task frame according to the requirement of the welding attitude. The developed machine vision technique and the mathematic framework pertaining to the generation of a welding task frame can readily be used for various three-dimensional welding tasks.     

Measurement error analysis and accuracy enhancement of 2D vision system for robotic drilling

Robotic drilling for aircraft structures demands higher accuracy on industrial robots than their traditional applications. Positioning error measurement and compensation based on 2D vision system is a cost-effective way to improve the positioning accuracy in robotic drilling. In this paper, we first discuss the principle of error measurement and compensation with a 2D vision system for robotic drilling and the determination of tool center point of the vision system so that the Abbe errors are eliminated in the measurement process. Measurement errors due to nonideal measurement conditions, i.e. nonperpendicularity of the camera optical axis to the workpiece surface and incorrect object distance, are mathematically modeled and experimentally verified. A method utilizing four laser displacement sensors is proposed to ensure perpendicularity of the camera optical axis to the workpiece surface and correct object distance in the measurement process, and hence to achieve high accuracy in 2D vision-based measurement. Experiments performed on a robotic drilling system show that the 2D vision system can achieve an accuracy of approximately 0.1 mm with the proposed method.     

Development of a monocular vision system for robotic drilling

Robotic drilling for aerospace structures demands a high positioning accuracy of the robot, which is usually achieved through error measurement and compensation. In this paper, we report the development of a practical monocular vision system for measurement of the relative error between the drill tool center point （TCP） and the reference hole. First, the principle of relative error measurement with the vision system is explained, followed by a detailed discussion on the hardware components, software components, and system integration. The elliptical contour extraction algorithm is presented for accurate and robust reference hole detection. System calibration is of key importance to the measurement accuracy of a vision system. A new method is proposed for the simultaneous calibration of camera internal parameters and hand-eye relationship with a dedicated calibration board. Extensive measurement experiments have been performed on a robotic drilling system. Experimental results show that the measurement accuracy of the developed vision system is higher than 0.15 mm, which meets the requirement of robotic drilling for aircraft structures.     

Development of a mobile robotic system for working in the double-hulled structure of a ship

Shipbuilding processes involve highly dangerous manual welding operations. Welding ship walls inside double-hulled structures presents a particularly hazardous environment for workers. This paper describes the “Rail Runner X” (RRX), a new robotic system that can move autonomously inside the walls of a double-hulled ship and automatically execute the required welding processes. The RRX robotic system is composed of a mobile platform and a welding robot consisting of a 3P3R serial manipulator. The robot is used to weld U-shaped trajectories located between two longitudinal stiffeners. The mobile platform enables traverse movements onto neighboring longitudinal stiffeners. The entire cross section of the robotic system is small enough to be placed inside the double-hulled structure via a conventional access hole from the outside shipyard floor. The overall engineering design process that led to the final robot solution developed is presented in this paper, including kinematic analysis data and experimental results for verifying the autonomous movement and welding performance.     

微操作机器人系统三维标定的研究与实现

提出三维标定方法 ,提出用这种方法标定的精度主要取决于三维标定块的加工精度和标定过程中对标定块的定位精度 ,给出实现三维标定的关键技术     

基于投影仪和摄像机的结构光视觉标定方法

为实现基于投影仪和摄像机的结构光视觉系统连续扫描,需要计算投影仪投影的任意光平面与摄像机图像平面的空间位置关系,进而需要求取摄像机光心与投影仪光心之间的相对位置关系。求取摄像机的内参数,在标定板上选取四个角点作为特征点并利用摄像机内参数求取该四个特征点的外参数,从而知道四个特征点在摄像机坐标系中的坐标。利用投影仪自身参数求解特征点在投影仪坐标系中的坐标,从而计算出摄像机光心与投影仪光心之间的相对位置关系,实现结构光视觉标定。利用标定后的视觉系统,对标定板上的角点距离进行测量,最大相对误差为0.277%,表明该标定算法可以应用于基于投影仪和摄像机的结构光视觉系统。     

Design,system model and development of customized electronic light barriers for robotic and mechatronic applications

In recent decades, indispensability of customized development of industrial-grade products has been widely recognized. The present paper describes the design, modeling and indigenous hardware development of such a product, namely, ‘Electronic Light Barrier’, with two separate (design) variations to suit user-specific applications in respective cases, pertaining to robotic and mechatronic systems. One of the versions of the said product has been crystallized as an obvious choice for the sensor-operated guarding system for machine tools and/or manipulators in a shop-floor environment, while the other variant is earmarked for specific application in industrial metrology. The developments have been aimed to facilitate round-the-clock industrial operation, with an insight to critical estimation of analytical modeling and system performance. Both varieties of the developed product encompass instrumented hardware, comprising infrared type sensory elements in a pre-conceived layout. Although the dimensionally larger variant is aptly suited for potential application related to workplace safety and ‘object’ detection and the smaller variant can act as a perfect choice for metrology-based operation, both variants rely on similar control algorithm and activation logic.     

Design and Calibration of an Opto-mechanical Appliance for 3D Non-contact Orthopedic Measurements Part II: Calibration Methodology and Experimental Results

A multistep iterative calibration methodology for the opto-mechanical system introduced in Part I is proposed. The methodology makes use of a monoview coplanar set of control points, whose number has been determined on the basis of both geometrical considerations and the results of a statistical analysis aiming at assessing the procedure stability in the case of noisy image data. The calibration procedure is carried out comparing the theoretical and observed images of the calibration pattern. Both synthetic and real data have been employed to test the calibration procedure, which proved to be accurate and efficient. The experimental results achieved by the calibrated system are satisfactory in terms of measurement precision.     

Design and performance evaluation of a novel robotic catheter system for vascular interventional surgery

Vascular interventional surgery (VIS) is an effective treatment method for vascular diseases. However, there are many problems in traditional VIS, such as surgeons are radiated by X-ray, the lack of well skilled surgeons, the security of the surgery will be reduced due to the Surgeons’ fatigue, high risk of the surgery. To solve these problems, a robotic catheter system is needed to protect the surgeons and enhance the safety of the surgery. In this paper, a novel robotic catheter system with master–slave structure for VIS has been developed. This system is designed with the consideration of the operation method in traditional VIS, which allows the surgeon to operate a real catheter on the master side, then the surgeon make full use of the natural catheter manipulation experience and skills obtained in conventional catheter operation. The salve manipulator operates the catheter insert into the blood vessel with following the operation of the surgeon, and the operating force of the salve manipulator is detected. On the master side, a novel damper-based magnetorheological (MR) fluid is designed to realize the force feedback, which is also used to reappear the operation force from the salve manipulator. The damper connected directly with real catheter is a piston structure using the MR fluid to realize the force feedback. It can transmit the feedback force to surgeon’s hand through the operating catheter connected with damper, which seems that the surgeon operates the catheter beside the patient. The operating transparency of the developed system has been enhanced. The mechanism of the developed system has been introduced in detail. Performance evaluation experiments for the developed robotic catheter system have been done. The experimental results indicated that the developed robotic catheter system is fit for VIS.     

Requirements and design of a grasping system for personal robots

In this paper, we present a robotic grasping system for deployment in personal robots. The system learns how to grasp objects from experiments. This approach allows it to satisfy a number of requirements that we have identified as prerequisite for operation in personal robot environments. The system design consists of three control layers, each describing the control strategy of a predefined behavior. Learning of the behavior is performed using groups of neural networks. Testing of the system was performed in a simulated environment using a specially built grasping simulator and using a 15 objects database. Results show that, on average, each object needed 12 successful experiments before an accurate grasping model was achieved. Failed experiments averaged to 25% of the total experiments.     

Development of a physically behaved robot work cell in virtual reality for task teaching

Robot task teaching on a real work cell is expensive and sometimes risky. This cost and risk can be avoided by using virtual reality technology. Using the simulated environment in virtual reality (VR), the operator can practise, explore and preview the operations for possible problems that might occur during implementation. It is therefore of practical importance to build the virtual robot work cell in VR that can facilitate the study of the performance of robotic tasks such as robotic assembly. This paper describes our work in incorporating physical behaviours of virtual objects into VR for robot task teaching. To facilitate the task teaching, we developed visual and audio cues which help visualise the dynamic interactions between virtual objects. Dynamic sensing capability is incorporated in the simulated environment. A simplified force sensor is modelled and simulated. The physical behaviours of the virtual objects are simulated using physics-based approach. A virtual robot work cell is built incorporating the developed features and an example for the task teaching is given. The implementation includes view tracking using virtual camera, visual and audio rendering, and the user interface developed in the VR. The current implementation was carried out on a PC-based VR platform, with the programs developed using Watcom C++.     

CYCLOPS: A mobile robotic platform for testing and validating image processing and autonomous navigation algorithms in support of artificial vision prostheses

While artificial vision prostheses are quickly becoming a reality, actual testing time with visual prosthesis carriers is at a premium. Moreover, it is helpful to have a more realistic functional approximation of a blind subject. Instead of a normal subject with a healthy retina looking at a low-resolution (pixelated) image on a computer monitor or head-mounted display, a more realistic approximation is achieved by employing a subject-independent mobile robotic platform that uses a pixelated view as its sole visual input for navigation purposes. We introduce CYCLOPS: an AWD, remote controllable, mobile robotic platform that serves as a testbed for real-time image processing and autonomous navigation systems for the purpose of enhancing the visual experience afforded by visual prosthesis carriers. Complete with wireless Internet connectivity and a fully articulated digital camera with wireless video link, CYCLOPS supports both interactive tele-commanding via joystick, and autonomous self-commanding. Due to its onboard computing capabilities and extended battery life, CYCLOPS can perform complex and numerically intensive calculations, such as image processing and autonomous navigation algorithms, in addition to interfacing to additional sensors. Its Internet connectivity renders CYCLOPS a worldwide accessible testbed for researchers in the field of artificial vision systems. CYCLOPS enables subject-independent evaluation and validation of image processing and autonomous navigation systems with respect to the utility and efficiency of supporting and enhancing visual prostheses, while potentially reducing to a necessary minimum the need for valuable testing time with actual visual prosthesis carriers.     

Auto‐calibration of a projector–camera stereo system for projection mapping

Standard camera and projector calibration techniques use a checkerboard that is manually shown at different poses to determine the calibration parameters. Furthermore, when image geometric correction must be performed on a three‐dimensional (3D) surface, such as projection mapping, the surface geometry must be determined. Camera calibration and 3D surface estimation can be costly, error prone, and time‐consuming when performed manually. To address this issue, we use an auto‐calibration technique that projects a series of Gray code structured light patterns. These patterns are captured by the camera to build a dense pixel correspondence between the projector and camera, which are used to calibrate the stereo system using an objective function, which embeds the calibration parameters together with the undistorted points. Minimization is carried out by a greedy algorithm that minimizes the cost at each iteration with respect to both calibration parameters and noisy image points. We test the auto‐calibration on different scenes and show that the results closely match a manual calibration of the system. We show that this technique can be used to build a 3D model of the scene, which in turn with the dense pixel correspondence can be used for geometric screen correction on any arbitrary surface.     

融合2D/3D摄像机的方法与获取高精度三维视觉信息的装置

三维飞行时间摄像机可实时同步获取场景三维信息和灰度图像信息.虽然它存在图像分辨率和质量较差等问题,但它可作为二维摄像机的互补.本文借鉴立体视觉技术,提出了一种2D/3D摄像机融合的三维视觉信息获取方案.论文首先基于固定空间关系和相近视野原则,设计2D/3D立体摄像机系统对空间场景同步成像.结合三维TOF摄像机成像特性,论文借鉴立体视觉技术完成二维摄像机的高质量二维彩色图像与插补后的三维摄像机深度图像的匹配关联.因此,本方法可实现场景的高精度彩色图像和对应三维空间信息的实时同步获取,同时保留了二维摄像机的高质量彩色二维成像和三维摄像机的快速稠密三维信息获取的优势.2D/3D摄像机图像融合匹配算法复杂度低,匹配精度和准确度取决于二维摄像机和三维摄像机自身性能、摄像机标定参数精度和深度图像插补算法,不会引入新的运算误差.试验结果验证了本文算法的有效性和精确度.     

基于二维激光雷达的自动室内三维重建系统

设计了一个基于二维激光雷达的自动室内三维重建系统.系统的硬件由一套自行设计的基于2D激光雷达的三维扫描系统和一台电脑构成.介绍了系统的软件模块,提出了结合最近点迭代(ICP)和通用多边形裁剪(GPC)的3D平面场景合成方法.ICP能够获得不同采集位置之间的位置变化,以此能将各个不同位置获得的3D场景转换到同一坐标系下.场景合成时的碎平面问题通过GPC方法来解决.实验结果表明:该系统成本低,精度高,能稳定可靠地实现室内场景的自动三维重建.     

Comparison of 3D and 2D menus for cell phones

Few prior studies have directly compared 3D and 2D menus for cell phones. Because the technology available for cell phone interfaces has changed in recent years, interface guidelines for cell phones need to be re-evaluated, especially with regard to the use of 3D interfaces. In the present study, we first compared performance of tasks for menus with different breadths using three 3D menus (revolving stage, 3D carousel, and collapsible cylindrical tree) shown on the small display screen of an iPhone® simulator. Performance was best with the revolving stage menu, and there was a nonsignificant tendency for it to be rated as preferred by the participants. Then, we compared the 3D revolving stage menu to a 2D overview menu, for tasks of different complexity and menus of different breadths, on an actual iPhone. The 3D menu was preferred by users at high breadth levels, and the 2D menu showed better performance than the 3D menu with low memory load. From the results, recommendations for the design of menus for small displays were developed.     

Feature matching and affine transformation for 2D cell animation

This paper presents an application for feature matching and affine transformation in computer graphics. The study case considered is 2D cell animation, which is still a labor-intensive process in current cartoon film production. One key problem in automating 2D cell animation in inbetween frame generation. The objective of our work is to investigate how to automatically generate inbetweening from pairs of hand-drawn 2D key frames. Our technique first establishes the correspondence of feature points and then estimates the affine transformation between each pair of 2D key frames. The inbetween frames are then automatically generated by interpolation.     

Integration of Hough Transform of lines and planes in the framework of conformal geometric algebra for 2D and 3D robot vision

This paper presents the application of 2D and 3D Hough Transforms together with conformal geometric algebra to build 3D geometric maps using the geometric entities of lines and planes. Among several existing techniques for robot self-localization, a new approach is proposed for map matching in the Hough domain. The geometric Hough representation is formulated in such a way that one can easily relate it to the conformal geometric algebra framework; thus, the detected lines and planes can be used for algebra-of-incidence computations to find geometric constraints, useful when perceiving special configurations in 3D visual space for exploration, navigation, relocation and obstacle avoidance. We believe that this work is very useful for 2D and 3D geometric pattern recognition in robot vision tasks.     

Development of a monitoring system for keyboard users' performance

To precisely evaluate the typing performance of a VDT user, a program named KBlog was developed to record the typing activity under the Microsoft Windows operating system. This program is small and simply framed in order to avoid overloading of the operating system. Without interfering with the typing task, this program can record the time of every pressing and releasing movement of each keystroke at the millisecond level. The accuracy and reliability of KBlog was tested by comparing time intervals recorded by KBlog to time intervals of keyboard output electrical signals recorded by an oscilloscope. In the regression analysis on these two indicators, results of high correlation coefficients of almost 1.000 and intercepts within acceptable levels of around 1 ms indicated sufficient accuracy and reliability of this program. Further applications are discussed in this paper concerning both laboratory and field researches.