Enhancing e-quality for manufacture using Kalman Filter calibrated visual robotic control

EQM integrates a host of advanced technologies to attain a holistic quality control strategy at each stage of the production processes. This differentiates it from conventional quality control practices. A critical problem in EQM is the inaccuracies in vision guided robotic positioning control, especially when the production equipment is situated remotely and controlled over the network. Such a situation introduces many variations in the process that are difficult to control or to eliminate (e.g., network delay, and lens distortion effects). This study applies Kalman Filtering techniques to overcome the shortcomings in the mathematical modeling of vision calibration, in which mathematical equations fail to address the dynamic state of the process. The Kalman Filtering techniques developed in this study effectively reduce the errors in the positioning accuracy of vision-generated robotic control. The Kalman Filtering techniques prove to be a valuable location estimation algorithm for remote robotic control, even when the data contain noise inherent in vision calibration and network-based communication. Experiments validate the proposed scheme; it significantly enhances the accuracy of vision-guided robotic control.     

Improvement of vision guided robotic accuracy using Kalman filter

This study addresses one of the most prevalent, yet difficult problems in vision calibration, namely, the accuracy of remote, vision-guided robotic control in tune with the notion of e-quality for manufacture (EQM). In all areas of robot work space, image distortion occurs due to imperfect lens curvatures, which in turn induces an inaccurate vision guidance. Non-uniform nature of image distortion is effectively rectified, using the Kalman Filtering technique. Consequently, the robotic positioning accuracy is significantly improved. In recent years, stringent quality standards and intense competition compelled many companies to adopt new, advanced technologies to further enhance their strategic competitiveness. EQM is an emergent technology better suited for today’s fast-changing, zero-defect production environment. The proposed methodology has great potential for improving the product quality and operational efficiency of networked robotic production system, which has been vindicated by the statistical analysis.     

GWM-view: Gradient-weighted multi-view calibration method for machining robot positioning

Multi-camera vision is widely used for guiding the machining robot to remove flash and burrs on complex automotive castings and forgings with arbitrary initial posture. Aiming at the problems of insufficient field of vision and regional occlusion in actual machining, a gradient-weighted multi-view calibration method (GWM-View) is proposed for the machining robot positioning based on the convergent binocular vision. Specifically, the mapping between each auxiliary camera and the main camera in the multi-view system is calculated by the inverse equation and intrinsic parameter matrix. Then, the gradient-weighted suppression algorithm is introduced to filter out the errors caused by camera angle variation. Next, the spatial coordinates of the feature points after suppression are used to correct the transformation matrix. Finally, the hand-eye calibration algorithm is implemented to transform the corrected data into the robot base coordinate system for the accurate positioning of the robot under multiple views. The experiment on the automotive engine flywheel shell indicates that the average positioning error is controlled within 1 mm under different postures. The stability and robustness of the proposed method are further improved while the positioning accuracy of the machining robot meets the requirements.     

Robot calibration using a 3D vision-based measurement system with a single camera

One of the problems that slows the development of off-line programming is the low static and dynamic positioning accuracy of robots. Robot calibration improves the positioning accuracy and can also be used as a diagnostic tool in robot production and maintenance. This work presents techniques for modeling and performing robot calibration processes with off-line programming using a 3D vision-based measurement system. The measurement system is portable, accurate and low cost, consisting of a single CCD camera mounted on the robot tool flange to measure the robot end-effector pose relative to a world coordinate system. Radial lens distortion is included in the photogrammetric model. Scale factors and image centers are obtained with innovative techniques, making use of a multiview approach. Results show that the achieved average accuracy using a common off-the-shelf CCD camera varies from 0.2 to 0.4 mm, at distances from 600 to 1000 mm from the target, respectively, with different camera orientations. Experimentation is performed on two industrial robots to test their position accuracy improvement using the calibration system proposed: an ABB IRB-2400 and a PUMA-500. The robots were calibrated at different regions and volumes within their workspace achieving accuracy from three to six times better when comparing errors before and after calibration, if measured locally. The proposed off-line robot calibration system is fast, accurate and easy to set up.     

Robust hand-eye calibration of 2D laser sensors using a single-plane calibration artefact

When a vision sensor is used in conjunction with a robot, hand-eye calibration is necessary to determine the accurate position of the sensor relative to the robot. This is necessary to allow data from the vision sensor to be defined in the robot's global coordinate system. For 2D laser line sensors hand-eye calibration is a challenging process because they only collect data in two dimensions. This leads to the use of complex calibration artefacts and requires multiple measurements be collected, using a range of robot positions. This paper presents a simple and robust hand-eye calibration strategy that requires minimal user interaction and makes use of a single planar calibration artefact. A significant benefit of the strategy is that it uses a low-cost, simple and easily manufactured artefact; however, the lower complexity can lead to lower variation in calibration data. In order to achieve a robust hand-eye calibration using this artefact, the impact of robot positioning strategies is considered to maintain variation. A theoretical basis for the necessary sources of input variation is defined by a mathematical analysis of the system of equations for the calibration process. From this, a novel strategy is specified to maximize data variation by using a circular array of target scan lines to define a full set of required robot positions. A simulation approach is used to further investigate and optimise the impact of robot position on the calibration process, and the resulting optimal robot positions are then experimentally validated for a real robot mounted laser line sensor. Using the proposed optimum method, a semi-automatic calibration process, which requires only four manually scanned lines, is defined and experimentally demonstrated.     

线阵相机的圆环旋转标定方法

由于传统线阵相机的标定过程复杂,且对标定物精度要求较高,难以保证缺陷的定位精度,本文提出一种线阵相机的圆环旋转标定方法以提高缺陷的定位精度。该方法设计一种新型的圆环形标定板,在静态标定基础上通过旋转线阵相机采集相机视线与圆的交点的坐标,得到旋转角度以及多组标定点,建立线阵相机的成像模型和径向畸变模型,通过非线性优化整体误差函数求解相机的内参和畸变参数,同时分析相机不同旋转角度对标定精度的影响。实验结果表明,当θ≤20°时,该方法的标定精度在0.35 pixel以内,满足实际检测的定位要求,并且在PCB缺陷检测中得到较好的验证。     

基于神经网络的工业机器人视觉抓取系统设计

针对机器人示教编程方法导致的工件位置固定、抓取效率低下的问题,研究神经网络在机器人视觉识别与抓取规划中的应用,建立了视觉引导方案,通过YOLOV5神经网络模型开发视觉识别系统,识别物体的种类,同时获取待抓取物体定位点坐标。提出了机器人六点手眼标定原理并进行标定实验,提出了针对俯视图为圆形或长方形物体的定位方法。最后针对3种物体进行了180次的抓取实验,实验的综合平均抓取成功率约为92.8%,验证了视觉识别和抓取机器人系统具备实际应用的可能性,有效提高了抓取效率。     

基于单目视觉的智能物料分拣机器人的设计

为解决目前工业物料分拣机器人发展的需求问题,以模拟自动化物流系统的作业流程为目标,提出了一种基于单目视觉的智能物料分拣机器人的设计。为实现物料的自动分拣过程,以STM32作为核心控制器,驱动OV2640摄像头对图像进行实时采集、处理与分析,实现颜色识别和目标定位,并将处理结果传送给驱动控制系统。在图像标定方面,采用读取TFT屏中目标的坐标和求取目标的实际坐标,通过MATLAB软件来进行数据拟合,找出两个坐标的函数关系,从而实现目标的定位。采用遍历腐蚀算法、增量式PID算法、DBCSAN算法和Dijkstra算法,分别完成对随机摆放的物料的自动识别和定位、电机控制和路径规划。实验与竞赛结果表明,该单目视觉分拣机器人的分拣准确率和效率高,能够实现智能分拣功能。     

Camera calibration issues in robot calibration with eye-on-hand configuration

One of the possible methods for accurate, fast, low-cost and automated robot calibration is to employ a single camera rigidly mounted to the robot end-effector together with a single camera calibration board. The end-effector pose is measured by calibration of the camera at every robot measurement configuration. This paper contends that, with several modifications, Tsai's radial alignment constraint (RAC) camera calibration method can be made a fast and sufficiently accurate pose measurement technique. This paper focuses on speed, accuracy and cost enhancement of RAC-based camera calibration. A fast RAC-based algorithm is proposed, which cuts the computation time of Tsai's original algorithm by about a 5: 1 ratio while keeping its accuracy within the tolerances required for a successful robot calibration. A low-cost method for estimation of the ratio of scale factors of the camera/vision system is also proposed. This method does not require a precision vertical micrometer stage to provide non-coplanar calibration points data for camera calibration. Finally, the phenomenon of perspective projection distortion of circular camera calibration points is fully analyzed and error compensation methods are proposed.     

机械臂绝对定位精度标定关键技术综述

目前我国飞机和卫星装配仍主要采用人工装配,装配技术陈旧、机器人绝对定位精度低等问题难以满足飞机和卫星高精度、高性能的要求,阻碍了工业机器人在航空制造行业的发展,因此,在机器人柔性自动化装配过程中,如何提高机械臂绝对定位精度的标定技术已成为学术界和工业界广泛关注的焦点。为了系统地分析和总结现有的研究成果,对绝对定位精度标定方法进行了分类探讨,归纳了国内外机械臂标定技术的研究现状,详细分析了误差不确定性、冗余参数的消除及最优测量结构选择性等关键技术,并对机械臂绝对定位精度标定技术的未来发展趋势进行了构想和展望。     

Self Localization Method Using Parallel Projection Model for Mobile Sensor in Navigation Applications

This paper presents a novel self localization method using parallel projection model for mobile sensor in navigation applications.The algorithm estimates the coordinate and the orientation of mobile sensor using projected references on visual image.The proposed method considers the lens non-linearity of the camera and compensates the distortion by using a calibration table.The method determines the coordinates and orientations with iterative process,which is very accurate with low computational demand.We...     

工业机器人加工轨迹双目3D激光扫描成像修正方法

为了提高工业机器人作业柔性,本文提出了一种基于双目CCD激光扫描3D成像的"眼在外"(Eye-to-hand:ETH)工业机器人末端(tool center point,TCP)运动轨迹在线修正方法.以激光切割机器人视觉引导为研究背景,降低加工过程机器人对物理工装定位精度的依赖.首先,为提高机器人视觉控制精度,研究了目...     

An on-line relative position and orientation error calibration methodology for workcell robot operations

Relative position and orientation inaccuracy always exists between a robot and the equipment with which it operates, especially in batch-type production cells that are subjected to dynamic changes. This inaccuracy causes robot relative positioning errors, and may even result in operation failure if the off-line programmed moving path is implemented without adjustment. To make use of off-line programming and simulation tools, an on-line calibration methodology for robot relative positioning inaccuracy was developed in this study. This methodology eliminates the need for time-consuming off-line calibrations relying on accurate models and expensive devices. An industrial robot system was enabled to detect and compensate automatically for relative positioning errors by incorporating a vision system, a 3-D force/torque sensor, and control strategies involving neural networks. The experimental results showed that this methodology is valid and robust in calibrating the relative position and orientation errors automatically without the need for mathematical models and complex off-line calibration procedures for model parameters. Consequently, batch-type production cells would be more flexible, adaptable and intelligent in accommodating dynamic workcell changes with less human effort.     

Accurate positioning of a drilling and riveting cell for aircraft assembly

Modern aircraft assembly demands assembly cells or machines with higher machining efficiency and accuracy. Thus, a dual-machine drilling and riveting cell is developed in this paper. We firstly discuss its physical design, as well as the automatic drilling and riveting process. With the automatic drilling and riveting cell, drilling and riveting production line of aircraft panels can be expected. The frame chain of the drilling and riveting cell is constructed to link the assembly cell to its task space, which is the kinematics base. System calibrations, including task space calibration, the sensor calibration of an orientation alignment unit, the floating calibration of the implicit hand-eye relationship, are explored. For high positioning accuracy, a multi-sensor servoing method is proposed for cell positioning. An orientation-based laser servoing strategy, which uses the feedback of the orientation errors measured by laser displacement sensors, is used to align drilling direction and camera shooting direction. Besides, A single-camera-based visual servoing is applied to align the tool center point (TCP) to reference holes, to obtain their coordinates for drilling position modification. Experiments of multi-sensor servoing for cell positioning are performed on an automatic drilling and riveting machine developed for the panel assembly of an aircraft in China. With the cell positioning method, the automatic drilling and riveting cell can approximately achieve an accuracy of 0.05 mm, which can adequately fulfill the requirement for the assembly of the aircraft.     

基于机器视觉的航空散热器智能定位钻孔技术

因某型航空散热器再制造的需要，需对产品大量的散热管进行重复的钻孔工作，为解决人工钻孔不便开发的问题开发出一种利用带机器视觉装置的工业机器人对散热器进行定位钻孔的技术。根据产品的特点选用了相应功能的工业机器人与机器视觉装置硬件，给出了智能定位钻孔系统的设计技术方案，建立了系统控制以及软件结构框架。通过最小二乘法对机器视觉采集的图像进行处理，给定了系统坐标标定的过程和方法,明确了机器人定位精度及视觉精度的计量方法。试验结果表明，该系统提高了钻孔效率和精度，可应用于实际生产。     

机器人手眼立体视觉标定研究

以固高GRB-400型机器人和CCD组成机器人手眼系统,分析了摄像机的成像模型,采用了基于直接将图像坐标映射到机器人参考坐标的“黑箱”思想,从图像坐标直接计算出目标位置的方法,用于立体定位的摄像机手眼标定,该方法通过保持机器人末端执行器到机器人参考坐标系旋转矩阵来简化复杂的相机标定过程,最后通过实验验证了该方法的可行性,并分析了实验误差产生的原因,并提出了相应的解决措施。     

基于OpenCV的遥操作工程机器人双目视觉定位技术

针对当前遥操作工程机器人双目视觉定位技术匹配精准度低,导致定位误差过大的问题,提出种基于OpenCV改进的遥操作工程机器人双目视觉定位技术。在分析HSV色彩空间后,建立适合工程机器人的颜色特征识别空间体系,通过分析图像特征及运动坐标确定圆形光点,利用提供的靶点目标,创建模板后通过双目视觉获取具有靶点特征的其他图像,将图像代入OpenCV技术函数库中。在OpenCV技术函数库中通过光流法对图像进行函数匹配,应用将目标的背景模型与图像的处理方式分割开来,提取背景与干扰因素的全部信息,利用二值化阈值处理运动目标的形态,实现无干扰图像显示,确保定位结果的准确性。实验结果表明,基于OpenCV的遥操作工程机器人双目视觉定位技术能够有效提高匹配精度,降低定位误差,具有一定的实际应用价值。     

A vision guided robotic system for flexible gluing process in the footwear industry

This paper presents a novel gluing machine comprising a Cartesian robot and a vision system. The vision system enables the location and reconstruction of the shape of objects to be glued; the detected information is then used to plan the trajectory of the robot whose end-effector is a glue gun and to move the robot with an error suitable to industrial gluing operations.A calibration procedure that enables transforming coordinates between the robot frame and the vision system frame is described. The calibration considers several mechanical inaccuracies and its effectiveness was evaluated using error maps.In particular, the paper examines objects to be glued along their edges, as frequently occurs for fabrics, leathers, and shoe soles. For this, two procedures to plan the trajectories of the robot are proposed: the first is for objects that can be treated as flat 2D objects, that is, their height variation is negligible; the second procedure is for 3D objects, that is, those with significant height variation.Several applicative examples are reported to highlight the flexibility of the gluing process.     

A novel kinematic parameters calibration method for industrial robot based on Levenberg-Marquardt and Differential Evolution hybrid algorithm

The poor absolute positioning accuracy of industrial robots is the main obstacle for its further application in precision grinding of complex surfaces, such as blisk, blade, etc. Based on the established kinematic error model of a typical industrial robot FANUC M710ic/50, a novel kinematic parameters calibration method is proposed in this paper to improve the absolute positioning accuracy of robot. The pre-identification of the kinematic parameter deviations of robot was achieved by using the Levenberg-Marquardt algorithm. Subsequently, these identified suboptimal values of parameter deviations were defined as central values of the components of initial individuals to complete accurate identification by using Differential Evolution algorithm. The above two steps, which were regarded as the core of this Levenberg-Marquardt and Differential Evolution hybrid algorithm, were used to obtain the preferable values for kinematic parameters of the robot. On this basis, the experimental investigations of kinematic parameters calibration were conducted by using a laser tracker and numerical simulation method. The results revealed that the robot positioning error decreased from 0.994 mm, initial positioning error measured by laser tracker, to 0.262 mm after calibration with this proposed hybrid algorithm. The absolute positioning accuracy has increased by 40.86% than that of the Levenberg-Marquardt algorithm, increased by 40.31% than that of the Differential Evolution algorithm, and increased by 25.14% than that of the Simulated Annealing algorithm. This work shows that the proposed kinematic parameters calibration method has a significant improvement on the absolute positioning accuracy of industrial robot.     

基于机器视觉的机械手装配系统设计

为了解决工业生产过程中料板的自动装配问题,提出了一种基于机器视觉的机械手定位装配检测系统,建立了实验平台。系统通过工业相机对料盘进行图像采集,将所采集的信息传送给工控机,借助图像处理程序,采用图像预处理实现了对图像的降噪,结合BLOB分析对图像特征进行了提取,获得了料盘上装配孔的位置信息,通过控制机械手动作,移动到装配位置,旋转相应角度,实现料板的自动定位装配。实验结果表明,该系统装配准确度高,误差小,满足了工业上的要求。