AR-based deep learning for real-time inspection of cable brackets in aircraft

In the process of aircraft assembly, there exist numerous and ubiquitous cable brackets that shall be installed on frames and subsequently need to be manually verified with CAD models. Such a task is usually performed by special operators, hence is time-consuming, labor-intensive, and error-prone. In order to save the inspection time and increase the reliability of results, many researchers attempt to develop intelligent inspection systems using robotic, AR, or AI technologies. However, there is no comprehensive method to achieve enough portability, intelligence, efficiency, and accuracy while providing intuitive task assistance for inspectors in real time. In this paper, a combined AR+AI system is introduced to assist brackets inspection in a more intelligent yet efficient manner. Especially, AR-based Mask R-CNN is proposed by skillfully integrating markerless AR into deep learning-based instance segmentation to generate more accurate and fewer region proposals, and thus alleviates the computation load of the deep learning program. Based on this, brackets segmentation can be performed robustly and efficiently on mobile devices such as smartphones or tablets. By using the proposed system, CAD model checking can be automatically performed between the segmented physical brackets and the corresponding virtual brackets rendered by AR in real time. Furthermore, the inspection results can be directly projected on the corresponding physical brackets for the convenience of maintenance. To verify the feasibility of the proposed method, experiments are carried out on a full-scale mock-up of C919 aircraft main landing gear cabin. The experimental results indicate that the inspection accuracy is up to 97.1%. Finally, the system has been deployed in the real C919 aircraft final-assembly workshop. The preliminary evaluation reveals that the proposed real-time AR-assisted intelligent inspection approach is effective and promising for large-scale industrial applications.     

A deep learning-enabled human-cyber-physical fusion method towards human-robot collaborative assembly

Human-robot collaborative (HRC) assembly has become popular in recent years. It takes full advantage of the strength, repeatability and accuracy of robots and the high-level cognition, flexibility and adaptability of humans to achieve an ergonomic working environment with better overall productivity. However, HRC assembly is still in its infancy nowadays. How to ensure the safety and efficiency of HRC assembly while reducing assembly failures caused by human errors is challenging. To address the current challenges, this paper proposes a novel human-cyber-physical assembly system (HCPaS) framework, which combines the powerful perception and control capacity of digital twin with the virtual-reality interaction capacity of augmented reality (AR) to achieve a safe and efficient HRC environment. Based on the framework, a deep learning-enabled fusion method of HCPaS is proposed from the perspective of robot-level fusion and part-level fusion. Robot-level fusion perceives the pose of robots with the combination of PointNet and iterative closest point (ICP) algorithm, where the status of robots together with their surroundings could be registered into AR environment to improve the human's cognitive ability of complex assembly environment, thus ensuring the safe HRC assembly. Part-level fusion recognizes the type and pose of parts being assembled with a parallel network that takes an extended Pixel-wise Voting Network (PVNet) as the base architecture, on which assembly sequence/process information of the part could be registered into AR environment to provide smart guidance for manual work to avoid human errors. Eventually, experimental results demonstrate the effectiveness and efficiency of the approach.     

Prediction maintenance integrated decision-making approach supported by digital twin-driven cooperative awareness and interconnection framework

Predictive maintenance is one of the important technical means to guarantee and improve the normal industrial production. The existing bottlenecks for popularization and application are analyzed. In order to solve these problems, a cooperative awareness and interconnection framework across multiple organizations for total factors that affect prediction maintenance decision-making is discussed. Initially, the structure and operation mechanism of this framework are proposed. It is designed to support the sharing of data, knowledge and resources. As a key supporting technology, the digital twin is also integrated into it to improve the accuracy of fault diagnosis and prediction and support making a maintenance plan with higher accuracy and reliability. Then, under this framework, an integrated mathematical programming model is established with considering the parameter uncertainty and an NSGA-II hybrid algorithm is utilized to solve it. Moreover, an adjustment strategy for a maintenance plan is discussed in response to the dynamic characteristics of the actual maintenance environment. Finally, a case, prediction maintenance decision-making for bearings in grinding rolls of the large vertical mill, is studied. Analysis results verify the advantage of the integrated solving mechanism based on the proposed framework. The framework and integrated decision-making approach can guide the implementation of predictive maintenance with higher accuracy and reliability for industrial enterprises.     

Intelligent scheduling of a feature-process-machine tool supernetwork based on digital twin workshop

Modern manufacturing enterprises are shifting toward multi-variety and small-batch production. By optimizing scheduling, both transit and waiting times within the production process can be shortened. This study integrates the advantages of a digital twin and supernetwork to develop an intelligent scheduling method for workshops to rapidly and efficiently generate process plans. By establishing the supernetwork model of a feature-process-machine tool in the digital twin workshop, the centralized and classified management of multiple data types can be realized. A feature similarity matrix is used to cluster similar attribute data in the feature layer subnetwork to realize rapid correspondence of multi-source association information among feature-process-machine tools. Through similarity calculations of decomposed features and the mapping relationships of the supernetwork, production scheduling schemes can be rapidly and efficiently formulated. A virtual workshop is also used to simulate and optimize the scheduling scheme to realize intelligent workshop scheduling. Finally, the efficiency of the proposed intelligent scheduling strategy is verified by using a case study of an aeroengine gear production workshop.     

梯级送风对冷藏车厢内温度场的影响分析

合理的厢内温度场是保证冷藏车运输货物品质、节能降耗的关键因素之一。为提高厢内温度场的均匀性,本文提出了单温区冷藏车的梯级送风模式。无梯级送风时,冷风在车厢内形成整体环流,流动方向较为集中,不利于整体降温;有梯级送风时,车厢顶部增加了风机,冷风速度得以提高,且流动方向更加分散,有利于提高整体降温速度和温度场均匀性。建立了冷藏车厢的仿真模型,利用CFD研究梯级送风对空仓时冷藏车厢内温度场的影响,并进行了实验验证。以射流区平均温度、车厢内整体平均温度、温度场不均匀系数和温度极差作为评价指标,对实验中采集到的温度数据进行对比分析。结果表明：梯级送风模式能够有效降低冷藏车厢内的射流区平均温度、整体平均温度、温度场不均匀系数,减小温度极差,提高降温速度;在射流区,梯级送风的影响最为显著,该处的降温幅度和降温速度都有明显优化;在车厢尾部近地面的拐角处,梯级送风的降温效果虽不明显,但降温速度明显加快。     

Elasto-geometrical error and gravity model calibration of an industrial robot using the same optimized configuration set

Position error is a significant limitation for industrial robots in high-precision machining and manufacturing. Efficient error measurement and compensation for robots equipped with end-effectors are difficult in industrial environments. This paper proposes a robot calibration method based on an elasto–geometrical error and gravity model. Firstly, a geometric error model was established based on the D-H method, and the gravity and compliance error models were constructed to predict the elastic deformation caused by the self-weight of the robot. Subsequently, the position error model was established by considering the attitude error of the robot flange coordinate system. A two-step robot configuration selection method was developed based on the sequential floating forward selection algorithm to optimize the robot configuration for calibrating the position error and gravity models. Then, the geometric error and compliance coefficient were identified simultaneously based on the hybrid evolution algorithm. The gravity model parameters were identified based on the same algorithm using the joint torque signal provided by the robot controller. Finally, calibration and compensation experiments were conducted on a KR-160 industrial robot equipped with a spindle using a laser tracker and internal robot data. The experimental results show that the robot tool center point error can be significantly improved by using the proposed method.     

GWM-view: Gradient-weighted multi-view calibration method for machining robot positioning

Multi-camera vision is widely used for guiding the machining robot to remove flash and burrs on complex automotive castings and forgings with arbitrary initial posture. Aiming at the problems of insufficient field of vision and regional occlusion in actual machining, a gradient-weighted multi-view calibration method (GWM-View) is proposed for the machining robot positioning based on the convergent binocular vision. Specifically, the mapping between each auxiliary camera and the main camera in the multi-view system is calculated by the inverse equation and intrinsic parameter matrix. Then, the gradient-weighted suppression algorithm is introduced to filter out the errors caused by camera angle variation. Next, the spatial coordinates of the feature points after suppression are used to correct the transformation matrix. Finally, the hand-eye calibration algorithm is implemented to transform the corrected data into the robot base coordinate system for the accurate positioning of the robot under multiple views. The experiment on the automotive engine flywheel shell indicates that the average positioning error is controlled within 1 mm under different postures. The stability and robustness of the proposed method are further improved while the positioning accuracy of the machining robot meets the requirements.     

Performance evaluation of chip breaker utilizing neural network

The continuous chip generated during turning operation deteriorates the workpiece precision and causes safety hazards for the operator. Appropriate control of the chip shape becomes a very important task for maintaining reliable machining process. In particular, effective chip control is necessary for a CNC machine or automatic production system because any failure in chip control can cause the lowering in productivity and the worsening in operation due to frequent stop. Therefore, a grooved chip breaker has been widely used for obtaining reliable discontinuous chips. In general, in order to develop a new cutting insert with a chip breaker, extensive time, research, and expense are required because several processes such as forming, sintering, grinding, and coating of products as well as different evaluation tests are necessary. In this study, the performance of commercial chip breakers was evaluated using a neural network that was trained through a back propagation algorithm. Important form elements (depth of cut, land, breadth, and radius) that directly influenced the chip formation were chosen among commercial chip breakers, and were used as input values of the neural network. As a result, the performance evaluation method has been developed and applied to commercial tools, which resulted in excellent performance. If the training data in the neural network is collected with greater consideration given to the effect of cutting conditions and the performance of chip breakers, it can be used for the design of chip breakers in the future.     

Properties of Ti(C,N) cermets synthesized by mechanically induced self-sustaining reaction

The properties of TiC_xN_1?x/(Ni or Co) cermets sintered by a pressureless method from powder mixtures, and obtained for the first time by a mechanically induced self-sustaining reaction process (MSR), were studied. The hardness, toughness, friction and wear coefficients, and oxidation resistance were determined. It was shown that cermets obtained from powdered materials synthesized in one single MSR step possessed improved mechanical properties, similar to those obtained in cermets with more complex bulk compositions. Higher wear resistances were observed in cermets whose hard phase was richer in carbon. The oxidation resistance of the cermets depended primarily on the binder composition. This resistance was better for those cermets with cobalt as the binder. Superior oxidation resistance was displayed when small amounts of W or Mo were incorporated into the binder.     

A robot motion position and posture control method for freeform surface laser treatment based on NURBS interpolation

Laser surface treatment, such as laser cladding, laser quenching and laser cleaning, for the freeform surface can dramatically enhance the properties of the working surface. During the laser surface treatment, the posture of the treatment tool to the surface influences the size and shape of the laser spot, which further influences the quality of the laser surface treatment. Thus, accurate posture and high synchronization between position and posture are crucial. Industrial robots have the irreplaceable advantages of high flexibility, large workspace and cost-effectiveness over the 5-axis machine tools, therefore, they have become more and more popular in freeform surface laser treatment. However, the research on industrial robot motion mainly concentrates on position control, while less attention has focused on posture control and synchronization of position and posture control. In this paper, a robot motion position and posture (P&P) control method based on the non-uniform rational B-spline (NURBS) interpolation is proposed to realize the high-order continuity and synchronization of P&P as well as the self-adaptive motion control. Firstly, simultaneously considering the position and posture, a novel path generation method based on 6-dimensional NURBS is proposed to provide smooth, continuous and collision-free P&P coordinates of the designed scanning path. Next, a self-adaptive path segmentation method is introduced to recognize the corners and divide the whole treatment path into safe and dangerous segments. Based on the kinematical characteristics of the industrial robot, the joint motion constraint is especially considered in addition to the chord error limit and the centripetal acceleration constraint, and the self-adaptive motion parameters are obtained to match with safe and dangerous segments to avoid mechanical shock and ensure the accessibility of joint servos. Moreover, a synchronized look-ahead method with a dynamically refreshed window is employed to obtain the optimal node speed between adjacent segments and relieve the heavy computational burden, after which a series of smooth, accurate and synchronized motion instructions for the freeform surface laser treatment is generated in real time. Finally, simulations and experiments of a NURBS scanning path for a turbine blade laser surface cladding are conducted. The results show the proposed method can realize the high-order continuity and synchronization of P&P as well as the self-adaptive motion control for the freeform surface laser treatment.