Path smoothing and feed rate planning for robotic curved layer additive manufacturing

Robotic curved layer additive manufacturing (a.k.a. multi-axis 3D printing) has been gaining attention recently owing to its simplicity and unique ability of printing complex shapes without using a support structure. However, as the printing path now is no long planar and the nozzle orientation is no longer fixed but changes continuously during printing, even though it could be smooth when defined in the workpiece coordinate system in both position and orientation of the nozzle, due to the inevitable numerical errors, it typically is unsmooth with many sharp-changing undulations when transformed to the coordinate system of the robot arm. As a result, the feed rate of printing has to be set extremely conservatively lest the printer would chatter or vibrate and seriously jeopardize the printing quality. In this paper, first, we present a practical B-spline based smoothing algorithm for removing sharp corners on the printing path while upholding the required cusp-height threshold on the printed surface. Next, for the smoothed printing path, we propose a feed rate scheduling strategy that will try to maximize the variable feed rate while subject to the kinematic constraints of the six joints of the robot arm. Both computer simulations and physical printing experiments are carried out to assess the proposed methodologies and the results give a positive confirmation on their advantages.     

A hybrid deep learning model of process-build interactions in additive manufacturing

Laser powder bed fusion (LPBF) is a technique of additive manufacturing (AM) that is often used to construct a metal object layer-by-layer. The quality of AM builds depends to a great extent on the minimization of different defects such as porosity and cracks that could occur by process deviation during machine operation. Therefore, there is a need to develop new analytical methods and tools to equip the LPBF process with the inspection frameworks that assess the process condition and monitor the porosity defect in real-time. Advanced sensing is recently integrated with the AM machines to cope with process complexity and improve information visibility. This opportunity lays the foundation for online monitoring and assessment of the in-process build layer. This study presents the hybrid deep neural network structure with two types of input data to monitor the process parameters that result in porosity defect in cylinders’ layers. Results demonstrate that statistical features extracted by wavelet transform and texture analysis along with original powder bed images, assist the model in reaching a robust performance. In order to illustrate the fidelity of the proposed model, the capability of the main pipeline is examined and compared with different machine learning models. Eventually, the proposed framework identified the process conditions with an F-score of 97.14%. This salient flaw detection ability is conducive to repair the defect in real-time and assure the quality of the final part before the completion of the process.     

Predictive exposure control for vision-based robotic disassembly using deep learning and predictive learning

Lighting conditions can affect the performance of vision-based robots in manufacturing. This paper presents a predictive exposure control method that allows the acquisition of high-quality images in real time under poor lighting conditions. This technique is particularly useful in robotic disassembly where a fixed and optimised lighting environment is difficult to construct due to the uncertain conditions of used components, and the optimal exposure conditions for each used component are different. We first develop a region-of-interest (ROI) extraction module capable of identifying ROIs under poor light exposure, in which the states of captured images under various lighting conditions are hypothesised to enhance the extraction ability of a deep learning-based object detector. The extraction results can help a robot obtain an optimal capture position and are incorporated with information about entropy to assess the image quality of ROIs in the proposed ROI quality assessment module. We further design an exposure-entropy prediction model based on predictive learning. This lightweight model is crucial in assisting the exposure time prediction module to achieve real-time searching for the optimal exposure time. The performance of the proposed exposure control method is validated using a screw-removal case study in the application to end-of-life electric vehicle battery disassembly. Together with the ROI extraction module and the ROI quality assessment module, the exposure time prediction module enables the accurate and efficient estimation of optimal exposure time and delivers high-quality images under poor lighting conditions. With our exposure control method, the robot vision system achieves satisfactory performance in the robotic disassembly of electric vehicle batteries.     

Automated defect inspection system for metal surfaces based on deep learning and data augmentation

Recent efforts to create a smart factory have inspired research that analyzes process data collected from Internet of Things (IOT) sensors, to predict product quality in real time. This requires an automatic defect inspection system that quantifies product quality data by detecting and classifying defects in real time. In this study, we propose a vision-based defect inspection system to inspect metal surface defects. In recent years, deep convolutional neural networks (DCNNs) have been used in many manufacturing industries and have demonstrated the excellent performance as a defect classification method. A sufficient amount of training data must be acquired, to ensure high performance using a DCNN. However, owing to the nature of the metal manufacturing industry, it is difficult to obtain enough data because some defects occur rarely. Owing to this imbalanced data problem, the generalization performance of the DCNN-based classification algorithm is lowered. In this study, we propose a new convolutional variational autoencoder (CVAE) and deep CNN-based defect classification algorithm to solve this problem. The CVAE-based data generation technology generates sufficient defect data to train the classification model. A conditional CVAE (CCVAE) is proposed to generate images for each defect type in a single CVAE model. We also propose a classifier based on a DCNN with high generalization performance using data generated from the CCVAE. In order to verify the performance of the proposed method, we performed experiments using defect images obtained from an actual metal production line. The results showed that the proposed method exhibited an excellent performance.     

基于深度学习的无人机巡检图像销钉故障检测

随着经济和社会的发展,发电量和用电量逐年上升。安全的电力保障关系到国计民生,在常年的使用过程中,由于电力传输的输电线路受到外界环境的影响,使得输电线路部件容易出现不同程度的破损,其中销钉是固定螺母的关键零件,销钉的脱落会导致各部件之间连接的不稳定,这给输电网络的安全运行带来了极大的隐患。随着深度学习技术在计算机视觉领域中的应用,使得机器自动识别销钉这一输电线路系统中的微小部件成为现实。采用Faster R-CNN算法对无人机巡检图像中的销钉脱落故障进行识别,并讨论了不同分类器对识别结果的影响,然后对ACF+Adaboost、Hough+LSD和Faster R-CNN检测方法进行比较。实验结果表明,基于Faster R-CNN的目标检测方法对于输电线路中销钉脱落故障的识别率可达到96%,同时对正常销钉的识别率最高可达98%。     

Optimization-based non-equidistant toolpath planning for robotic additive manufacturing with non-underfill orientation

Additive manufacturing (AM) technology has achieved universal application in a great number of fields, such as aerospace, medicine, and military industry. As a significant factor causing weak mechanical properties and part flaws, underfill is inevitable in AM based on the conventional equidistant toolpaths with a limited extra cost. To eliminate underfill caused by sharp corners and voids, this paper develops an optimization-based non-equidistant toolpath, i.e., isoperimetric-quotient-optimal toolpath (IQOP). Firstly, an optimization problem minimizing the isoperimetric quotient of toolpaths is designed to generate smooth toolpaths and is convexified. Then, a unilateral rolling circle method is proposed to guarantee the well-defined condition of the optimization-based toolpath planning process. Finally, the application of the depth tree makes the IQOP method adopt slices with complex boundaries and topological structures. The experimental results show that the proposed IQOP achieves an average 88.5% lower underfill rate than the contour-parallel toolpath (CP). IQOP significantly outperforms the dense CP (DCP) on toolpath smoothness and printing efficiency, with better performance on underfill. With obvious advantages on toolpath smoothness and underfill rate, IQOP achieves higher printing quality than CP in the real world. The proposed approach also provides a general framework of non-equidistant toolpath planning for complex slices in AM and computer numerical control (CNC) milling.     

基于深度学习的安卓恶意应用检测

针对传统安卓恶意程序检测技术检测准确率低,对采用了重打包和代码混淆等技术的安卓恶意程序无法成功识别等问题,设计并实现了DeepDroid算法。首先,提取安卓应用程序的静态特征和动态特征,结合静态特征和动态特征生成应用程序的特征向量;然后,使用深度学习算法中的深度置信网络（DBN）对收集到的训练集进行训练,生成深度学习网络;最后,利用生成的深度学习网络对待测安卓应用程序进行检测。实验结果表明,在使用相同测试集的情况下,DeepDroid算法的正确率比支持向量机（SVM）算法高出3.96个百分点,比朴素贝叶斯（Naive Bayes）算法高出12.16个百分点,比K最邻近（KNN）算法高出13.62个百分点。DeepDroid算法结合了安卓应用程序的静态特征和动态特征,采用了动态检测和静态检测相结合的检测方法,弥补了静态检测代码覆盖率不足和动态检测误报率高的缺点,在特征识别的部分采用DBN算法使得网络训练速度得到保证的同时还有很高的检测正确率。     

Deep learning based online metallic surface defect detection method for wire and arc additive manufacturing

Wire and arc additive manufacturing (WAAM) is an emerging manufacturing technology that is widely used in different manufacturing industries. To achieve fully automated production, WAAM requires a dependable, efficient, and automatic defect detection system. Although machine learning is dominant in the object detection domain, classic algorithms have defect detection difficulty in WAAM due to complex defect types and noisy detection environments. This paper presents a deep learning-based novel automatic defect detection solution, you only look once (YOLO)-attention, based on YOLOv4, which achieves both fast and accurate defect detection for WAAM. YOLO-attention makes improvements on three existing object detection models: the channel-wise attention mechanism, multiple spatial pyramid pooling, and exponential moving average. The evaluation on the WAAM defect dataset shows that our model obtains a 94.5 mean average precision (mAP) with at least 42 frames per second. This method has been applied to additive manufacturing of single-pass, multi-pass deposition and parts. It demonstrates its feasibility in practical industrial applications and has potential as a vision-based methodology that can be implemented in real-time defect detection systems.     

An ontology-oriented knowledge methodology for process planning in additive layer manufacturing

The goal of this research is the generation of a novel knowledge with process-oriented ontology and the informal model. With regard to the proposed ontology, it establishes an improvement to related ontologies because it involves the demands of fabrication engineering and, specifically, the layer-upon-layer manufacturing planning process with various AM systems. Generally, task of AM planning indicates to make repeated and essential decisions which are always on the basis of the engineers’ knowledge and experience in additive manufacturing. Hence, it is a suitable field towards the execution of a knowledge-based engineering system. To represent the knowledge at an upper tier, the IDEF0 diagrams is introduced for identifying the sequence of tasks contained in the AM planning. They are a vital resources for defining the sequence of tasks and the messages flow. Afterward, these messages are analyzed thoroughly by applying schematic graphs, and then they are categorized into knowledge segments. Eventually, each knowledge segment is further divided into knowledge entities. At the same time, the relationships among them are also defined.Meanwhile, knowledge modeling involved generating an ontology of design feature which is utilized as a general information model in both computer-aided design and process planning, expression of fabrication criteria that depict the basis and properties for picking fabrication parameters. In a first method, the ontology has been examined utilizing an essential activity in AM planning: the task related to the confirmation of parameters over component quality. In this task, decisions have to be made in the orientation, slicing and the other process parameters. In this research, the differences between geometric and dimensional tolerance fabrication is considered in the generated methodology. The knowledge needed to aid all decisions is expressed utilizing the proposed ontology.     

Investigation of humping phenomenon for the multi-directional robotic wire and arc additive manufacturing

Wire Arc Additive Manufacturing (WAAM) is a promising technology for fabricating medium to large scale metallic parts with excellent productivity and flexibility. Due to the positional capability of some welding processes, WAAM is able to deposit parts with overhanging features in an arbitrary direction without additional support structures. The dimensional quality of the overhanging parts may however deteriorate due to the humping effect, which appears as a series of periodic beadlike protuberances on the weld deposits. There has been significant research on the humping phenomenon in the downhand welding, but it is doubtful whether the existing theories of humping formation can be applied in the positional deposition during WAAM process. This study has therefore provided an experimental work to investigate the formation of the humping phenomena in the positional deposition during additive manufacturing with the gas metal arc welding. Firstly, the mechanism of humping formation was analysed to explain humping occurrence for positional deposition. Then, the mechanism was validated through experiments with different welding parameters and positions. Finally, a series of guidelines are summarised to assist the path planning and process parameter selection processes in multi-directional WAAM.     

Automated bank cheque verification using image processing and deep learning methods

Multimedia Tools and Applications - Automated bank cheque verification using image processing is an attempt to complement the present cheque truncation system, as well as to provide an alternate...     

Hybrid path planning method based on skeleton contour partitioning for robotic additive manufacturing

Establishing manufacturability design criteria for multidimensional complex parts can significantly reduce the production cost, shorten the manufacturing cycle, and improve the production quality of directed energy deposition. Therefore, there is an urgent need to establish a high-performance manufacturing design strategy for complex parts. Proposed here is a skeleton contour partitioning hybrid path-planning method that takes full advantage of the excellent geometric reducibility of the contour offset method and the outstanding flexibility of the zigzag path method, eliminating the influences of sharp corners and degradation on forming quality in the contour offset method. First, reference contours are obtained by subjecting the original contours to an inward–outward twice-offset process; incompletely filled regions are obtained by Boolean operations on the original and reference contours, and these regions are the ones to be optimized. Second, the optimized regions are merged into skeleton fill regions, and the fill paths are generated by a polygon trapezoidal partitioning recombination algorithm and an algorithm for generating optimal zigzag paths. Finally, the contour offset paths are split and regrouped based on the skeleton regions and are connected into a continuous forming path for each subregion, then all the forming paths are converted into robot printing tool paths from the skeleton-region filling paths to the contour-offset ones. The actual forming results for several parts with different geometric features are verified and compared with those of the traditional path-planning method, and it is concluded that the proposed method converges rapidly to the details of complex components and is highly feasible and applicable.     

糖尿病性视网膜图像的深度学习分类方法

目的 糖尿病性视网膜病变（DR）是目前比较严重的一种致盲眼病,因此,对糖尿病性视网膜病理图像的自动分类具有重要的临床应用价值。基于人工分类视网膜图像的方法存在判别性特征提取困难、分类性能差、耗时费力且很难得到客观统一的医疗诊断等问题,为此,提出一种基于卷积神经网络和分类器的视网膜病理图像自动分类系统。方法 首先,结合现有的视网膜图像的特点,对图像进行去噪、数据扩增、归一化等预处理操作;其次,在AlexNet网络的基础上,在网络的每一个卷积层和全连接层前引入一个批归一化层,得到一个网络层次更复杂的深度卷积神经网络BNnet。BNnet网络用于视网膜图像的特征提取网络,对其训练时采用迁移学习的策略利用ILSVRC2012数据集对BNnet网络进行预训练,再将训练得到的模型迁移到视网膜图像上再学习,提取用于视网膜分类的深度特征;最后,将提取的特征输入一个由全连接层组成的深度分类器将视网膜图像分为正常的视网膜图像、轻微病变的视网膜图像、中度病变的视网膜图像等5类。结果 实验结果表明,本文方法的分类准确率可达0.93,优于传统的直接训练方法,且具有较好的鲁棒性和泛化性。结论 本文提出的视网膜病理图像分类框架有效地避免了人工特征提取和图像分类的局限性,同时也解决了样本数据不足而导致的过拟合问题。     

Image steganography using deep learning based edge detection

This paper introduces a deep learning-based Steganography method for hiding secret information within the cover image. For this, we use a convolutional neural network (CNN) with Deep Supervision based edge detector, which can retain more edge pixels over conventional edge detection algorithms. Initially, the cover image is pre-processed by masking the last 5-bits of each pixel. The said edge detector model is then applied to obtain a gray-scale edge map. To get the prominent edge information, the gray-scale edge map is converted into a binary version using both global and adaptive binarization schemes. The purpose of using different binarization techniques is to prove the less sensitive nature of the edge detection method to the thresholding approaches. Our rule for embedding secret bits within the cover image is as follows: more bits into the edge pixels while fewer bits into the non-edge pixels. Experimental outcomes on various standard images confirm that compared to state-of-the-art methods, the proposed method achieves a higher payload.

     

小样本深度学习方法实现LED TV屏缺陷检测

为实现当前工业4.0时代电子类企业智能制造的全过程,引入机器视觉完成产品缺陷检测,用于解决缺陷问题多样性在算法能力的不足。首先对已标注小样本数据集通过深度学习得到初始特征模型,接着针对该特征模型施以迁移学习方法用以实现LED TV的检测,并将已检测样本进一步用于增量学习完成模型参数的修正,最后采用全连接神经网络FCNet (Fully Connected Neural Network)完成分类,探讨了一种运用机器视觉实现LED TV的光学屏检技术;并给出了检测样品作为补充的样本数据集增量学习模型。实践表明,本文提出的方法能进一步提升工业机器人智能制造阶段自动化检测的准确率,最终实现工业生产的柔性和智能化水平,并为机器视觉的应用提供示范。     

A multi-objective iterative learning control approach for additive manufacturing applications

Iterative learning control (ILC) is a method for improving the performance of stable, repetitive systems. Standard ILC is constructed in the temporal domain, with performance improvements achieved through iterative updates to the control signal. Recent ILC research focuses on reformulating temporal ILC into the spatial domain, where 2D convolution accounts for spatial closeness. This work expands spatial ILC to include optimization of multiple performance metrics. Performance objectives are classified into primary, complementary, competing, and domain specific objectives. New robustness and convergence criteria are provided. Simulation results validate flexibility of the spatial framework on a high-fidelity additive manufacturing system model.     

Autonomous robotic inspection and manipulation using multisensorfeedback

A six-degree-of-freedom industrial robot to which was added a number of sensors-vision, range, sound, proximity, force/torque, and touch-to enhance its inspection and manipulation capabilities is described. The work falls under the scope of partial autonomy. In teleoperation mode, the human operator prepares the robotic system to perform the desired task. Using its sensory cues, the system maps the workspace and performs its operations in a fully autonomous mode. Finally, the system reports back to the human operator on the success or failure of the task and resumes its teleoperation mode. The feasibility of realistic autonomous robotic inspection and manipulation tasks using multisensory information cues is demonstrated. The focus is on the estimation of the three-dimensional position and orientation of the task panel and the use of other nonvision sensors for valve manipulation. The experiment illustrates the need for multisensory information to accomplish complex, autonomous robotic inspection and manipulation tasks     

Fault detection based on one-class deep learning for manufacturing applications limited to an imbalanced database

Despite extensive studies for the industrial applications of deep learning, its actual usage in manufacturing sites has been extremely restrained by the difficulty in obtaining sufficient industrial data, especially for production failure cases. In this study, we introduced a fault-detection module based on one-class deep learning for imbalanced industrial time-series data, which consists of three submodules, namely, time-series prediction based on deep learning, residual calculation, and one-class classification using one-class support vector machine and isolation forest. Four different networks were used for the time-series prediction: multilayer perception (MLP), residual network (ResNet), long–short-term memory (LSTM), and ResNet–LSTM, each trained with the one-class data having only the production success cases. We adopted the residuals of the deep-learning prediction as an elaborated feature for the construction of the one-class classification. We also tested the fault-detection module with the actual mass production data of a die-casting process. By adopting the features elaborated by the deep-learning time-series prediction, we showed that the total accuracy of the one-class classification significantly improved from 90.0% to 96.0%. Especially for its capability to detect production failures, the accuracy improved from 84.0% to 96.0%. The area under the receiver operating characteristics (AUROC) also improved from 87.56% to 98.96%. ResNet showed the best performance for detecting production failures, whereas ResNet–LSTM produced better results for ensuring the production success. Our results suggest that the one-class deep learning is a promising approach for extracting important features from time-series data to realize a one-class fault-detection module.     

AR-based deep learning for real-time inspection of cable brackets in aircraft

In the process of aircraft assembly, there exist numerous and ubiquitous cable brackets that shall be installed on frames and subsequently need to be manually verified with CAD models. Such a task is usually performed by special operators, hence is time-consuming, labor-intensive, and error-prone. In order to save the inspection time and increase the reliability of results, many researchers attempt to develop intelligent inspection systems using robotic, AR, or AI technologies. However, there is no comprehensive method to achieve enough portability, intelligence, efficiency, and accuracy while providing intuitive task assistance for inspectors in real time. In this paper, a combined AR+AI system is introduced to assist brackets inspection in a more intelligent yet efficient manner. Especially, AR-based Mask R-CNN is proposed by skillfully integrating markerless AR into deep learning-based instance segmentation to generate more accurate and fewer region proposals, and thus alleviates the computation load of the deep learning program. Based on this, brackets segmentation can be performed robustly and efficiently on mobile devices such as smartphones or tablets. By using the proposed system, CAD model checking can be automatically performed between the segmented physical brackets and the corresponding virtual brackets rendered by AR in real time. Furthermore, the inspection results can be directly projected on the corresponding physical brackets for the convenience of maintenance. To verify the feasibility of the proposed method, experiments are carried out on a full-scale mock-up of C919 aircraft main landing gear cabin. The experimental results indicate that the inspection accuracy is up to 97.1%. Finally, the system has been deployed in the real C919 aircraft final-assembly workshop. The preliminary evaluation reveals that the proposed real-time AR-assisted intelligent inspection approach is effective and promising for large-scale industrial applications.