Real-time penetration state monitoring using convolutional neural network for laser welding of tailor rolled blanks

In this paper, an innovative monitoring system capable of diagnosing the penetration state during the laser welding process is introduced, which consists of two main blocks: a coaxial visual monitoring platform and a penetration state diagnosis unit. The platform can capture coaxial images of the interaction zone during the laser welding through a partially transmitting mirror and a high-speed camera. An image dataset representing four welding states was created for training and validation. The unit mainly consists of an embedded power-efficient computing TX2 and image processing algorithms based on a convolution neural network (CNN). Experiment results show that the platform can stably capture state-of-the-art welding images. The CNN used for a diagnosis of the penetration state is optimized using an optimal network structure and hyperparameters, applying a super-Gaussian function to initialize the weights of the convolutional layer. Its latency on TX2 is less than 2 ms, satisfying the real-time requirement. During the real laser welding of tailor-rolled blanks, a penetration state diagnosis with an accuracy of 94.6 % can be achieved even if the illumination changes significantly. The similar accuracy between the validating set and a real laser welding demonstrates that the proposed monitoring system has strong robustness. The precision and recall ratios of the CNN are higher than those of other methods such as a histogram of oriented gradients and local binary pattern.