•Review the state-of-the-art of the development of intelligent welding systems (IWS), its concepts, and its system architectures.•Comprehensively review the fundamental technologies of IWS, including ...sensing, feature selection, modeling, decision-making, and learning.•Review the applications of IWS, including monitoring, control, and quality assessment.•Review the emerging platforms and their potential application to IWS.•Suggest areas for future research.
Welding systems are being transformed by the advent of modern information technologies such as the internet of things, big data, artificial intelligence, cloud computing, and intelligent manufacturing. Intelligent welding systems (IWS), making use of these technologies, are drawing attention from academic and industrial communities. Intelligent welding is the use of computers to mimic, strengthen, and/or replace human operators in sensing, learning, decision-making, monitoring and control, etc. This is accomplished by integrating the advantages of humans and physical systems into intelligent cyber systems. While intelligent welding has found pilot applications in industry, a systematic analysis of its components, applications, and future directions will help provide a unified definition of intelligent welding systems. This paper examines fundamental components and techniques necessary to make welding systems intelligent, including sensing and signal processing, feature extraction and selection, modeling, decision-making, and learning. Emerging technologies and their application potential to IWS will also be surveyed, including Industry 4.0, cyber-physical system (CPS), digital twins, etc. Typical applications in IWS will be surveyed, including weld design, task sequencing, robot path planning, robot programming, process monitoring and diagnosis, prediction, process control, quality inspection and assessment, human-robot collaboration, and virtual welding. Finally, conclusions and suggestions for future development will be proposed. This review is intended to provide a reference of the state-of-the-art for those seeking to introduce intelligent welding capabilities as they modernize their traditional welding stations, systems, and factories.
Display omitted
•The procedures, methods and principles of vision-aided robotic welding are summarized and analyzed.•Various groove types and joint types are detected by active and passive vision ...sensing methods.•Various algorithms and control methods are compared and discussed for seam tracking.
Vision-aided robotic welding has been applied in the industrial field for decades. This paper summarizes the procedures of vision-aided robotic welding. Various methods in vision sensor calibration and hand-eye calibration have been illustrated. The recognition, calculation and guidance control are the basic stages of visual positioning for SWP (start welding point). Various groove types (I-groove, V-groove, Y-groove, U-groove etc.) and joint types (butt joint, lap joint, fillet joint, T-joint etc.) can be detected by six active vision sensing methods and three passive vision sensing methods. Weld path detection, tracking algorithm and control strategy are the necessary procedures to realize seam tracking. Various seam trajectories (straight line, zig-zag, sine, half-moon, pipe curve, spatial curve etc.) can be compensated by several common control methods (PID control, fuzzy control, iterative learning control, trajectory-based control etc.). The selection of control method is determined by weld path detection and seam tracking algorithm. In the end, the future development of vision-aided robotic welding has also been presented.
Use of sensors in robotic welding for controlling the weld quality leads to replacement of manual welding operation in dangerous work environment in presence of high temperature and fumes even in ...small or medium scale enterprises. The seam tracking operation is very essential for extracting weld seam position which can be fed to robot controller for instructing robot along the weld seam path. The seam tracking operation can be executed by different types of sensors having their own merits and demerits. In this paper, different sensors and techniques used for seam tracking task in robotic welding have been discussed in detail. Each sensor has different method or technology of weld seam feature extraction which have been described by different authors in different ways. The chief tasks for seam tracking have been found to be weld starting and end point detection, weld edge detection, joint width measurement and weld path position determination with respect to robot co-ordinate frame. Thus sensors have a very important role in robotic welding for fully automating the system with in process real time monitoring of weld process parameters with the sensor feedback. In further discussion the practical use of different sensors in industries with a comparison of their advantages and disadvantages have been discussed. This Paper presents the role of sensors in robotic welding and a detail study of methodologies of weld seam position and geometry feature extraction by different sensors typically used for weld seam tracking.
Automatic joint detection is of vital importance for the teaching of robots before welding and the seam tracking during welding. For narrow butt joints, the traditional structured light method may be ...ineffective, and many existing detection methods designed for narrow butt joints can only detect their 2D position. However, for butt joints with narrow gaps and 3D trajectories, their 3D position and orientation of the workpiece surface are required. In this paper, a vision based detection method for narrow butt joints is proposed. A crosshair laser is projected onto the workpiece surface and an auxiliary light source is used to illuminate the workpiece surface continuously. Then, images with an appropriate grayscale distribution are grabbed with the auto exposure function of the camera. The 3D position of the joint and the normal vector of the workpiece surface are calculated by the combination of the 2D and 3D information in the images. In addition, the detection method is applied in a robotic seam tracking system for GTAW (gas tungsten arc welding). Different filtering methods are used to smooth the detection results, and compared with the moving average method, the Kalman filter can reduce the dithering of the robot and improve the tracking accuracy significantly.
Calibration is a crucial issue in laser vision robotic welding systems (LVRWS). Traditional calibration methods rely on complex manual teaching and fixed pattern of robot end-effector poses, which ...are time-consuming and will occasionally fall into local optimal solution. To address these issues, a fast calibration method based on automatic path planning is proposed in this paper. By ensuring that the camera's optical axis coincides with the center of the calibration board regardless of the robot end-effector's attitude, images covering all feature points will be collected. Based on the camera optical axis constraint, we establish the robot end-effector motion constraint equation. Then, an automatic path planning method based on optical axis constraint equation, rotation constraint, and motion selection strategy is proposed, which improves the diversity of the calibration data. During data acquisition, a hand-eye parameters optimization process is performed simultaneously by minimizing the reprojection error of visual feature points. The actual calibration tests show that the proposed method can significantly improve the calibration efficiency and robustness. In addition, a variety of weld measurement experiments, such as S-type and C-type welds and fillet weld, were conducted under arbitrary welding torch's attitude to evaluate the accuracy and robustness of the method. The mean absolute error (MAE) and root mean square error (RMSE) of the position are within 0.2mm.
As a traditional fusion welding method, arc welding occupies most of the total welding production. However, the current industrial welding robots with the “teaching and playback” mode cannot satisfy ...the requirements of modern welding manufacturing. To overcome this major challenge, advanced sensing technology can be used to efficiently imitate and reproduce the welder's senses and brain. Many recent studies on application of sensing technology have promoted the development of robotic arc welding toward intelligent welding. In this paper, from the perspective of intelligent welding systems (IWS), the application of advanced sensing technology in the pre-process, in-process, and post-process stages of intelligent robotic arc welding is summarized and discussed. First, the development and application of various sensing technologies and multisensor fusion technologies for intelligent arc welding are reviewed and discussed. Subsequently, according to the different objectives of each welding stage, the advanced sensing technologies, including those for weld path recognition, weld seam tracking, weld pool monitoring, weld quality diagnosis, and weld bead inspection, are summarized and compared. Finally, a summary is provided and future prospects are put forward. This paper reviews the research progress of sensing technology for different monitoring objectives of intelligent robotic arc welding, and to provide a basis for future work.
Display omitted
•A Faster R-CNN model is trained to divide typical welding seams into continuous and discontinuous types in accordance with laser stripe images.•The welding guidance is realized ...through point cloud processing, test results proved that the accuracy can meet actual production demand.•An adaptive feature extraction algorithm based on laser vision sensor is studied, which is suitable for multiple welding seams tracking.•Tracking tests prove that the average errors of continuous seam and discontinuous seam are respectively 0.29 mm and 0.28 mm, which can maintain satisfying robustness even under complex working conditions.
Intelligent robotic welding is an indispensable part of modern welding manufacturing, and vision-based seam tracking is one of the key technologies to realize intelligent welding. However, the adaptability and robustness of most image processing algorithms are deficient during welding practice. To address this problem, an adaptive feature extraction algorithm based on laser vision sensor is proposed. According to laser stripe images, typical welding seams are classified into continuous and discontinuous welding seams. A Faster R-CNN model is trained to identify welding seam type and locate laser stripe ROI automatically. Before welding, initial welding point is determined through point cloud processing to realize welding guidance. During seam tracking process, the seam edges are achieved by a two-step extraction algorithm, and the laser stripe is detected by Steger algorithm. Based on the characteristics of two kinds of welding seams, the corresponding seam center extraction algorithms are designed. And a prior model is proposed to ensure the stability of the algorithms. Test results prove that the algorithm has good adaptability for multiple typical welding seams and can maintain satisfying robustness and precision even under complex working conditions.
•Current automatic programming approaches for robotic manufacturing systems have been reviewed.•A hybrid offline programming method that combines CAD-based and vision-based offline approaches is ...proposed.•The vision-based and CAD-based activities of the proposed programming method with the support techniques are presented.•Vision & CAD interactive activities are proposed to identify the workplace, detect the deviation, and compensated the path.
Offline programming is an intuitive and automatic programming generation technique that does not use real robotic systems, thus greatly decreasing the downtime required for system programming, and resulting in enormous savings in terms of labor costs. Currently, offline programming can be generally categorized into computer-aided-design-based (CAD-based) and vision-based approaches; these two types of offline programming approaches have been widely applied in robotic welding systems. However, owing to the highly complex and diverse workpieces needed in the shipbuilding industry, neither of the aforementioned offline programming approaches can fully support the automatic generation of welding programs.
In this paper, a hybrid offline programming method systematically combining CAD-based, vision-based, and vision & CAD interactive activities is proposed to overcome the limitations of current automatic program generation methods for robotic welding systems. In the vision-based activities, the positions of the workpieces are obtained by using geometrical features gathered from the workpieces’ images, whereas in the CAD-based activities, the welding tasks are assigned to different mobile components of the welding torch; then, their welding paths are planned according to the workpieces’ CAD models. The vision & CAD interactive activities enable the mapping between the point cloud of a workpiece and its CAD model, so that the deviations caused by assembly errors can be detected and path compensation data can be determined. The effectiveness of the proposed hybrid offline programming method is demonstrated by integrating it into a subassembly welding robotic system. The experimental results indicate that this method can significantly improve the efficiency, accuracy, and flexibility of the robotic welding system.
PDF
