Coal mine belt conveyor sorting robot foreign object identification and positioning system As an important equipment for coal mine production, the safe operation of coal mine belt conveyor is an important foundation to ensure the normal production of coal mine. However, in the process of coal production and transportation, coal mine conveyor belts can be affected by foreign objects mined out by comprehensive mining or comprehensive excavation, etc., which can lead to serious accidents such as torn belt and broken belt. Traditional foreign object detection methods such as manual detection, radar detection and metal detectors are inefficient, costly, difficult to deploy and maintain, and have safety hazards. With the continuous development of machine vision technology, domestic and foreign institutions and scholars have conducted a lot of research on the application of machine vision technology in coal mine belt conveyor condition monitoring and target detection. Although machine vision has a certain theoretical basis in coal mine belt conveyor target detection and identification, the current coal mine belt conveyor sorting robot target identification is mainly for gangue identification, and there is less research on foreign object target identification that causes belt penetration, tearing, etc., and there is less research on precise positioning of target foreign objects.   Hereby, MingDe designed a coal mine belt conveyor sorting robot foreign object identification and positioning system, which can identify and locate different types and shapes of foreign objects on the conveyor belt.   Through a multi angle, multi-dimensional stereo high-precision camera, the intelligent robot sorter quickly scans the ore on the conveyor belt. The self-developed CRM-CNN foreign object recognition algorithm accurately locates the 3D position of the debris, controls the robot to grab  the foreign object, and puts it into the foreign object collection box. Product characteristics   1: Based on deep learning technology ,combined with a large database of ore foreign objects, The intelligent robot sorter has a high foreign object recognition rate. 2: Using multi-angle and multi-dimensional industrial stereo camera and geometric 3D recognition algorithm, the intelligent robot sorting machine can accurately measure and position the depth, direction and position of foreign objects. 3: Highly flexible control, for the newly emerged foreign objects can be added at any time. 4: Specially developed high protection levels robot arm, faster and more flexible, can effectively adapt to various conveying speeds and harsh industrial environments 5: Highly intelligent, unattended and optional remote monitoring   Experimental Verification  The reliability of the coal mine belt conveyor sorting robot foreign object recognition and localization system and its algorithm is verified by the experimental prototype with rod-shaped foreign objects as the experimental object. The experimental results of the system prototype show that the foreign object recognition rate of the coal mine belt conveyor sorting robot foreign object recognition and positioning system is not affected by the size, material and color, etc., and it can realize the acquisition, processing, feature extraction, recognition and position positioning of the target foreign object image of the conveyor belt, and the recognition rate is more than 99.5%, and the average error of target foreign object position positioning is about 1%.  

MingDe Material removal robot robotic manipulator  Robot Arm Foreign Object Removal Robot Material removal robot   A robot is an intelligent machine that can work semi-autonomously or fully autonomously. Robots can be programmed and automatically controlled to perform tasks such as working or moving. Product Principle Through a multi angle, multi-dimensional stereo high-precision camera, the intelligent robot sorter quickly scans the ore on the conveyor belt. The self-developed CRM-CNN foreign object recognition algorithm accurately locates the 3D position of the debris, controls the robot to grab  the foreign object, and puts it into the foreign object collection box. Mingde Product characteristics 1: Based on deep learning technology ,combined with a large database of ore foreign objects, The intelligent robot sorter has a high foreign object recognition rate. 2: Using multi-angle and multi-dimensional industrial stereo camera and geometric 3D recognition algorithm, the intelligent robot sorting machine can accurately measure and position the depth, direction and position of foreign objects. 3: Highly flexible control, for the newly emerged foreign objects can be added at any time. 4: Specially developed high protection levels robot arm, faster and more flexible, can effectively adapt to various conveying speeds and harsh industrial environments   5: Highly intelligent, unattended and optional remote monitoring   Application areas It is mainly used for ore sorting, sorting of anchor rods, steel brazier, rags, wood, iron parts, waste filling pipeline and other debris in the process of ore production and transportation, replacing manual hand sorting, laying the foundation for reducing personnel labor, lowering equipment failure rate, reducing staff and increasing efficiency.        

Design of a 3D vision-guided robot depalletizing system for multi-gauge materials Material Removal Robot Robot Arm Robotic Manipulator Abstract: In industrial manufacturing and logistics, material depalletization by robots is one of the common applications. Material depalletization is a scenario where goods of different gauges (i.e., goods of different sizes, weights, or textures) are loaded on pallets for delivery. Earlier robot depalletization was only applicable to the unloading of single goods and required the goods to be arranged in a fixed order, and the robot did not have perception capability; the vision-guided robot depalletization system described in this paper is equipped with real-time environment perception capability to guide the grasping action, thus solving the problems of variable sizes of objects to be unloaded and irregular placement of multi-gauge material depalletization systems.   Keywords: 3D vision recognition, robot, hybrid palletizing, object positioning, depalletizing algorithm   In industrial manufacturing and logistics, various industrial robots can be used to optimize the flow of goods, and one of the common applications is the depalletization of materials. "Robotic depalletization" usually refers to the process of sequential unloading of materials from pallets using robotic arms and can be used to replace simple but heavy manual labor. In logistics, there are scenarios where goods of different gauges (i.e., different sizes, weights, or textures) are delivered in boxes, as shown in Figure 1. However, early robotic depalletizing systems were mainly controlled manually to complete robot gripping, which was only applicable to the unloading of a single cargo and required the cargo to be arranged in a fixed order, and the robot did not have the perception capability to react to external changes. However, multi-gauge material depalletization systems require robots to have real-time environmental awareness to guide the gripping action because the objects to be unloaded are variable in size and irregularly placed. With the development of various optical sensors, computer vision technology has been gradually introduced into robot grasping tasks to improve the robot's ability to acquire external information. A vision-guided robot depalletizing system usually contains five modules, which are vision information acquisition module, object localization and analysis module, grasping position calculation module, hand-eye coordinate conversion module, and motion planning module, as shown in Figure 2. Among them, the first three modules are the main part of the vision system, responsible for acquiring and processing visual information and providing object poses. The last two modules are mainly used to provide control information to the robot and complete the grasping function. In the following, we will introduce each module, common methods and implementation cases. I. Vision information acquisition module The role of the vision information acquisition module is to capture visual information and provide input for subsequent steps. At present, the commonly used visual inputs include 2D RGB images, 3D point cloud images and combined 2D and 3D RGB-D images. Among them, vision-assisted robotic arm grasping based on 2D RGB images is currently a mature solution in industry, which transforms the robot grasping problem into the problem of doing object target detection or image segmentation on RGB images. However, 2D vision lacks absolute scale information of objects and can only be used under specific conditions, such as scenarios with fixed pallets and known material sizes. For scenarios where the material gauge is unknown, the vision module is required to provide the robot with accurate absolute size information of the object to be grasped, so only 3D point cloud images or RGB-D images with a combination of 2D and 3D can be used. Compared with RGB information, RGB-D information contains spatial distance information from the camera to the object; compared with 3D point cloud images, RGB-D information contains rich color texture information. Therefore, RGB-D images can be used as the visual information input of the multi-gauge material depalletizing system. Object positioning and analysis module The object positioning and analysis module receives the data input from the vision information acquisition module, analyzes the materials present in the scene, and obtains key information such as their position and pose, and then inputs this key information into the grasping pose calculation module. Generally speaking, the material localization problem in robotic depalletizing system can be transformed into a target detection or image segmentation problem in the vision field. The RGB-D vision-based robot grasping solution can first perform 2D target detection or 2D image segmentation on the RGB image for the material, and then fuse the depth map to output the absolute size of the object and the grasping pose; or directly do target detection or segmentation on the 3D point cloud map. The following will be a brief introduction to the related work. 1.2D target detection The input of 2D target detection is the RGB image of the scene, and the output is the class and position of the object in the image, and the position is given in the form of border or center. The methods for target detection can be divided into traditional methods and deep learning based methods. Traditional target detection methods generally use a sliding window to traverse the entire image, with each window becoming a candidate region. For each candidate region, features are first extracted using SIFT, HOG and other methods, and then a classifier is trained to classify the extracted features. For example, the classical DPM algorithm uses SVM to classify the modified HOG features to achieve the effect of target detection. The traditional method has two obvious drawbacks: firstly, it is very time-consuming to traverse the whole image with a sliding window, making the algorithm's time complexity high and difficult to apply to large-scale or real-time scenarios; secondly, the features used often need to be designed manually, making such algorithms more experience-dependent and less robust. 2. Two-dimensional image segmentation Image segmentation can be regarded as a pixel-level image classification task. Depending on the meaning of the segmentation result, image segmentation can be divided into semantic segmentation and instance segmentation. Semantic segmentation classifies each pixel in an image into a corresponding category, while instance segmentation not only performs pixel-level classification, but also differentiates different instances on the basis of specific categories. Relative to the bounding box of target detection, instance segmentation can be accurate to the edges of objects; relative to semantic segmentation, instance segmentation needs to label different individuals of similar objects on the graph. In depalletizing applications, we need to extract the edges of materials precisely to calculate the grasping position, so we need to use instance segmentation techniques. The existing image segmentation techniques can be divided into traditional methods and deep learning based methods.   Most of the traditional image segmentation methods are based on the similarity or mutation of gray values in an image to determine whether pixels belong to the same class. The commonly used methods include graph theory-based methods, clustering-based methods and edge detection-based methods.   Deep learning-based methods have substantially improved the accuracy of 2D image segmentation compared to traditional methods. Typical deep neural network frameworks, such as AlexNet, VGGNet, GoogleNet, etc., add a fully connected layer at the end of the network for feature integration, followed by softmax to determine the category of the whole image. To solve the image segmentation problem, the FCN framework replaces these fully-connected layers with deconvolution layers, making the output of the network from a one-dimensional probability into a matrix with the same resolution as the input, which is the pioneering work of applying deep learning to semantic segmentation. 3. 3D target detection 3D target detection enables robots to accurately predict and plan their behavior and paths by directly computing the 3D position of objects to avoid collisions and violations. 3D target detection is divided into monocular camera, binocular camera, multiocular camera, line surface LIDAR scan, depth camera and infrared camera target detection according to the type of sensor. In general, stereo/multi-vision systems consisting of multi-vision cameras or LiDAR enable more accurate 3D point cloud measurements, where multi-view-based methods can use parallax from images of different views to obtain depth maps; point cloud-based methods obtain target information from point clouds. In comparison, since the depth data of points can be measured directly, the point cloud-based 3D target detection is essentially a 3D point delineation problem and is therefore more intuitive and accurate.   Third, the capture pose calculation module The gripping posture calculation module uses the position posture information of the target object output from the second module to calculate the gripping posture of the robot. Since there are often multiple graspable targets in a multi-gauge material depalletizing system, this module should solve the two problems of "which one to grasp" and "how to grasp". The first step is to solve the "which" problem. The goal of this problem is to select the best crawl target among many crawl targets, and the "best" here often needs to be defined by the actual requirements. Specifically, we can quantify some indicators that have an impact on the crawling judgment according to the actual situation, and then prioritize these indicators. The second step is to solve the problem of "how to catch". We can choose to analyze and calculate the grasping posture by mechanical analysis, or we can first classify the object by learning method, and then select the grasping point according to the classification, or directly regress the grasping posture.   Fourth, the hand-eye coordinate conversion module With the third module, we have obtained a feasible gripping pose. However, the gripping pose is based on the pose in the camera coordinate system, and the gripping pose needs to be converted to the robot coordinate system before motion planning can be performed. In depalletizing systems, hand-eye calibration is usually used to solve this problem. Depending on the camera fixation position, the hand-eye calibration method can be divided into two cases. One is that the camera is fixed on the robot arm and the camera moves together with the arm, called Eye-in-hand, as shown in Figure 3. In this relationship, the position relationship between the robot base and the calibration plate remains constant during the two movements of the robot arm, and the solved quantity is the position relationship between the camera and the robot end coordinate system. The other type of camera is fixed on a separate stand, called Eye-to-hand, as shown in Figure 4. In this case, the attitude relationship between the end of the robot and the calibration plate remains the same during the two movements of the arm, and the solution is the attitude relationship between the camera and the coordinate system of the robot base. Both cases are eventually transformed into a solution problem with AX=XB, and the equation can be transformed into a linear equation using Lie group and Lie algebra to solve for the rotation and translation quantities, respectively. Fifth. Motion planning module This module mainly considers the kinematics, dynamics, mechanical analysis, and motion planning of the robot to plan a feasible motion path that does not collide with the environment. By multiplying the grasping pose in the camera coordinate system obtained by the grasping pose calculation module with the conversion matrix calibrated by the hand-eye coordinate conversion module, we can get the grasping pose in the robot arm coordinate system. Based on this posture, the motion planning can be carried out and the robot arm can be guided to complete the depalletizing task. Therefore, the input of the motion planning module is the starting and target positions of the robot arm, and the output is the motion path of the robot arm.   The complete motion planning algorithm can be split into the following three steps. Step 1: Inverse kinematic solving. In order to avoid problems such as singularities, robotic arm motion planning is generally performed under joint space. Therefore, we should first perform the inverse kinematic solution based on the input poses to obtain the joint values corresponding to the poses. Step 2: Path planning. With the path planning algorithm, we can get the motion path of the robotic arm. The goal of this step is twofold: one is obstacle avoidance, to ensure that the robotic arm does not collide with other objects in the scene during its movement; the second is to improve the operation speed in order to increase the operation efficiency of the system. By planning a reasonable motion path, the running time of a single grasp of the robotic arm can be made shorter, thus improving efficiency. Step 3: Time interpolation. Although we can already get a feasible motion path through path planning, however, this path is composed of one location point after another. When the robot arm is running along this path, it needs to keep acceleration and deceleration, so it will have an impact on the running speed. For this reason, we need to perform temporal interpolation to obtain the velocity, acceleration, and time information for each point on the path as the robot arm moves to that point. In this way, the robot arm can run continuously and smoothly, thus improving efficiency.   Sixth. Implementation Example Based on the above research, a complete vision system consisting of 3D depth camera, lighting system, computer, and vision processing software can be used in the piece box material identification scenario to obtain some special information about real objects, and the information obtained through this system can be used to accomplish some special tasks, such as obtaining the box position through the vision system, which can guide the robot to grasp and obtain the box quantity information as a calibration for the task. The main components of this system, as shown in Figure 5. The 3D camera and light system are mainly used for photo imaging, where the 3D camera can obtain depth data within a certain range. And the digital image imaging is related to the illumination system. The computer, on the other hand, includes general-purpose computing and storage devices for saving images, processing images through specialized vision software, and also for network communication with other systems. The image display facilitates the operator to operate the vision processing software and monitor the system operation. Large-capacity storage is used for permanent or temporary storage of images or other data. Specialized vision software, on the other hand, includes digital image processing, image data analysis, and some special functions.   Generally speaking, a 3D depth camera has a frame rate of 1 to 30 fps, RGB image resolution of 640×480, 1280×960, special 1920×1080, 2592×1944, and a depth range of about 500mm to about 5000mm.   And depending on the price, there are different precision and range. Here is an example of a brand of 3D camera with parameters as shown in Figure 6 and accuracy as shown in Figure 7. With the 3D camera, you can get RGB images and depth images of special scenes, and according to the processing and analysis of these images (see Figure 8), you can get some information about the position, number, and information of objects in the scene. The rectangular box in Figure 9 is the box grabbing position map identified after processing. The order of upper left, lower left, upper right and lower right is "2, 3, 3, 2" respectively, that is, the robot hand will grasp two boxes on the left, three boxes on the left, three boxes on the right and two boxes on the right according to the position information given by the image recognition system. Seventh. Summary In this paper, we have introduced the framework and common methods of 3D vision-guided multi-gauge material robot depalletizing system, and defined several basic modules that the framework needs to have, namely, vision information acquisition module, object localization and analysis module, grasping position calculation module, hand-eye coordinate conversion module, and motion planning module, and explained the main tasks and common methods of each module. In practical applications, different methods can be used to implement these modules as needed without affecting the functions of other modules and the system as a whole.    

Robotics and automation: New robots improve material handling efficiency The advent of robotics means that warehouse workers can spend less time on material handling and more time digging in and serving customers. It also creates opportunities for workers to manage and "train" new equipment. For years, warehouses have been quietly incorporating robots into their material handling operations, but over the past 18 months, the trend has moved into high gear due to a surge in e-commerce demand and a general labor shortage during the new crown epidemic.   This demand is not expected to stabilize anytime soon. The surge in automation could have a ripple effect on the workforce, namely rewriting the job descriptions of many workers in manufacturing, logistics and retail. Not only will robots speed up operations, especially in areas with severe labor shortages, they will also create new jobs for workers who can manage, maintain and "train" automated equipment, suppliers say.   How are robots changing jobs in the warehouse material handling industry? How are people "training" robots to do certain jobs in DC? Currently, most robots are programmed by people who either write the software code or physically guide the robot's arm into the right position. But the next generation of robotics increasingly relies on artificial intelligence (AI) to guide the direction, giving workers complete freedom to perform other DC tasks. How is robotics changing the material handling process in the mine? In the process of mining and transportation, ores are often mixed with wood, steel nails, rags, plastic parts, waste filling pipes and other sundries. These sundries has seriously affected the safety and effectiveness of equipment in the transportation, crushing, grinding and beneficiation . In the past, manual sorting was usually used to remove it, but there are serious safety and occupational health risks in manual sorting, as well as problems such as incomplete manual sorting. The robot for mining can effectively solve the above problems. Through a multi angle, multi-dimensional stereo high-precision camera, the intelligent material removal robot quickly scans the ore on the conveyor The self-developed CRM-CNN foreign object recognition algorithm accurately locates the 3D position of the debris, controls the robot to grab the The self-developed CRM-CNN foreign object recognition algorithm accurately locates the 3D position of the debris, controls the robot to grab the foreign object, and puts it into the foreign object collection box.  

Application of Artificial Intelligence in Coal Mine Robotics   Abstract With the rapid development of artificial intelligence technology, its application in coal mines has become more and more extensive. In the process of coal mine production, the urgency of the demand for robot replacement has accelerated the industrial application of coal mine robots and the application of artificial intelligence technology in coal mine robots. The application of artificial intelligence technology in coal mine robots is analyzed and explored, the main research contents of artificial intelligence technology and its application in industry are introduced, the current situation of the application of artificial intelligence in coal mine production is analyzed, the concept of applying artificial intelligence technology to coal mine robots effectively is elaborated, and the prospect of the development of artificial intelligence in coal mine robots is prospected.   Keywords artificial intelligence, coal mine robot, intelligent perception, intelligent decision making, intelligent monitoring，material removal robot 0 Introduction The underground production and operation process of coal mine has the problems of many people going down the shaft, high risk of disaster, high accident rate, harsh operating environment and serious environmental pollution [1]. Facing the high-risk underground operations, coal mine robots become one of the important ways to achieve the goal of safe and efficient underground coal mine production. Coal mine robots can assist or replace people to complete some dangerous mining operations and achieve safe and efficient production in coal mines. In order to achieve "no one is safe", robots are the trend to replace miners in underground operations.   With the strategy of "Made in China 2025", "German Industry 4.0" and "American Industrial Internet", 5G communication, Internet of Things, big data, cloud computing and artificial intelligence The gradual maturity of technologies such as 5G communication, Internet of Things, big data, cloud computing and artificial intelligence has greatly promoted the transformation and upgrading of China's traditional manufacturing industry [2]. As an emerging science and technology, artificial intelligence can make computer technology more accurate, fast, and convenient to complete complex scientific calculations that the human brain is incapable of undertaking, and achieve partial replacement, extension, and enhancement of the human brain, thus creating intelligent machines that can complete complex and dangerous operations instead of humans [3].   Future coal mine production will develop toward unmanned, autonomous, intelligent, and efficient, in which artificial intelligence technology will play an irreplaceable role and diverse artificial intelligence technologies will be applied to coal mine robots [4]. Although the current application of artificial intelligence in the field of industrial coal mines is still in the fumbling period, however, with the increasingly widespread application of artificial intelligence technology in the field of coal mines, the construction of unmanned operation mines is inevitable [5].   1 The urgent problems in the coal industry China's coal industry has experienced more than 40 years of development, and the mining of coal mineral resources gradually tends to be intelligent, but there are still some bottlenecks that need to be solved.   1.1 Technology and equipment need to be upgraded Although the mining and transportation of coal in China have gone through the stages of digitalization, automation and informatization, the overall technical level and production equipment are still lower than those of developed countries [6].In 2019, the former State Administration of Coal Mine Safety Supervision proposed to accelerate the industrialization and application of coal mine robots for digging, coal mining, transportation, safety control, support and rescue. The current coal mine robot is no longer just completing simple repetitive operations, it can sense the surrounding environment and give real-time feedback to the outside world, but it does not yet have independent thinking, identification, reasoning, judgment and decision-making capabilities, and still needs human participation to complete some complex work tasks.   1.2 Serious safety hazards The coal industry is a high-risk industry, and there are various hazards in every step of production, water, fire, gas, coal dust, geological formations and other disasters are frequent, and the unknown complex underground environment seriously threatens the life safety of underground operators. Although the intelligent monitoring and early warning technology of coal mines based on the Internet of Things, big data and cloud computing has largely reduced the occurrence of accidents and guaranteed the safe production of coal mines, there are still many problems. The poor accuracy and sensitivity of sensors lead to incomplete and untimely collection of precursor information; monitoring systems are independent of each other and have a single function, and the integration and application integration depth of the cloud platform is not deep enough; monitoring system database security is weak; monitoring equipment lacks deep learning as well as self-adaptive capability [7].   1.3 Serious environmental pollution Coal mines produce coal dust during the mining process, and also produce harmful gases such as carbon monoxide and carbon dioxide to pollute the atmosphere [8]. At the same time, the production effluent from coal mining contains a large amount of heavy metals and acidic substances, which can easily seep into the soil or enter the groundwater to pollute the geology and water sources. Coal mining projects will encroach on a large amount of vegetation and agricultural land, and the land is prone to collapse after mining leading to the destruction of the surface layer [9].   2 The main research content of artificial intelligence 2.1 Pattern recognition Pattern recognition in artificial intelligence technology uses the powerful data collection, analysis and processing functions of advanced computer technology to simulate human perception and recognition of the external environment by setting the corresponding programs in advance. Intelligent robots incorporating pattern recognition can better simulate human sensory abilities, recognize characters, sounds, images, scenes and their fused information with high accuracy, and accurately perceive and model the surrounding environment through the acquisition of multi-source information [10].   Machine vision in artificial intelligence technology, as one of the most important environmental perception modalities, simulates human visual capabilities to improve the robot's understanding of the downhole environment, operational processes, and feedback phenomena. Intelligent robots incorporating machine vision are, first, able to adapt well to the downhole operating environment and collaborate well with other artificial devices; second, able to capture more information about the external landscape and to understand and dig deeper into the content of images through stereo vision, visual inspection, and dynamic image analysis techniques; and third, able to judge the underground feedback phenomena of the operating process and feed back information about the robot's status to the motion control system [11].   2.2 Expert system Expert systems are technologies that model the knowledge and experience of human experts and are used to solve problems such as system decisions, processes, and failures. Through artificial intelligence techniques, knowledge systems are created for downhole systems that simulate humans to solve practical problems encountered during operations. Human experts can predict system failures, determine failure points and generate troubleshooting solutions based on the current state of the system, such as equipment displays and sounds, operational data parameters, and the state of the product, when solving real-world problems. Therefore, expert systems are commonly used for fault prediction, diagnosis and troubleshooting. In addition, in the manufacturing industry, expert systems are also used for production planning decisions, production process optimization, production coordination, and optimization of equipment parameters.   2.3 Machine Learning Machine learning in artificial intelligence technologies mimics human learning capabilities through model frameworks and algorithms to automatically extract intrinsic laws through training data, environmental information, and feedback to improve system performance and enhance environmental adaptation and robustness. Robots that incorporate machine learning have human-like law extraction and knowledge summarization capabilities to identify effective information from the large amount of information resources collected and learn to improve their own intelligence. Machine learning technology can effectively solve a series of problems in unexpected situations and largely reduce labor and production costs [12].   2.4 Distributed Artificial Intelligence Distributed artificial intelligence system coordinates the scheduling and control of heterogeneous multi-intelligent body systems by scientifically and rationally combining artificial intelligence and computer technology, so as to enhance the performance of the artificial intelligence system, improve the task execution capability, and increase the efficiency of the cooperative work of each independent system in the intelligent robot. When the intelligent robot encounters some unexpected situations, it can still guarantee each subsystem to carry out normal work. The current distributed artificial intelligence system is still in the initial stage of research and development, and the technical difficulty lies in how to coordinate the operation rules of different systems [13].   3 Status of application of artificial intelligence in coal mine robots 3.1 Application of artificial intelligence in the motion control of coal mine robots In order to ensure that coal mine robots can operate properly in complex underground environments, research scholars have applied artificial intelligence technologies such as expert systems and artificial neural networks to robot motion control methods, algorithms and collaborative operations. By simulating human expert thinking and knowledge level, coal mine robots can solve some complex multidimensional nonlinear problems, reduce the amount of operations for dynamical system analysis, parameter setting and data processing, and improve control efficiency and accuracy.   Wang Nian et al [14] researchers designed an intelligent mine robot based on embedded ucos and used GSM network to realize remote control of the device; Zhang Chuancai et al [15] researchers used BP neural network to establish a measurement method to determine the robot turning angle based on motor speed and running time, which can provide angle parameters for robot path planning; Wang Xuesong et al [16] researchers personnel approximated the kinetic uncertain parameters based on the improved Elman neural network and sent control commands for the coal mine robot servo system using a neuro-fuzzy controller; Song Xin et al [17] researchers applied neural networks in the field of robot control to accomplish actions such as robotic arm multi-joint coupling control, end trajectory planning, and hydraulic valve control.     3.2 Application of Artificial Intelligence in Intelligent Perception and Danger Prediction of Coal Mine Robot Mining inspection robots realize all-round perception of underground environment information by carrying various sensors, real-time monitoring of instrument and equipment failure, personnel safety and disaster information such as gas, coal dust, water and fire, and timely issuance of early warning to reduce the occurrence of coal mine accidents. For several technical difficulties such as inaccurate identification and untimely monitoring in complex underground environments, researchers use deep learning, pattern recognition, and expert system technologies to further enhance the robot's accurate identification and real-time monitoring of underground emergent hazards.   Researchers in Lu Wanjie et al [18] used deep learning algorithms based on convolutional neural networks to model and train coal mine equipment so that the underground inspection robot could accurately identify the type of coal mine equipment; researchers in Zhang Fan et al [19] proposed a mining image reconstruction method based on residual neural networks for the disturbing effects of underground noise on the visualized operating environment, which effectively improved the clarity of monitoring images and Nie Zhen et al [20] used a genetic algorithm based on BP neural network to build a tunnel gas environment intelligent detection system and obtain real-time data of gas concentration distribution on different tunnel sections in the path of coal mine inspection robots; Pan Yue et al [21] used BP neural network to establish a diagnostic model for fan faults and establish a mapping between fan fault types and fan rotor vibration frequency bands, thus realizing fan fault diagnosis. relationship, and then achieve fan fault diagnosis; Yan Junjie et al [22] researchers based on artificial neural network to establish a diagnostic model for coal mine machinery gear faults, using the input signal to train the neural network model, classify the output signal, and then determine the gear fault.   3.3 Application of artificial intelligence in autonomous positioning navigation and map construction for coal mine robots Achieving autonomous positioning and navigation in complex unstructured coal mine environments requires consideration of both the inability of GPS technology to be applied directly downhole and the need to overcome interference from external factors such as dust, temperature, humidity, noise, and airflow, which places higher demands on autonomous and accurate positioning and navigation technology for robots in restricted and closed environments downhole. Map construction, positioning navigation, path planning, and real-time obstacle avoidance of coal mine robots based on artificial intelligence technology have become hot spots for applied research.   Bai Yun [23] proposed variable structure fuzzy neural network and applied it to the environment sensing process of snake underground rescue robots, fusing multi-source sensor data to achieve obstacle recognition and environment modeling of snake robots in harsh environments; Fu Hua et al [24] researchers used artificial neural network model to model and dynamically describe the workspace of intelligent coal mine monitoring system, using neural network model for robot obstacle avoidance path planning; Zhang Yaofeng et al [25] researchers used Elman network-based compensation for ultrasonic sensor measurement error of underground robot, which greatly improved the accuracy of ultrasonic ranging and obstacle detection; Zhai Guodong et al [26] researchers summarized binocular vision technology in coal mine rescue robot to obtain accident scene information and achieve autonomous obstacle avoidance and path planning, including pattern classification and recognition, visual measurement and 3D reconstruction, combined measurement and localization, and visual servo control; Ma Hongwei et al [27] researchers constructed a depth camera-based machine vision system and proposed a depth vision-based navigation method, in which the robot is equipped with an RGB-D depth camera for data acquisition to achieve map creation and autonomous navigation.   4 Research on intelligent coal mine robots There are various kinds of artificial intelligence technologies, and the main research contents applied to the field of coal mine robots include multimodal fusion intelligent perception, knowledge learning and intelligent decision making, and intelligent control cooperative operation. Through perception, learning, decision making, and collaborative control, the intelligent development of coal mine robots is realized.   4.1 Multi-modal fusion intelligent perception The coal mine robot is equipped with various explosion-proof, high-precision and high-reliability sensors to build an intelligent perception system with multimodal fusion of vision, hearing, smell, touch, etc., to complete intelligent recognition and analysis, abnormal sound recognition, abnormal temperature monitoring, smoke detection, harmful gas concentration detection, autonomous obstacle avoidance, autonomous grasping and other operations.   (1) Research on machine vision recognition and visual detection technologies in coal mine application scenarios. Through image processing and understanding, the robot is able to, firstly, identify and monitor equipment digital meters, LCD screens, indicators, valves, etc.; secondly, detect pipeline liquid dripping, tape running and cracking; thirdly, carry out personnel intrusion, personnel on duty, personnel dressing detection; fourthly, identify and track foreign objects such as gangue, anchor rods, road logs, iron pipes, etc. that appear on the tape.   (2) Research on technologies such as robot hearing i.e. sound detection and recognition in coal mine application scenarios. Using high-sensitivity sound pickup sensor, high-speed DSP digital signal processor, combined with adaptive dynamic noise reduction processing technology, audio feature extraction and detection model algorithm recognition technology to identify abnormal sound in the mine.   (3) Research on intelligent recognition technology for robotic olfaction i.e. gas detection in coal mine application scenarios. Accurate detection of methane, hydrogen sulfide, carbon monoxide, oxygen and other gases in the environment and whether smoke exceeds the limit, timely detection of gas leaks and early warning of fires.   (4) Research on haptic technology for robots in coal mine application scenarios. Collect the temperature of objects such as motors, pumps, bearings, rollers, tapes, etc. through contact or non-contact and analyze the data; through force sensing equipment, real-time monitoring of contact force, gripping force, operating force, internal stress, and realize force sensing and safety control.   4.2 Knowledge learning and intelligent decision making In view of the current problems of incompatible coal mine robot system protocols and lack of information sharing and integration, we will deeply integrate coal mine robots with new generation information technology, build a generalized, standard and flexible system for mutual learning and knowledge sharing of coal mine robots, and break through the technical bottlenecks of coal mine robot scene understanding, safety detection, precise positioning, autonomous perception and efficient navigation. Realize cloud-based online services for common technologies of coal mine robots to solve the limitations of individual robots and improve the intelligent decision making of coal mine robots.   (1) Establish a learning and generalization framework that integrates the individual and the whole. At the individual level, a single robot integrates sensing, decision-making, control, collaboration, and human-robot interaction information during operation, and conducts incremental, real-time, online training through an artificial intelligence learning framework represented by neural networks to dynamically adjust the robot's operational state and achieve full-cycle optimal control and decision-making. At the overall level, multi-robots upload and distribute their learned knowledge among themselves through the new generation of information technology, so that when a robot faces a brand-new operational task, it can quickly familiarize itself with the operational characteristics with the knowledge results of other robots, reduce re-learning time, and enhance the overall system's task flexibility and adaptability.   (2) Establish an operation mode in which the robot body and the cloud are integrated. Breakthrough the traditional robot R&D and integration model, and realize a new robot R&D and integration route that integrates the local lightweight robot body with the high-performance data processing capability in the cloud with the help of "5G + cloud computing". The algorithms that require strong computing power, such as intelligent environment perception, pattern recognition, map construction, and autonomous navigation, are moved to the cloud, and the local robot uploads the data of on-board sensors and actuators to the cloud in real time, and optimizes the calculation of perception, modeling, and execution through the powerful data processing and computing power of the cloud. The result is sent to the local robot in real time, which reduces the computational burden of the local robot and shifts more hardware resources to the sensor and execution side to achieve a lightweight, streamlined, and high-performance operational robot design.       4.3 Intelligent control of cooperative operation Integrating deep learning and laser/visual SLAM technologies into coal mine robots, combined with intelligent sensing system of multimodal fusion, realizes the functions of autonomous movement, precise positioning, position adjustment, intelligent operation planning, autonomous operation and intelligent disaster sensing of coal mine robots in complex mine environment, and realizes intelligent collaborative control of detection, digging and support operation processes.   (1) Integrating neural network technology into the collaborative operation control and planning of multiple coal mine robots. Self-organization, self-grouping and self-coordination of mobile robots in mines to achieve integration of heterogeneous equipment. Through intelligent task decomposition, task assignment, and load balancing technologies, a swarm of robots in complex environments in mines is formed, and technologies such as autonomous navigation in underground space, multi-sensor state sensing, intelligent operation planning, and multi-robot collaborative control are applied to realize efficient collaborative operations among multiple robots in workface digging, drilling, extraction, transportation, and support.   (2) Extending the mode of human interaction with a single robot to human interaction with multiple robot groups, and realizing the intervention and collaboration of operators on robot groups. During the operation of coal mine robots, each heterogeneous robot with different functions forms a complex multi-robot collaborative swarm. At the same time, the multi-robot collaborative swarm needs to be able to collaborate with the operator in depth. Through AI technology, we can break through the simple "command-execution-display" mode of existing human-robot interaction technology, and integrate human intervention into the control cycle to realize a new mode of human-robot interaction with "human in the loop", and realize the "unmanned underground system group + unmanned underground system group + unmanned underground system group". underground unmanned system group + underground operator" mode of operation, to improve the overall system's operational efficiency, task flexibility and robustness.   In order to achieve the goal of smart coal mine, we will carry out research on "coal mine robot +", "coal mine robot + 5G" to realize comprehensive sensing and interconnection, full domain information sharing and multi-channel human-robot interaction; "coal mine robot + cloud computing "Coal mine robot + cloud computing" realizes the compatibility of lightweight and low-cost robot ontology and high-performance learning computing capability; "coal mine robot + big data" realizes dynamic prediction, information integration, and provides data basis for robot evolutionary learning; "coal mine robot + AI "Mine robot + AI" realizes intelligent autonomous perception, optimal analysis and decision making, and knowledge learning evolution, thus forming a complete intelligent system of three-dimensional perception, autonomous learning and cooperative control in the mine.   5 Future Outlook Artificial intelligence has been widely applied in the field of coal mine robotics, and more research results have been achieved. However, as an emerging frontier technology, artificial intelligence still has limitations.   (1) The current AI technology is mainly oriented to a single task, and a general AI framework that can face multiple tasks has not yet been realized. For example, the models trained for image recognition cannot be used for sound detection and recognition; the algorithm framework for recognizing a specific target object cannot be extended to the recognition of arbitrary target objects, and the data set needs to be constructed and retrained when a new classification target appears. This feature limits the application of AI in complex task scenarios.   (2) Artificial intelligence algorithms need to rely on a large amount of data, and operations such as data collection, processing, calibration and alignment need to be done manually, which is less efficient. How to use a smaller amount of data to achieve higher performance has become one of the current research hotspots of artificial intelligence methods.   (3) There are many types of coal mine robots, and there exist a large number of sensing devices, driving devices and actuating devices. The data formats of each device are diverse, and it is difficult to form a unified data interface, making the data between each system independent of each other. The incompatible data makes it difficult for the AI system to coordinate the robots in each part of the coal mine production process, and to obtain enough data to form a closed-loop unified plan for the entire production process.   (4) The environment in which coal mine robots operate is extremely dangerous, so current AI systems alone cannot guarantee a high level of safety and stability. How to integrate the AI system with the operator's manual intervention and integrate human intervention into the whole AI system's operation loop becomes one of the key elements to be addressed in the next step.   In the future, AI systems applied to coal mine robots will develop toward generalization, low overhead, unification, and human-machine collaboration, with the emergence of a general AI algorithm framework for multiple tasks that continuously learns and evolves online using small amounts of data and low-cost training methods, capable of integrating key data from all aspects of coal mine production for integrated computing and scheduling, and able to collaborate with each other and humans to achieve It is capable of collaborating with humans to achieve efficient, safe, and autonomous coal mine production.   6 Conclusion With the development of artificial intelligence technology, the coal mining industry will see a major change. With the efficient model building, parallel computing, and planning capabilities of AI, the intelligence and automation of coal mine robots will reach a new level, truly realizing the unmanned and safe requirements of coal mine production. At the same time, artificial intelligence will enable a significant increase in coal mine production efficiency and promote the safe, healthy and sustainable development of the coal mine industry.