Professor Liang Libin from Xi'an Jiaotong University was invited to review the research progress of intelligent optical assisted technology and laser ablation therapy in minimally invasive intervention, and was selected as a highlight article in the biomedical photonics special issue of China Laser.
Original link: Liang Libin, Li Liang, Gao Tingting, Wang Guangzhi, Ding Hui, Wan Mingxi, Zhang Zhenxi Research progress on intelligent optical assisted technology and laser ablation therapy in minimally invasive intervention [J] China Laser, 2023, 50 (15): 1507201
1、 Background Introduction
Minimally invasive intervention is a non-surgical procedure that can effectively treat many complex diseases with minimal trauma, and has been widely used in clinical practice in recent years. Usually, minimally invasive intervention techniques use laparoscopes, needle devices, or wires to reach the lesion through percutaneous puncture or natural lumens (blood vessels, trachea, etc.), combined with physical resection, thermal injury, radiation, and other methods for treatment; However, due to the use of small incisions in the surgical process and the need to minimize damage to normal tissues, the doctor's surgical field of view and operating space are limited.
Image guidance is the third eye of the doctor during minimally invasive intervention, which can provide auxiliary information for the doctor during the operation. However, traditional imaging guidance, including CT, MRI, ultrasound, and X-ray fluoroscopy, still has the following problems in obtaining and presenting surgical information: 1) inability to provide stereoscopic vision; 2) The operator's attention needs to frequently switch between the screen and the surgical position, resulting in hand eye coordination issues; 3) Difficulty in real-time, continuous, and accurate tracking of the relative position of lesions and surgical tools. Therefore, the surgical effect highly relies on the doctor's experience and spatial imagination. In addition, the existing minimally invasive intervention surgery also has limitations in terms of treatment forms (such as mechanical resection, hyperthermia, or radiation therapy) or intraoperative treatment status monitoring, making it difficult to accurately control the treatment scope.
Optical technology, as a means that combines energy and information properties, has been widely applied in the field of biomedical science in recent years, while also providing new solutions to address the aforementioned issues. Augmented reality (AR) technology can fuse surgical scenes with virtual images and present them to doctors, providing new intraoperative information visualization solutions; Optical tracking and perception technology can provide quantitative spatial information for locating lesion tissues or surgical tools, laying the foundation for precise surgical operations; In addition, laser energy is concentrated and has good spatial directionality. Laser ablation technology can flexibly transmit laser to the lesion through slender optical fibers, and cooperate with intraoperative MRI temperature monitoring to achieve more accurate treatment results. In the context of the development of artificial intelligence, this article focuses on the development of optical assisted technology and laser ablation therapy, and reviews them from three directions: AR assisted technology, optical tracking and perception technology, and laser ablation therapy.
2、 Key technological progress
1. AR assisted technology in minimally invasive intervention
AR assisted technology provides a new way of information visualization for minimally invasive intervention, which commonly includes three categories, as shown in Figure 1.
Figure 1 shows three typical types of AR applied to minimally invasive interventions. (a) Display based AR; (b) Projection based AR; (c) AR based on optical perspective
The principle of display based AR is to use a camera to record the optical information of the surgical scene, and fuse the actual image with the virtual image in the computer. Finally, transparency and color adjustment are performed, and the fused optical information is output through the display. The principle of projection based AR is to project virtual images directly onto the surface of objects in actual scenes by designing optical paths, achieving visual enhancement effects. AR devices based on optical perspective typically include image source devices and display optical systems. The key components of traditional display optical systems are semi transparent and semi reflective mirrors. The image source device projects virtual images onto the mirror, and the projected light path is overlaid with the actual scene light path reflected and incident, achieving the fusion effect of virtual and reality. Three types of AR clinical applications are shown in Figure 2. The three AR technologies each have their own advantages and disadvantages in practical use, as shown in Table 1.
Figure 2 Clinical study example of AR assisted minimally invasive intervention. (a) Display based AR guided thoracolaparoscopic surgery schematic; (b) The in-situ projection enhancement system and projection enhancement diagram of the C-arm X-ray machine; (c) Robot projector camera system and its projection AR function; (d) Experimental schematic of pedicle screw implantation guided by HoloLens; (e) Schematic diagram of extraventricular drainage guided by HoloLens
Table 1 Advantages and Disadvantages of Different AR Technologies
Overall, head mounted optical perspective AR technology is currently the trend and hotspot of development. The combination of intelligence and AR assisted technology is an important development direction, making registration and interaction in augmented reality more intelligent, accurate, and efficient. On the one hand, new optical devices such as structured light and depth cameras provide convenience for intraoperative surface acquisition. On this basis, deformation registration algorithms based on deep learning and physical simulation have shown potential, making registration more convenient, accurate, and real-time. Secondly, interaction with AR devices is an important issue that needs to be faced in its clinical promotion. The current development trend is to combine perception of multiple information and artificial intelligence technology. The application of new technologies will make augmented reality more intelligent in interaction, able to understand human operations, and reduce the burden on operators.
2. Optical tracking and perception technology in minimally invasive intervention
In minimally invasive surgery, the patient's lesions are mostly located deep inside the body, and the treatment process is usually performed subcutaneously, making it difficult for doctors to directly observe. Therefore, it is necessary to locate, track and perceive the anatomical structure and surgical instruments. Related technologies include optical tracking technology, fiber Bragg grating (FBG) technology, optical imaging technology, and optical based 3D surface reconstruction technology.
The optical tracking system is the core part of surgical navigation equipment, mainly utilizing the principle of linear propagation of light. It generally consists of a dual (multi) camera, tracking markers, and controller, as shown in Figure 3 (a). After completing tool calibration and spatial registration, the optical tracking system can obtain the real-time spatial position relationship between the tool and the surgical object, equivalent to the GPS of the doctor's surgery. With the increasing popularity of minimally invasive intervention surgery, there is a greater demand for intelligent surgical navigation, which requires optical tracking systems to not only accurately obtain poses, but also perceive more surgical scene information, such as NDI's latest Polaris Vega ® The VT optical tracking system can not only track and locate, but also perform real-time video capture.
FBG is a type of fiber optic sensor that has the advantages of small size, high sensitivity, anti electromagnetic interference, and easy distributed layout. It can measure various signals such as deformation, stress, temperature, and vibration. There are periodic gratings inside the FBG fiber core that have a selective effect on the wavelength of the incident light. When broadband light sources enter the fiber optic, only specific wavelengths of light signals will be reflected, presenting peaks in the reflection spectrum. When equidistant fiber Bragg gratings are subjected to external forces, temperatures, and other energy, the spacing between the gratings will change, causing a shift in the reflection peak. By analyzing the offset, the physical information changes at the grating location can be determined, as shown in Figure 3 (c). With the deepening application of flexible instruments in minimally invasive interventions, FBG has become a hot topic in the field of minimally invasive medical sensing. It has great potential for application in force sensing between instruments and tissues, tissue temperature sensing, and instrument deformation sensing.
Figure 3 (a) Typical light tracking system; (b) Optical tracking system applied in robot surgical navigation; (c) The principle of measuring stress and temperature using fiber Bragg grating
Optical imaging technology and optical based 3D surface reconstruction technology play an important role in perceiving tissue morphology. For example, 3D surface reconstruction technology is widely used in laparoscopic surgery to perceive the surface geometric information of tissues. Various optical imaging technologies can obtain images of the surface layer of tissues for guiding surgery, such as optical coherence tomography (OCT), photoacoustic imaging (PAT), and confocal imaging. In recent years, artificial intelligence technology has further promoted the development of optical tracking and perception, and some traditional challenges are gradually being overcome, such as spatial registration and occlusion problems in optical tracking, 3D surface reconstruction problems, and 3D image segmentation and recognition.
3. Laser ablation technology in minimally invasive intervention
The principle of laser ablation therapy is to implant slender optical fibers into the lesion through skin or natural channels. The laser is transmitted by the optical fibers to act on the tissue, causing protein denaturation and tissue coagulation due to tissue heating, thereby achieving the therapeutic goal. In existing laser ablation equipment, transparent conduits are added to the outside of the optical fiber to reduce optical density and prevent overheating; At the same time, a cooling mechanism was introduced, as shown in Figure 4 (a). Usually, lasers have various effects on tissues, such as photochemical reactions, thermal interactions, photo etching, plasma induced etching, and photo induced rupture. Among them, the clinical application of thermal interaction is the most typical, which can be used in neurosurgery and various tumor treatments. Laser ablation combined with real-time MR temperature monitoring can achieve precise and minimally invasive treatment of brain tumors, which is an important technological advancement in the field of neurosurgery, as shown in Figure 4 (b). In addition, laser ablation is also suitable for the treatment of vascular lesions, especially thrombolysis, as shown in Figure 4 (c).
Figure 4 (a) Structure of laser ablation fiber; (b) Schematic diagram of brain laser ablation combined with MR temperature monitoring; (c) Schematic diagram of laser ablation thrombolysis
3、 Summary and Outlook
In the context of the rapid development of artificial intelligence, optical assisted technology and laser ablation technology have injected new vitality into precision minimally invasive intervention surgery. AR technology provides a new way of visualization, enhancing the confidence of doctors in their operations; Optical tracking and sensing technology makes surgery more intelligent and precise; Laser ablation has also demonstrated advantages in precise tumor treatment. Currently, relevant research is constantly emerging, and we have reason to believe that with the continuous development of optical assisted technology and laser ablation technology, future minimally invasive intervention surgeries will become more intelligent, accurate, and personalized.
Introduction of Corresponding Authors
Liang Libin, an assistant professor and master's supervisor at Xi'an Jiaotong University, mainly engages in research on computer-aided surgery, surgical navigation, personalized surgical planning and simulation, and explores intelligent assisted technology for minimally invasive surgery based on medical imaging. Published papers in IEEE Transactions on Medical Imaging and the Medical Image Processing Summit MICCAI. Currently undertaking multiple projects including the National Natural Youth Science Foundation, Postdoctoral Fund, and Zhejiang Provincial Natural Science Foundation.
Note: This article (including images) is a reprint, and the copyright of the article belongs to the original author. If there is any infringement, please contact us to delete it.
