Enhancing surgical training through immersive and interactive virtual reality experiences in robotic surgery

Introduction

Robot-assisted surgery has demonstrated remarkable advantages compared to conventional laparoscopic procedures, particularly in terms of superior visualization, enhanced maneuverability, and improved ergonomic considerations (1-3). Its adoption continues to expand across various medical specialties, encompassing thoracic surgery and bariatric surgery.

It is paramount to acknowledge that the success of establishing a robotic surgery program hinges on critical elements, with training emerging as a pivotal component (4,5). The introduction of novel surgical technologies, techniques, or procedures necessitates a well-structured training and certification process to ensure their safe integration into daily practice without compromising patient safety or procedure efficacy (6).

The practice of observing surgeries performed by an expert team has emerged as a pivotal stage in the education not only of the surgeon but also of the entire surgical team (5-7). However, completing this stage, often requiring visits to centers in distant locations, is frequently hindered by constraints related to time and expenses.

Recent advancements in virtual reality (VR) experience design and VR head-mounted display (HMD) devices have paved the way for the creation of immersive 360° experiences that replicate an in-person visit to remote surgical sites. VR 360° offers a portable solution that faithfully reproduces a real-world environment, affording users the ability to navigate and interact with various elements within the scene (8,9). These immersive experiences grant access to insights concerning the setup of the operating room, instrument utilization, and technical intricacies of the surgical procedure (10).

In this concise report, we present the conceptualization and development of an immersive, interactive VR 360° experience that faithfully simulates a visit to a robotic surgery operating room. This simulation encompasses both robot-assisted thoracic surgery (lobectomy) and robot-assisted bariatric surgery (duodenal switch).

Materials and methods

360 immersive and interactive tour to the operating room

The immersive visits to the operating room during the performance of robotic thoracic (lobectomy) and robotic bariatric (duodenal switch) procedures were developed by capturing 360° videos with a ONE RS1 camera (Arashi Vision Inc., Shenzhen, China). For this purpose, the camera, which has the capability to record 360° videos without the need for movement, is placed in a relatively central location within the operating room, typically close to the operative table. Given that the goal is to provide information about the setup of the operating room, only 5 to 10 minutes of the procedure need to be recorded. This video can be used as a loop if more recording time is required during the design/edition of the experience. The recording begins after the patient is fully draped, and all necessary measures to not reveal protected health information (PHI) are taken into consideration.

The images/videos were exported for editing with Virtual Tour Pro (3Dvista, Granada, Spain). Using this software, the course of the experience was determined. Three different scenes are presented: during the initial scene, instructions on navigating between scenes and how to interact with the “hotspots” are provided. The second scene corresponds to the visit to the actual operating room, where interactive elements are available for obtaining information related to knowledge/strategies, operating room configuration, and instrumentation. This information is presented graphically or textually to demonstrate. The third and final scene corresponds to critical steps of the procedures, which are demonstrated through the presentation of edited segments that highlight key elements of the surgical procedure. The videos related to technical aspects are 2D recordings incorporated into the 3D environment of the experience using the mentioned software (Figure 1). The designed experiences focus on robot-assisted thoracic surgery (Video 1), and bariatric surgery (Video 2). Once the video editing is completed, it is exported to the cloud.

Figure 1 Virtual reality immersive and interactive experience. Scene I: instructions. Scene II: operating room. Scene III: critical steps.

The entire experience can be downloaded to be used on any HMD, such as the “Meta Quest 2”, which operates independently, meaning it does not require a connection to a computer to function, providing great versatility. The interface used to recreate the experience on the HMDs is 3D Vista Tour Viewer, which is freely available in the application menu of the devices. The visit can also be conducted in 2D environments; however, this reduces the quality of the experience and does not provide the benefits associated with the fully immersive environment.

Discussion

VR is unquestionably a groundbreaking technology on the ascent, offering numerous applications that extend well beyond mere entertainment. The core objective of VR is to craft immersive and interactive experiences that faithfully replicate genuine environments (8,9).

The utilization of VR as an instructional and training tool has been documented across various domains, including sports, computing, geography, language learning, and the military industry, among others (11-14). In the medical realm, its applications have been diverse, encompassing pain management for patients, aiding in emotional and physical recovery, and proving to be a highly effective tool for addressing mental health conditions (15,16). Several studies have reported that immersive experiences have a profound impact on emotional responses, translating into heightened attention and engagement (10,17).

VR experiences can either be entirely computer-generated or based on the utilization of cameras and 360° videos. When experienced with a HMD, panoramic 360° experiences are considered fully immersive (18). HMDs synchronize image presentation with the user’s head movements, maintaining a consistent position and orientation relative to the user’s eyes. This approach is not only more accessible but also offers a considerably superior cost-benefit ratio compared to entirely computer-generated experiences (18).

The use of 360° videos as a tool to orient medical students or assistants before entering the surgical environment has already been documented (19,20). Harrington et al. introduced a technique-centric approach to surgical instruction, employing a 360° video supplemented with additional elements related to the procedure. This approach demonstrated a significant increase in attention compared to traditional two-dimensional videos (10).

The use of this application could be particularly useful for the training of operating room staff in advanced robotic surgery procedures, to the extent that the staff becomes more familiar with the configuration of the operating room and the critical steps of the surgery, operating room efficiency could be increased. However, given that there are multiple variables that can intervene in the process, objectively determining the impact becomes difficult.

Observing live cases using this technology is technically possible, however, other technical factors related to latency and potential privacy violations limit its use. The transmission of 360 videos requires an adequate and stable internet speed that is not always available. Using this type of technology with suboptimal bandwidth decreases the quality of the experience, in which case we think it would be preferable to use conventional 2D videos. Undeniably, the acceptance of virtual experiences varies among individuals. It is worth noting that today’s trainee surgeons are markedly different from previous generations, having grown up with digital technologies integrated into both their personal and professional lives from an early age. Consequently, their level of acceptance tends to be higher, and the incorporation of this technology can serve as a motivational boost.

The use of experiences like those described is not limited to robotic surgery. Additionally, we believe that far from being a tool with limited applications in low resources environments, it rather increases the possibilities of exposing surgeons to novel techniques and approaches without the need to travel to other regions/countries, which requires time and financial resources. While it requires an initial investment associated with the acquisition of HMDs, it is also true that there is a wide range of alternatives with various costs, some of which are really affordable. The cost of developing experiences like the one described is considerably lower than that of fully virtual experiences, which to some extent will allow for greater adoption.

In the near future, these applications could constitute a fundamental step in training for new procedures not only in medicine/surgery but in several fields. The current technology is sufficient to create experiences aimed at system configuration and step-by-step guidance. The implementation of new devices that allow the incorporation of other senses into the experience, such as haptics, will contribute to increasing the level of training and might allow the practice of psychomotor skills.

We hold the belief that the extensive journey from novice to expert surgeon, especially for complex procedures, will soon be substantially shortened, thanks to the potential for remote training made possible by extended reality. However, it is important to acknowledge that the evidence remains somewhat limited, and the imperative to continue researching the acceptability and effectiveness of this modality is more pressing than ever. To determine objectively the impact of the use of experience as described and its difference from traditional training methods will require extensive comparative studies, where controlling all the variables in the process will be a difficult task. Currently, we are actively conducting studies to assess the usability, utility, and potential limitations of these kinds of tools.

Acknowledgments

Funding: None.

Footnote

Peer Review File: Available at https://jovs.amegroups.com/article/view/10.21037/jovs-23-50/prf

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at https://jovs.amegroups.com/article/view/10.21037/jovs-23-50/coif). A.T. and L.H. have received payment or honoraria from Intuitive Surgical, for lectures, presentations, speakers bureaus, or educational events. The other authors have no conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. All procedures performed in this study were in accordance with the ethical standards of the institutional and/or national research committee(s) and with the Helsinki Declaration (as revised in 2013). Ethical approval was obtained (Bioethics Committee of the University Hospital of Caracas, CBE No. 02/2022). Written informed consent was obtained from the patient for the publication of this article and accompanying images. A copy of the written consent is available for review by the editorial office of this journal.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. Teixeira A, Jawad M, Ghanem M, et al. Robot-assisted duodenal switch with DaVinci Xi: surgical technique and analysis of a single-institution experience of 661 cases. J Robot Surg 2023;17:923-31. [Crossref] [PubMed]
2. Kent MS, Hartwig MG, Vallières E, et al. Pulmonary Open, Robotic, and Thoracoscopic Lobectomy (PORTaL) Study: An Analysis of 5721 Cases. Ann Surg 2023;277:528-33. [Crossref] [PubMed]
3. Sánchez A, Rodríguez O, Jara G, et al. Robot-assisted surgery and incisional hernia: a comparative study of ergonomics in a training model. J Robot Surg 2018;12:523-7. [Crossref] [PubMed]
4. Herrera L, Escalon J, Johnston M, et al. Development of a robot-assisted thoracic surgery (RATS) program. Lessons learned after 2500 cases. J Robot Surg 2023;17:405-11. [Crossref] [PubMed]
5. Linsky PL, Wei B. Training in robotic thoracic surgery. J Vis Surg 2018;4:1. [Crossref] [PubMed]
6. Sachdeva AK, Russell TR. Safe introduction of new procedures and emerging technologies in surgery: education, credentialing, and privileging. Surg Clin North Am 2007;87:853-66. vi-vii. [Crossref] [PubMed]
7. Gill A, Randell R. Robotic surgery and its impact on teamwork in the operating theatre. J Perioper Pract 2016;26:42-5. [Crossref] [PubMed]
8. Pirker J, Dengel A. The Potential of 360° Virtual Reality Videos and Real VR for Education-A Literature Review. IEEE Comput Graph Appl 2021;41:76-89. [Crossref] [PubMed]
9. Barteit S, Lanfermann L, Bärnighausen T, et al. Augmented, Mixed, and Virtual Reality-Based Head-Mounted Devices for Medical Education: Systematic Review. JMIR Serious Games 2021;9:e29080. [Crossref] [PubMed]
10. Harrington CM, Kavanagh DO, Wright Ballester G, et al. 360° Operative Videos: A Randomised Cross-Over Study Evaluating Attentiveness and Information Retention. J Surg Educ 2018;75:993-1000. [Crossref] [PubMed]
11. Siu KC, Best BJ, Kim JW, et al. Adaptive Virtual Reality Training to Optimize Military Medical Skills Acquisition and Retention. Mil Med 2016;181:214-20. [Crossref] [PubMed]
12. Ji S, Li K, Zou L. The Effect of 360-Degree Video Authentic Materials on EFL Learners’ Listening Comprehension. In: 2019 International Joint Conference on Information, Media and Engineering (IJCIME), Osaka, Japan; 2019:288-93.
13. Tsai HH, Hou XY, Chang CT, et al. Interactive Contents with 360-Degree Panorama Virtual Reality for Soil and Water Conservation Outdoor Classroom. In: 2020 International Symposium on Educational Technology (ISET) Bangkok, Thailand; 2020:78-82.
14. He T, Chen X, Chen Z, et al. Immersive and collaborative Taichi motion learning in various VR environments. In: 2017 IEEE Virtual Reality (VR), Los Angeles, CA, USA; 2017:307-8.
15. Cherniack EP. Not just fun and games: applications of virtual reality in the identification and rehabilitation of cognitive disorders of the elderly. Disabil Rehabil Assist Technol 2011;6:283-9. [Crossref] [PubMed]
16. Uvais NA. Virtual reality applications in Schizophrenia. Ind Psychiatry J 2015;24:210. [Crossref] [PubMed]
17. Carlos Munoz-Carpio J, Cowling M, Birt J. Doctoral Colloquium—Exploring the Benefits of Using 360 0 Video Immersion to Enhance Motivation and Engagement in System Modelling Education. In: 2020 6th International Conference of the Immersive Learning Research Network (iLRN), San Luis Obispo, CA, USA; 2020;403-6.
18. Ritter KA, Chambers TL. Three-dimensional modeled environments versus 360 degree panoramas for mobile virtual reality training. Virtual Real 2022;26:571-81. [Crossref]
19. Fukuta J, Gill N, Rooney R, et al. Use of 360° Video for a Virtual Operating Theatre Orientation for Medical Students. J Surg Educ 2021;78:391-3. [Crossref] [PubMed]
20. Francis ER, Bernard S, Nowak ML, et al. Operating Room Virtual Reality Immersion Improves Self-Efficacy Amongst Preclinical Physician Assistant Students. J Surg Educ 2020;77:947-52. [Crossref] [PubMed]