Implementation of a Teleteaching Room Project for Surgeons Students: Aurélie Zanin, Mathieu Trocmé, Hélène Morel, Nathalie Chon-Sen, Céline Pagniez, Supervisor: Jean Chrétien Students, ENSPS Ecole Nationale Supérieure de Physique de Strasbourg, Louis Pasteur University, Strasbourg, France, e-mail:(aurelie.zanin,mathieu.trocme,helene.morel,nathalie.chonsen,celine.pagniez)@ensps.u-strasbg.fr, Supervisor, External Affairs Division, Alcatel, Marcoussis, France, e-mail:[email protected]

Abstract —This project, initiated by Alcatel concerns the creation of a teleteaching room for surgical applications. It is meant for the Institute of Research into Cancer of the Digestive System (IRCAD) in Strasbourg, France and takes place within the study program of a French engineering school, the ENSPS. It aims to enable a surgeon teacher to communicate in real-time with his students through an IPbased network. The surgeon in charge can perform his operation and send the video in real-time to his students working in different places. He can also send PowerPoint lectures, look after his students’ work and correct them. This project has been divided into three parts: the compression and decompression of the video stream, the transfer in real-time of data between several computers with switch possibilities and the implementation of a graphical interface meant for worldwide surgeons. The issues at stake are important. Two years ago, such technologies lead to the “Opération Lindbergh”. From New-York, Pr. J.Marescaux, founder of IRCAD, performed for the first time ever a surgical operation on a patient based in Strasbourg. This project follows the track of this achievement.

information and training. Ideal surgical apprenticeship would require the continuous presence of specialized experts in all departments of surgery. Since this is not realistic, surgeons are obliged to seek expertise elsewhere, in centers providing surgical education courses. There they meet recognized experts and acquire new surgical techniques through the means of video, live surgical demonstrations, and training on animals. To connect surgeons around the world without being limited by distance, teletraining facilities have to be developed [1]. A. The IRCAD institute, Fig.1

Index Terms —Teleteaching, IP technologies, Minimal Access Surgery.

I. INTRODUCTION The “Implementation of a teleteaching room for Surgeons” project aims to develop techniques and tools supporting Minimal Access Surgery (MAS or MIS for Minimally Invasive Surgery). In MAS, access is gained using natural openings or by applying minimal incisions. The surgeon operates with the help of long articulated instruments and an endoscopic video camera. The image of the operation is observed on a monitor. This technique allows the reduction of surgical access trauma and avoids post-surgical complications. It shortens significantly both the operation time and the hospitalization. In the operating room, new surgical instruments, imaging systems and robotic devices are being developed and integrated. Although still picture transmission is already operational, transmission and sharing of moving pictures of an acceptable quality require more study. In the near future, the association of advanced telecommunication means and new MAS technologies should allow the surgeon to perform remote operations. Any expert surgeon could remotely be solicited for advice on the operation underway or even better directly help a fellow surgeon. The rapid evolution of scientific knowledge and technical know-how in surgery explains the demand of surgeons for easy and full access to high-quality

Fig.1 The IRCAD institute

This project is focused on an experimental site inside the IRCAD building (Institut de Recherche contre les Cancers de l’Appareil Digestif / Institute for Research into Cancer of the Digestive System) assuming that “satellite” medical centers will be able to connect to the IRCAD system for diagnosis, training and information research. This institute is located in the University Hospital of Strasbourg, France. The European Institute of Tele-Surgery (EITS) is a department of the IRCAD [2]. It was created in 1994 to develop new computer and training technologies in the field of surgery through teaching and research programs. It provides a theoretical and practical training to all surgeons who want to specialize in telesurgery. Multimedia computer-aided learning in surgery is essential for the EITS. It will introduce important changes in surgical education. At IRCAD, surgeons will soon be able to acquire, test, and consolidate new skills from any location, using computer technology. In return they will receive accreditation and clinical privileges allowing them to practice in accordance with the most recent guidelines from scientific societies.

B. The IRCAD -EITS experimental operating theater The purpose of the IRCAD-EITS experimental operating room is to provide the surgeon with a real interactive teaching system, Fig.2.

switching network allowing the same performances as the former ATM network to replace it. II. IMPLEMENTATION OF THE TELETEACHING ROOM PROJECT The objective was to build up an application allowing to transfer different forms of information between the expert surgeon and his trainee surgeons in real-time. This application, including the development of a graphical interface, has been developed using a 3 computer-based prototype. The work has been divided into three parts: the compression and decompression of images, the routing of the continuous flow of information and the graphical interface implementation. A. Image quality requirements

Fig.2 Three surgical workstations in the IRCAD operating theater

The experimental operating room is made up of 17 experimental surgical tables, equipped with high quality surgical instruments. These tables allow the surgeon to operate under ideal conditions, without paying attention to the technical aspects of the material. Each surgeon has two screens in front of his operating table: the first is linked to the endoscopic video camera and displays the real surgical procedure the surgeon is performing, as in conventional operating rooms. The second screen is the “teaching screen”. It has to display surgical images, PowerPoint presentations or any other lecture supplies, to one or to all of the 17 tables together. So the teacher or the expert can show the trainee surgeon what he has to do during a procedure without having to move to the student. He can also correct his mistakes. So far, teacher and trainee surgeons have been in the same room but this technology is meant to connect surgeons from all around the world. C. The pr oject history The initial technology used at IRCAD was a videoconference system based on ISDN technology (Integrated Service Digital Network). Unfortunately this did not provide enough interactivity between all the tables. An experimental ATM-based system, with a local ATM switch, developed by Alcatel Business System between 1994 and 1997 took over within the framework of an Eureka project called MASTER (Minimal Access Surgery by Telecommunications and Robotics) [3]. This system offered a high quality real-time transmission of over 8 MBytes between each table. It could be used locally but also between any centers in the world linked to an ATM transmission system. Experimental ATM transmissions were performed between centers located in France (Strasbourg and Rennes) and America (Strasbourg and Montreal, Canada). Nevertheless, when the prototype switch broke down in 2000, such circuit switching network technology was not in use anymore. So, the IRCAD institute decided to use an analog system, allowing only images transmission. Since then, students of the ENSPS (Ecole Nationale Supérieure de Physique de Strasbourg, France / National School of Higher Education in Physics of Strasbourg) are working on implementing a packet

This application is intended to be used by surgeons who visualize their movements through an endoscopic video camera. Given the precision required, high quality images are needed. Amongst the several video coding standards available in 1995, the MPEG-2 standard in SIF (Standard Interchange Format) mode has been chosen. SIF images appeared to provide the best compromise between image quality (12 bits/pixels) and size with regard to the network data bit rate in use. The two main criteria taken into account were the following ones. Firstly, the resolution, that is to say the respect of details. Textures of organs and tissues must be respected with great care. An endoscopic surgeon recognizes and differentiates organs only thanks to their textures. The larger the picture, the finer the details. But the larger it is, the more memory is necessary (in fact, the square of the image size). Ideally it would have the same resolution as a television set (625 lines) but that would require far too much processing capability. With a resolution of 352x288 pixels, the SIF format appeared to be the most satisfying. It largely preserves quality and is easily exploitable. The second main criterion was the depth of color. A good restitution of the colors is necessary for the surgeon to differentiate between different tissues. Reliable digitalization of images implies at least 12 bits per pixel. It is this value that has been adopted. The precision of the surgeon’s movements requires a sufficient fluidity of the animation. The refresh rate was fixed to 25 img/s which is the same as that of the human eye. This provides realism in movement phenomena such as heart and pulse palpitation for which a good restitution is absolutely necessary. With such constraints a bit rate of 30 Mbits/s is required. The images have therefore to be compressed. The compression format retained was JPEG (Joint Experts Photographic Group) because of its standardization and great flexibility. JPEG compression is a destructive process. It can irremediably destroy the finest details if the quality factor applied is too high. This quality factor yields a compression rate that is variable depending on the complexity of the image. A quality factor yielding a compression rate of about 1:10 has been chosen. This provides a good compromise between image quality and size whilst avoiding the apparition of parasitic patterns, Fig.3.

Fig.3 JPEG compressed images with different compression rates

Concerning the movies, several standards for digital video compression exist. MPEG2 is based on key images depending on the previous and following images for which only changes are recorded. This is not really of interest for real-time video, especially because of this interpolation process. Twelve pictures are required before starting. With a frame rate of 25 img/s i.e. one image every 40 ms, a delay of about 500 ms appears. This is not acceptable for realtime remote operation. As researchers have shown, a lag time of 100 ms to 200 ms is reasonable for teleoperation [4,5], but above this threshold there is no more real-time interactivity between the remote surgeon movements and the actual operation he or she is performing. Furthermore, if the key image of one sequence happens to get lost, the whole sequence is lost. To avoid this, the MJPEG standard (standing for Motion JPEG) has been used. It is safer for this kind of application although more space-consuming. All the images are indeed compressed independently. No start time is necessary and if one image is lost, it does not affect the display of the whole sequence underway. Moreover MJPEG is not normalized. This makes it more flexible. B. Trans fer of images in real -time Three bits of information have to be transmitted either to one table or to all the 17 tables together: firstly, images of the operation being performed by the mentor; secondly, any type of external video sources such as recorded computerbased images or PowerPoint lectures; thirdly, any images from the 16 other experimental tables to be corrected and sent back in real-t ime to the trainee surgeon. This can be displayed on the main control screen to be analyzed or used for accreditation purpose. Different kinds of transfer are used according to the task to be undertaken: sending pre-recorded files (fixed images) and exchanging real-time or pre-recorded movies (animated images). Both require sending requests between the computers. The transfer of pre-recorded files consists in the transport of images such as PowerPoint slides. To make sure the data has been completely sent, it uses a TCP/IP

protocol. Files are exported in a JPEG form to enable their integration to the interface part. Requests aim at asking for the opening of a socket to send an image using IPC (Inter Processes Communication). Requests do not require much space but are numerous. To make this communication reliable a TCP/IP protocol is used. The transfer of real-time movies needs more network resources than the other transfers. A central switchboard transmits a continuous flow of images to the students’ teaching screens in real-time. TCP/IP protocol cannot be used in a multicast context. Hence it has been decided to use the UDP/IP protocol. As explained previously, projects to configure switchboards especially for movies have been abandoned. With the development of computers, the choice of a packet switching network had to happen. Thus, it has been decided to send JPEG images as packets. This implies some problems to keep a continuous flow. Current practice is a FIFO process (First In First Out). After being compressed, the images are stored in a queue of buffers, waiting to be sent. The images are transported one by one and stored again to be decompressed, Fig.4. As explained previously, images can have different sizes according to the compression quality factor and to their complexity. To facilitate the transfer, it has been decided to send fixed size packets allowing to hold one image. The process of compression and decompression and the transfer process are therefore, clearly separated. The transport of images is made in an asynchronous way whereas the compression and decompression has to be carried out in a synchronous way. Several threads synchronize the buffers used by the process. If the packet is lost, the reception table displays the last image successfully received until getting a new image. Nevertheless, it turned out that this approach had an important drawback. Storing images this way can create an significant lag time. This is unacceptable for surgeons’ applications. That is why a new method has to be attempted. A good choice would be to display on the screen only the last image sent and destroy previous images. For this, a cyclic procedure has to be implemented. A fixed number of buffers could contain the images. Each buffer contains one image. The compression process would fill in the buffers one by one and when finished, overwrite the first one, Fig.5. A pointer would contain the address of the last full buffer. The same procedure is used for the decompression process. To avoid access struggle, a number of four buffers would be a good choice.

Compression process

I4 I3 m m a a Queue of g buffers g Computer 1e e 4 3

Decompression process

I2

Transfer

I1 m a g Queue of buffers Computer 2e 1

Fig.4 Current FIFO philosophy

III. SETTING UP OF THE PROTOTYPE … …

… I3

I2

Transfer

I4

Compression process

… I1

Decompression process

Fig.5 Transfer new philosophy

As aforementioned, an experimental prototype is installed at the ENSPS to work out and test theses features. It consists in three computers (PIII 600 MHz, 256 MB RAM, 20 GB HD) running under Linux and networked according to a star topology thanks to a 100 Mbits/s multicast switch, Fig.7. The choice of this operating system was made for its stability, flexibility and cost. IRCAD provided us with CCD video cameras. Several tests have been made concerning the quality of images and the transfer rate of the real-time video.

C. Graphical interface properties The mentor in the laboratory displays all video sources from the 16 experimental tables on the main control screen in order to analyze them or use them for accreditation purposes. The teacher surgeon monitors either one trainee surgeon in particular or all of them in a cyclic way. Through the student computing interface, trainee surgeons visualize all types of information sent by the teacher and call him in case of emergency. According to the surgeons’ requirements, two interfaces have been implemented: the teacher’s with many possibilities and the student’s which is plain. Both are simple to use and do not require any apprenticeship time. The following functionalities have been placed at the teacher disposal: cyclic monitoring of the trainee surgeons’ work, broadcast of a lesson in the form of PowerPoint slides (turned into JPEG images that can be run in a cyclic way), showing of a pre-recorded video, drawing tools to point out bad or good movements in real-time, and request system for students needing help. The teacher’s interface is divided into four parts, Fig.6. On the top left corner, PowerPoint presentation can be displayed. Pre-recorded or real-time movies are shown on the bottom left corner. The choice of the tables to monitor is made from the bottom right corner. On the top right corner, the teacher surgeon can choose to perform a specific action from the former list. The student’s interface is intuitive and simple. The few functionalities suggested to the trainee surgeons are meant for anyone so that the trainee surgeons can focus on their work.

Student

Student

Teacher

Fig.7 Prototype

A. Hardware All the workstations are linked together to a central switchboard using a star network topology. Owing to its standardization and cost, the network protocol used is Ethernet. Switched Ethernet-based systems decrease significantly the rate of collisions. The switch used is an Alcatel LSS 210 Stack which is a 100 base TX (100 Mbits/s) full-duplex switch. It was one of the first multicast switch available. JPEG real-time compression and decompression demand many computing resources. The CPU must therefore be released by hardware compression and/or decompression. Linux Media Labs [6] proposes flexible acquisition cards responding to these features (video camera acquisition frequency of 50 img/s, compression rate from 3.5 to 50.0). LML33 cards are half-duplex cards with a JPEG compression/decompression hardware built-in chip (Zoran ZR36060). The compression is therefore done by the card, the decompression by a homemade program. B. Software

Fig.6 Expert surgeon graphical interface

The programming language used is C for its rapidity and flexibility regarding work with memory. The C library selected to implement the interface is SDL (Simple Direct mediaLayer) for its simplicity of use and its high performance in events management. This is a graphical programming library, which manages movies and sounds. It owns tools for network programming called SDL_net. This library enables multithread programming but it does not take into account multicast emission and reception. A patch has been added to enable it. GTK (the GIMP ToolKit) is also used.

C. Our student specifications The project lies within the scope of a second year of an engineering school. Every year, students are working on the project for several months. It is very rewarding for a group of students to work on such a project, combining industrial and economical realities with research possibilities. To trace the history of this 5-year project a website [7] has been set up as of the beginning of this project. Its primary goal is to collect all the information regarding the work already achieved and to make it available to everyone. Many files (reports, photos, videos) can be downloaded from its different parts. It also has to be seen as an exchanged platform for all the intervening parties in this project: students, supervisors, clients. Minutes of key steps and proceedings are being displayed within a logbook. Schedulers for the coming weeks as well as all the exchanged mails are also available. IV. POTENTIAL MARKET EVOLUTION A. Cost The prototype used by the students’ team consists of 3 computers equipped with a LML33 card, 1 multicast switch and 2 CCD video cameras. This costs about 6,000 €. The teleteaching room consists of 1 multicast switch and 17 workstations i.e. 17 computers each of them equipped with a LML33 card and an endoscopic video camera. The cost of the most up to date digital laparoscopic video system is 50,000 €. The total cost of the teaching and training laboratory is estimated at about 875,000 €. B. Potential market evolution At the present time, training is limited by the availability of training structures and of teaching experts. This project will help surgeons perform design, and adapt training programs. It will optimize training sessions by decreasing training time and increasing the profitability of the time spent on the guinea pig. Access to a multimedia dedicated training laboratory will reduce the training costs and the loss of time away from their work place. Economical consequences can also be evaluated by analyzing the potential market for the developed devices. Although essentially dedicated to the medical market, the project hardware (computers, switches, video cameras…) is not specific to the medical world and can also be used for many other business communication systems such as communication, surveillance, broadcast, and video games. V. CONCLUSION Today, minimal access surgery is taking over traditional surgery [8]. MAS surgical technologies will be quickly developed over the next decade. According to the evolution of technical know-how, easy access to high-quality training is required. Complete integrated multimedia training system does not currently exist on the market. Furthermore no public hospitals can nowadays afford an entire telesurgical room. With the development of such IP-based technologies, this will be possible. As far as training is concerned, there will be no difference between centers heavily equipped in

audiovisual devices such as IRCAD, and centers with smaller evolutionary systems such as small hospitals or clinics. Improvements still have to be performed before obtaining an optimal training room available to anyone, especially concerning the network. Transfer of voice, realtime video storage and retrieval through the setting up of a multimedia library database could be added to the application. Such a dedicated data server including multiple media diffusion and the possibility of interaction including sound, live, and demonstration videos, graphic text and multimedia animation could be provided through several sets of applications and tools [9]. The first would be a general communication feature including private switching function and videos (such as videoconferencing) complemented by new applications and services such as distant video supervision, documents handling, multi-media library, and Internet access. The second would include applications related to the operating theater control. It would integrate many operating tables controls into one homogeneous user interface and allow the management of all video flow switching across the training room. VI. ACKNOWLEDGMENTS The authors would like to thank J. Chrétien, PhD, external affairs director at Alcatel, D. Mutter, MD, PhD and J. Marescaux, MD, FRCS for their constructive comments and valuable suggestions. VII. REFERENCES [1]

B.Malassagne, D.Mutter, J.Leroy, M.Smith, L.Soler, J.Marescaux, “Teleeducation in Surgery: European Institute for TeleSurgery Experience”, World J ournal Surg ery, vol.25, no.11, November, 2001, pp.1490-1494.

[2]

IRCAD/EITS: http://www.ircad.org

[3]

J.Chrétien, “The MASTER project: ATM in the surgical ward”, Alcatel Telecom Revi ew, Special Telecom 95 issue, 1995, pp.82-88.

[4]

J.Marescaux, J.Leroy, M.Gagner, F.Rubino, D.Mutter, M.Vix, SE.Butner, MK.Smith, “Transatlantic Robot-Assisted Telesurgery”, Nature , vol.413, September, 2001, pp.379-408.

[5]

J.Marescaux, J.Leroy, F.Rubino, M.Vix, M.Simone, D.Mutter, “Transcontinental Robot Assisted Remote Telesurgery: Feasibility and Potential Applications”, Annals of Surgery, vol..235, April, 2002, pp.487-492.

[6]

Linux Media Lab: http://linuxmedialabs.com

[7]

Project: http://www-ensps.u-strasbg.fr/projetircad/

[8]

J.Marescaux, D.Mutter, ”Inventing in the future in surgery”, Minimal Invas ive Therapy allied Techno logies, no.7, 1998, pp.69-70.

[9]

Web Surgery: http://www.websurg.com