home page

ChIR : Surgery, Computer Science and Robotics

A U G M E N T E D   R E A L I T Y


The challenge consists in guiding the surgeon's gesture by introducing preoperative results in the operating theatre.

  • The best incision sites are computed during the planning of the intervention. They can be reprojected on the patient after the external registration with the preoperative acquisitions.
  • During the intervention, the field of view delivered by the endoscope is limited. In the cardiac context, superimposing cartographic informations in endoscopic images would allow the surgeon to find his targets.

External registration and incision sites reprojection

patient The patient is on the table, in the operating room. A scan (resp. MRI) of the patient is done before surgery. MRI
reconstruction Just before surgery, the patient's torso is reconstructed in 3D by a stereo-based vision system. The skin has been segmented from the patient's scan, on which the 3 incision sites have been determined. scan
The segmented scan and the 3D reconstruction are registered by an ICP algorithm.
The incision sites can then be reprojected on the patient's skin.

Intraoperative registration with the preoperative models

The superimposition of preoperative information in the endoscopic images requires to solve the problem of registration between the preoperative acquisitions and the intraoperative scene. The external registration of the patient is one step but the difficulty comes from the displacement of the organs in the operating configuration and their self-motion. The intraoperative information must be used.

3D reconstruction of the operating field

3D reconstruction
  • Surface reconstruction

  • Separation of the instruments

Image overlay

  • Directionnal informations

  • Target artery overlay