Trimiti - a real-time stereoscopic vision system for neurosurgery training using surgical microscope

Abstract

In this research work, we propose and realize a real-time stereoscopic vision system for neurosurgery training using a surgical stereoscopic microscope. We connect a pair of high resolution IP cameras to a surgical microscope through a dual port microscope beam splitter and a pair of microscope video adaptors. We make use of the specifically designed chessboard calibration pattern and an open source OpenCV library for system calibration. We perform local calibration to align individual IP camera views with each other, whereas global calibration is carried out to line up local calibrated camera views parallel with the user defined reference coordinate system. We calibrate our stereoscopic system using a single input image of the calibration pattern. We perform real-time stereo image remapping for the comfortable stereoscopic vision using a GPU. A media server is developed and introduced in between the IP camera pair and viewer applications to support multiple stereo image streams as well as to provide one more layer of protection to IP cameras. We make use of a NVIDIA 3D Vision Pro system and a 3D ready 120Hz monitor for active stereoscopic vision. We utilize HDMI-1.4 frame packing for passive stereoscopic vision on a standard 3D-TV. We measure network latency of our stereoscopic system with and without using the media server application, which is within acceptable limits. Finally, we compare NVIDIA 3D Vision Pro based active stereoscopic vision and passive stereoscopic vision using a 3D-TV with their pros and cons.