(Darmstadt) Researchers at the Fraunhofer Institute for Computer Graphics Research IGD in Darmstadt have developed and process that combines medical ultrasound with augmented reality (AR). “AR ultrasound” will make it possible for the first time for doctors to view mapped ultrasound images with the help of AR glasses, allowing them to look directly at the patient and not have to spend the entire time looking at the ultrasound monitor.
 

Various possible applications

The technology will improve the success rate of ultrasound-guided biopsies in the future. “The sectional planes through the body appear directly over the represented structures on the patient, allowing the doctor to see the needle in the correct position and adjust the puncture channel more easily,” explained Matthias Noll, deputy head of the Visual Healthcare Technologies Competence Center at Fraunhofer IGD. The safety of surgical procedures will also increase in that surgeons can use the new technology to plan the procedures in more detail. Soon, even physical anomalies or tissue changes will be directly visible in the AR image through techniques such as color-highlighting.
 

Precise positioning with Fraunhofer software

The topographical representation of ultrasound images is based on an external optical ultrasound probe/AR glasses tracking system. Fraunhofer IGD’s software analyzes their positions and uses various calibrations to compute the positions of both objects in relation to one another. The ultrasound image is then displayed in the examiner’s field of view on the AR glasses.
 

Hospital trials

In the next three months, AR ultrasound will be tested in a hospital setting. “In order be able to tap into the ultrasound machine’s signal, we need an interface with the ultrasound software or we use the monitor cable,” said Noll. The only other requirements are a conventional ultrasound machine, the AR glasses, the tracking system and a laptop to make the AR calculations.
 

From the visual computing experts

AR visualization of ultrasound images is the latest development from Fraunhofer IGD, a world leader in applied visual computing research. The institute has been developing new ultrasound applications since the turn of the millennium. Another of Fraunhofer IGD’s emphases is AR, a technology that has been increasing in importance in recent years. Most recently, developers have created a process that uses AR to help surgeons better and more quickly localize metastatic lymph nodes.