Medical Augmented Reality

While collecting the links for my “Printing Human Tissue” post I stumbled upon a blogpost about Augmented Reality for Medicine at MedGadget. I realised that I never explicitly looked for AR in medicine. So here are some interesting links and videos I’ve found:

Developing a New Medical Augmented Reality System (from 1996)

Computer Aided Medical Procedures & Augmented Reality (TU Munich)

UNC Ultrasound/Medical Augmented Reality Research (from 2000)

3 more videos after the link:

UID open house

I managed to connect a HD camera to vvvv. I can use the higher video resolution to track the AR-tags now. So I have a much more stable tracking than with the crappy Creative Webcam I used before. Also auto-focus and automatic light adjustments help a lot.
I made this project just for fun because of the open house at our institute this Sunday. There is actually no real use for it. Also I was a bit disappointed by myself. I wanted to do something much cooler (like a AR-Pong), but I guess my vvvv-skills are not as good as I thought.

Also finally learned how to use the laser-cutter in our institute (thanks Mikko!). Eventually I didn’t end up using it for the open-house-prototype but I will definitely use it for the final design. I love it!

physical cube to view/manipulate a 3D model

I think 3D models of body scans could be the future, or at least take a bigger role than it is today. Introducing a natural interface for viewing these models could help the acceptance amoung radiology professionals. Here is another rough sketch around this idea. It’s an actual working prototype… unfortunately it’s very flickery, but I think it still shows the basic idea of the concept. manipulating a physical object and have a digital model exactly the same way.

Video and Hard/Software Sketching Tests

Because my last two weeks were so slow, I had the feeling I should just start building stuff. And while doing so I realised why this is still my favourite part in the design process – make things (work). Sketching with hard/software and video (still) takes more time than sketching on paper, but I think it triggers more ideas and gets more feedback from other people… also its more fun!

Augmented reality is one direction I’d really like to push this project to. This is a quick and (very) dirty test in video sketching. Basic idea is a 2D image on a interactive table – use a handheld device to look at it in 3D.

I was using After Effects with some keying effects to key out my hand with the mobile device. Mocha motion tracking (for AE) to track the mobile device screen. I imported a 3D model of a brain into AE and tried to match the 3D camera movement from inside AE. Images on the table are exported from Osirix (medical image viewer).

Notcot has a nice collection of augmented reality projects using fiducial markers. These kind of projects are really inspiring and I am sure I could use something similar for a radiology interface. I tried to work with the ARToolKit to make some sketches with this technology, but it is not working as I want it to work (yet).

Here is a first screenshot of my sketch… its tracking the fiducial markers quite well (blue and red squares are virtual), but I can’t get the 3D model (teapot) to react accordingly.

off topic:

Uh yeah… in this screenshot you also see some cool “windows vista hacking”. I had to use a virtual keyboard to make a screenshot because my macbook doesn’t have a “Print” key and Windows Vista Basic doesn’t have the “snipping tool”