Natural interactions for medical image analysis

I’ve updated the thesis report with my reflections on the ideatiom and concept phase. You can download the latest version here:

theses v1.2 – 2009/04/17

This is a prototype for pointing and drawing in a 3D space. I think this could be helpful for annotating and marking 3D body scans.

I used a simple color tracker to detect the pink top of the pen. 1 webcam is tracking x and y position, the 2. webcam is tracking the z position of the pink-pen-top. But you could also imagine this working with tracking a finger or even multiple fingers.

I think it is a very interesting to make these 3D interface explorations… but how reasonable are they for the radiologists work? At this stage of the 3D concepts I still see a lot of problems.

We had our mid-semester review yesterday with all my fellow students, Matt from Tellart, Mattias and Mikael. For my presentation I prepared an overview of my research and also sorted my concepts from the ideation phase. I sorted and categorized 3 main concepts for my mid-semester review – the plan is to choose one of these in the end of this week and start the “design phase”.

This picture is an index of things I’ve done for this project so far:

I got some interesting feedback during and after my presentation. Read on to see the feedback…

Read the rest of this entry »

I’ve worked on my initial concept sketches for interfacing with a healthcare network and combined the 4 different concepts in one more complex concept. I did some flowcharts and mind maps to get an overview of my ideas.

This chart shows the basic workflow of a radiologists (Homescreen with schedule —-> Case viewing and Image analyzing —-> Report). I connected all the information from the network that could be helpful in each of the 3 steps. My main ideas for this still circle around crowd- and data sourcing, social networks, emotional connections and telecommunications.

Based on this and my initial concept sketches I’ve created some simple wireframes of an interface (view the pdf here).

I stumbled upon some interesting projects and interesting links since I closed my research phase (thanks to everyone who pointed me to some of these!). Approaching the end of the Ideation phase I thought it might be time to go through all of these links and highlight the most interesting here.

Read the rest of this entry »

Dicom image viewers have the ability to show scanned images with different filters applied. It helps in situations where the radiologist is interested in only some of the body parts. For example there could be a filter that shows only the bones and another filter that shows only the blood vessels. I was thinking about how to combine these different “layers” of information and came up with this concept of “viewing-lenses”. You can drag these lenses over the picture and see different layers of information.
Imagine a touch table with tangible-sheets like in the Microsoft Second Light project… or maybe is better to be able to change the shape and size of the layers. Another idea that came to my mind was combining the lenses – one that shows the bones, another that shows the blood-vessels, put them over each other to see bones and blood-vessel at the same time.

Here is a quick flash prototype to illustrate the idea:

Taking the whole idea and mixing it with my cube interface concept could also be interesting…

I took my paper-multitouch-sketches and transfered them into a video-sketch today. It’s still a rough concept and not all my ides work out so well, but while editing the video I already got a ton of new ideas how to change and improve the multitouch-concept. Also before the video I was thinking about adding some graphical user interface (GUI), but now I really like the simplicity of the visuals and I think it is really important for the radiologist not to get distracted by the GUI. That could be the real advantage in a touch interface for the radiologist.

In my last meeting with my mentor Mine we discussed the idea of the physical-cube-interface. Mine had a good point by mentioning the difference between looking at a 3D model on your own to the situation in a video conference. I think the augmented reality approach (in which you see the actual video stream with your own hands and the cube, augmented with the digital 3D model) is great for discussing a case online in a video conference, but doesn’t make much sense while analyzing a case on your own. If you are on your own, you don’t really need to see your own hands on the screen.

So I think there should be 2 concepts – one for working alone and another one for discussing and showing a case online.

In the second version of my physical-cube-interface prototype you can not see the video stream. I tried to make the tracking more stable and also implemented some of my initial ideas for viewing actual radiology data. This is still a very early prototype but I have the feeling that I am on the right track here. While playing around with the prototype I become more familiar with the advantages and problems. It already feels really natural to view a 3D model. But I think the other ideas I’ve implemented (pressure sensor, move cube left/right to change transparency) need some more consideration. At this stage there is too much things happening at the same time.

While doing the first storyboards I started thinking of the screen setup. Nowadays radiologists normally use 3 screen – 1 for patient data and schedule and 2 for viewing the images. They use this setup because they need high-resolution screens for viewing the images. High resolution screens will get cheaper in the future so I’d like to combine the 3-screen-setup in one big high-resolution screen (possibly with touch recognition). Here are some concepts of how this could look like:

Jannes: I wonder why there are no specialized tools for radiologists. Your main tools are still mouse and keyboard.
Radiologist: Our main tools are our eyes!

I think I began fantasysing about an Eye Tracking Project since I’ve started my first interaction-design course 2.5 years ago. A main problem was that there is no way for me to prototype it… using eyetracking to control a computer is a very new way of interfacing and it’s hard to just think of it without being able to actually try it out.

Still I have this idea of using eyetracking to enhance the radiologists work. Radiologists watch and compare pictures, they need to espy patterns and abnormalities in these pictures. But for navigating through the pictures they need to use mouse and keyboard.

My simplest concept in this direction is not eyetracking but headtracking. You can move your head to see a 3D model from different perspectives (just like in real life). Johnny Chung Lee has a nice presentation using the wiimote for headtracking.

Something more interesting could be a eyetracking and touch interface combined. Looking at a picture selects it, making a touch gesture at the same time zooms it. Here is a quick video sketch I did for this concept… the concept is not brilliant, but I think it has potential.