Natural interactions for medical image analysis

Interaction-Design Master Thesis

Posts Tagged ‘tangible

3D pointing and drawing

leave a comment »

This is a prototype for pointing and drawing in a 3D space. I think this could be helpful for annotating and marking 3D body scans.

I used a simple color tracker to detect the pink top of the pen. 1 webcam is tracking x and y position, the 2. webcam is tracking the z position of the pink-pen-top. But you could also imagine this working with tracking a finger or even multiple fingers.

I think it is a very interesting to make these 3D interface explorations… but how reasonable are they for the radiologists work? At this stage of the 3D concepts I still see a lot of problems.

Written by Jannes

March 20, 2009 at 17:33

physical cube to view/manipulate a 3D model – v0.2

leave a comment »

In my last meeting with my mentor Mine we discussed the idea of the physical-cube-interface. Mine had a good point by mentioning the difference between looking at a 3D model on your own to the situation in a video conference. I think the augmented reality approach (in which you see the actual video stream with your own hands and the cube, augmented with the digital 3D model) is great for discussing a case online in a video conference, but doesn’t make much sense while analyzing a case on your own. If you are on your own, you don’t really need to see your own hands on the screen.

So I think there should be 2 concepts – one for working alone and another one for discussing and showing a case online.

In the second version of my physical-cube-interface prototype you can not see the video stream. I tried to make the tracking more stable and also implemented some of my initial ideas for viewing actual radiology data. This is still a very early prototype but I have the feeling that I am on the right track here. While playing around with the prototype I become more familiar with the advantages and problems. It already feels really natural to view a 3D model. But I think the other ideas I’ve implemented (pressure sensor, move cube left/right to change transparency) need some more consideration. At this stage there is too much things happening at the same time.

Written by Jannes

March 10, 2009 at 10:41

Tacticle “Displays” ?

leave a comment »

What does a radiologist do to locate and identify a tumor? He is looking at pictures and compares them to find known patterns or shapes in the images that are related to the tumor. But there is a lot more attributes to a tumor that would identify it. In mammography it is the first thing to do, touch the breast and get tacticle feedback about lumps. What if there were a sophisticated “tactile display”? So we could actually feel tissue with our hands to look for patterns/shapes. Hands are super sensitive, try to think how easy it is to feel a grain of sand or hair. It could give us additional feedback about softness/hardness, texture, surface, shape, composition…

This got me starting to think about other features that are not considered in radiology yet:
color, temperature and smell of tissue.


picture resources:
Foam Display
Nasa – Touch the invisble sky

further reading:
Wikipedia – Braille Display (display for the blind)

Written by Jannes

March 6, 2009 at 11:15

Microsoft Surface for medical applications

leave a comment »

I found two medical related projects for the Microsoft Surface table.

This one was shown at CES 2009. It’s a tool for medical education. It seems like a lot of revolutionary projects in the medical field are done for education (see my post about HeartWorks and the VisualBody). I guess its because real medical applications have to follow a lot of restrictions and need to be carefully tested.

Like HeartWorks, this project also utilizes tangible interactive objects. In this case its a physical model of a brain that can be placed on the table to enrich the brain model with digital data. Of course they use multitouch, all kinds of online communications and information from the cloud.  Also they show a flexible display. That’s cool, but doens’t really make sense.

VitraView by InterKnowlogy is the second medical project for Microsoft Surface I’ve found.

Read the rest of this entry »

Written by Jannes

January 30, 2009 at 16:41

Anatomy Education Applications (and Illustrations)

leave a comment »

Anatomy illustrations are widely used in medical education and as far as I know Radiologists use them to identify diseases. Looking at medical illustrations and education practices is interesting for this project. In medical illustrations people thought about how they illustrate the human body to show specific important things they want to tell you about. In my opinion a goal of the development of medical scans is to reach the quality of medical illustrations (generated by a computer). This would probably make the work of Radiologists much easier. There is a ton of information about medical illustrations out there…  So I close the research on this topic for now.

A thought that came to my mind while looking at all the fascinating illustrations: What if medical scans would have such a high resolution that they could display individual cells or even atoms?


Medi-Mation – Medical & Scientific Visualization

Following you’ll find 3 interactive educational tools for human anatomy. They show interesting ways how to visualize the human body and also some nice interface ideas how to view it.

Read the rest of this entry »

Written by Jannes

January 28, 2009 at 11:10

Visualizing Time and Movement

with one comment

Body-changes over time (like muscle movement, blood flow, brain activity) could be interesting for Radiologists to see. Todays medical scanners are able to track these changes and produce high resolution 4D images. But what is an appropriate way to view and analyse this data. A 3D model in which you can modify the timeline and the viewing direction is the most simpel way to view these images. But are there other smarter solutions and how do we combine images from different scan sessions, scans that are maybe days, weeks or years apart. With upcoming electronic and linked patient records this could be possible. (I need to check if this is needed at all.)

Recreating Movement
is a diploma thesis by Martin Hilpoltsteiner at the University of Applied Sciences Wuerzburg, Germany, Communication Arts. It has some very unique ways of analyzing movement in videos. The basic idea is to extract all single frames of a movie and arrange these in different ways to make it easier to analyze movements. It also uses coloring and tracking filters to emphasize special features of the movement. In his work Martin gave some examples of analyzing sports movements, and car crash tests. I think it could also be applied to analyze medical video or 4D data.

Sports movement – Here you can see the movement of a baseball player. The background is removed so you can only see the player. In the example videos you can also see comparison of two different movements or the same movement with different filters.


Read the rest of this entry »

Written by Jannes

January 27, 2009 at 14:14