Natural interactions for medical image analysis

Interaction-Design Master Thesis

Archive for the ‘Phase 01 – Research’ Category

Medical Augmented Reality

leave a comment »

While collecting the links for my “Printing Human Tissue” post I stumbled upon a blogpost about Augmented Reality for Medicine at MedGadget. I realised that I never explicitly looked for AR in medicine. So here are some interesting links and videos I’ve found:

Developing a New Medical Augmented Reality System (from 1996)

Computer Aided Medical Procedures & Augmented Reality (TU Munich)

UNC Ultrasound/Medical Augmented Reality Research (from 2000)

3 more videos after the link:

Read the rest of this entry »


Written by Jannes

May 5, 2009 at 22:53

Printing Human Tissue

with one comment

I was thinking about printing out an actual scan of a deformed child’s skull in real size in 3D. Its part of my augmented 3D interface for radiologists to report and discuss their findings. Unfortunately its still very expensive to print in 3D, but I was messing around with the generated 3D model a bit…. slicing it up was a great idea, because i am saving a lot of support material during the print. So I am down to 2000 sek , thats about 200 euros (from originally 10000 sek).

I think 3D printing will become very cheap and quite common in the near future. Having your personal 3D printer at home and printing at prices similar to 2D paper prints will definitely be around at some point. Therefor I think its not a big deal to suggest 3D printouts for some cases in the radiology department. Especially since we are scanning 3 dimensional bodies, its only natural to look at them in 3D and also touch them. Touching could especially be interesting if the 3D printout is actually a print of human tissue, and so behaves and feels exactly like the part in the patients body. Imagine the radiologist can print out the tumor he/she identified and give it to the surgeon. The surgeon can see and feel the tumor and use it as reference during the surgery – this idea was inspired by an actual fact during some surgeries. There are some surgeries nowadays in which the removed tissue is send back to the radiology department, so the radiologist can scan and compare it to the initial body scan and see if everything was removed as planned.


Printing human tissue has already been done. There is a lot of examples in medicine research to grow bones and living organs to replace damaged body parts in human bodies. Also there are some examples in the arts using “artifical” human tissue. See yourself:

Individually Manufactured Replacement Bones in Clinical Trial (medGadget)
A new way to print bones
Thumbs up for 3D bone printer
Print me a heart and a set of arteries
Printing Organs on Demand
Tiny Doll made of living cells

Biojewellery (jewellery made by growing bone from your own DNA)

Thanks to Matt and Mikko, both their projects here at Umea Institute of Design are really inspiring for this concept.


3D-Doctor a 3D imaging software is already using normal 3D printers to print out models of bones from body scans. Its not real bone tissue… but still very interesting.
Advanced Custom Made Implants
uses the same technology to print custom made implants, but also no real bone tissue.

and here is an interesting article about reproducing the natures complex internal bone structure to produce strong artifical bones.

Written by Jannes

May 5, 2009 at 21:51

3D screen concept

leave a comment »

I was wondering how we could display 3D images. There is a couple of different technologies to do it.

Caveman is virtual environment that is also used to view anatomic 3D models. of course there is also projects about stereopscopic 3D viewing (with fancy 3D glasses) for radiology.


I am not sure not about these technologies. I don’t like wearing weird glasses and also whenever I used these 3D glasses it stressed my eyes a lot. The radiologist is working with device all day long and also its annoying to give glasses to everybody whenever you want to show something. The Caveman virtual environment is cool, but doesn’t fit into the all-day workflow. It uses a whole room and also you need special headsets.

Here is an idea I had about viewing 3D images (or at least 3D slices). It’s 3 screens arranged to form a corner of a cube/box. So each screen is displaying one of the 3 planes of a 3D system. The second image is mixed with my idea of augmented reality, the brain is a physical model… its position is tracked, so you could rotate and move it to change the projection in the 3 screens.


Written by Jannes

March 5, 2009 at 12:18

week 6 summary – research report 1.0

leave a comment »


Here is the first version of my research report. I just put together all the findings from the past weeks. So if you were reading this blog, than there is probably nothing new in the report for you. It’s just the first part of my official Thesis-report. Also its not finished, there is still a few blank pages and content might change later on. I’ve created a static page for the report on the right side of this blog, you’ll always find the latest version there too.

download: research report v1.0 – 2009/03/01


Written by Jannes

March 1, 2009 at 10:34

week 4 & 5 summary

leave a comment »

Week 4 & 5 have been a bit slow for me to be honest. I am not sure why I did so few progress in these weeks… I think one part of it was that I moved to Sweden. You can clearly see that I haven’t done to much progress on my blog (lack of blog posts) – this blog gives me no chance to hide. Anyway…

I visited the Umea hospital in week 5 and it was really interesting to see the differences to the Netherlands (see last post). Sweden is a bit ahead in the healtcare networking thing.


Also I started in my Ideation phase in week 4 with a brainstorm at the Philips Design office.


Written by Jannes

February 26, 2009 at 15:45

Norrlands University Hospital, Umea, Sweden

leave a comment »

The basic workflow at the radiology department at NUH is basically the same as described earlier for UMC Utrecht.
However there are some minor differences between the two hospitals. It seems like NUH Umea and healthcare network in Sweden in general is a bit ahead of other countries.
At UMC Utrecht radiology requests forms are still written on paper and than scanned and digitalized. Because some of their systems is already upgraded to digital networked technology and some is not, they have this weird mix of paper and digital data.
This is different in Sweden, here they nearly completed the transfer to computer technology. Requests to the radiology department are send and edited in an email-like hopsital internal system.
Sweden also has a nation-wide healthcare network called Sjunet. All hospitals are connected to this network and can share information. This could be patient history and pictures from previous body scans.
In my “directions: 4 – prognosis” I pointed out the trend to more prognosis instead of diagnosis-only at the radiology department. At NUH Umea there is a designated conference room for this purpose. Radiologist present specific cases to a group of specialists. Together they will discuss the case and propose a treatment for the patient. The whole system is designed for these rounds. Radiologists can choose rare, difficult or interesting cases and schedule them for a round. They just drag-and-drop the cases to a digital calender. Some cases are automatically scheduled for rounds by the computer according to a protocol defined by the hospital.


Written by Jannes

February 26, 2009 at 15:42

week 3 summary – “Phase 02 – Ideation” directions

leave a comment »

I had the chance to speak to radiology-professionals for the first time last week. I analyzed their workflow and collected their thoughts and ideas. I tried to combine this with my research from the past 2 weeks. I pointed out problems in the work of radiologists and trends of radiology. This lead to 4 general topics I’d like base my ideation upon. I am going to use these directions for first brainstorm sessions at the Philips Design Department this week.

“Phase 02 – Ideation” directions:

1. Data Explosion
Technology is changing fast. New ways for scanning bodies are developed. Different technologies are combined (multi-modality) to get an even more detailed pictures of the body. A brain-scan can have up to 20000 pictures. Also patient history becomes easier accessible, scans from the past can be included in the analysis.
This is a lot of data to view for the radiologist, there is a need for specialized tools to be able to see what is important in all this data. data-visualizations should be an important part of these tools. radiologists are not dependent on viewing 2D slices anymore.

2. Comparing Cases (connected data in a Healthcare 2.0 world)
Nowadays radiologists use physical books, medical atlases to compare the body scans with examples of known diseases. But the trend goes to digital data, there are more possibilities for visualizing and comparing diseases than with a book. Patient history and information becomes easily accessible and can be incorporated into the diagnosis. Radiologists also like to collect their own special cases for future references. PACS is already the first step to centralize these collections and patient data, what is the next step?

3. Reporting for who?
Reports are normally generated automatically according to the protocol. But who views these reports? Different specialists need different information. New technology delivers more options of showing information (e.g. video and 3D). In the healthcare network of the future even patients could be able to access their own scans, but they don’t know how to read this information. A big advantage of the connected world is the ease of sharing knowledge, the system should be designed for this.

4.  Prognosis in the Radiology Department
Today a radiologist has about 3-5 minutes to view and analyze a case and make a diagnosis. Officially its not his/her job to make a prognosis (e.g. propose what treatment is needed). But fast developing technologies in the radiology field would make prognosis possible in the radiology department. And there is already a gray area: radiologists and surgeons work together to analyze images and decide on a treatment. Is this the radiology of the future – high specialized physicians working together to create a prognosis, maybe even in the radiology reading room? how should a reading room be designed to accomplish a seamless workflow for the individual specialists? what is the role of CAD and teleradiology in making prognosises?

Written by Jannes

February 9, 2009 at 10:50