ImageCLEF 2007 - Medical Automatic Annotation Task

Retrieval tasks

You might also be interested in

• ImageCLEF main site
• Medical automatic annotation task of ImageCLEF 2006
• Results of the Medical automatic annotation task of ImageCLEF 2006
• Non-medical automatic annotation task of ImageCLEF 2006
• The website of the medical automatic annotation task of ImageCLEF 2005
• The results of the medical automatic annotation task of ImageCLEF 2005
• ImageCLEF FAQ

Automatic image annotation

Automatic image annotation or image classification can be an important step when searching for images from a database. Base on the IRMA project a database of 11,000 fully classified radiographs taken randomly from medical routine is made available and can be used to train a classification system. 1,000 radiographs for which classification labels are not available to the participants have to be classified. The aim is to find out how well current techniques can identify image modality, body orientation, body region, and biological system examined based on the images. The results of the classification step can be used for multilingual image annotations as well as for DICOM header corrections.

The classification task this year will consider the complete IRMA code and it will be up to the groups to what level of detail the images will be annotated. Therefore, errors in the annotation will be counted depending on the depth in the tree, and the difficulty of the choice.

Database & Download

Training set and extended Training set now available:

The training data and the extended training data (dev set for tuning the parameters) is now available.

Downloads (you will need the username/password that is provided when registering for ImageCLEF 2007):

Note: if you are not able to download the data using the links directly you have to go to the "Information on the datasets"-page, login using the credentials that are provided upon registration for ImageCLEF and then you can retry the links or navigate to on the IRMA page to the downloads.

• Information on the datasets
• training data
• development data
• codes of all 11,000 images
• Transfer agreement (not necessary for participants in ImageCLEF 07)
• test data

RESULTS

• Results are now available

Error counting scheme

In ImageCLEF 2007, the medical automatic annotation task considers the complete IRMA code and penalizes misclassifications at different levels of the code differently.

A detailed description of the error counting scheme is here

The submission format will be

<imageno> <imagecode>
<imageno> <imagecode>
...

2034 1121-127-700-500
2229 1121-110-411-700
2630 1121-120-942-700
2633 1121-120-951-700
2711 1121-120-921-700
...

For assistance, we offer some tools:

• a python tool that does error calculation in the manner described above on files of this file format.
  It requires
  • a file describing the complete code
  • a file with the correct code for the 11000 training images
  if started without any options it will give the errors for some codes which are encoded in the program itself, if started with a submission file as parameter (submission file for a subset of the training set, ideally, dev-set) it will print the error score for this file.
• a tool that converts submission files from 2006 to the format for 2007.
  it requires
  • a file with the correct code for the 11000 training images (same as above)
  • a file with the correct classes of 2006 for every image

Schedule

• release of training data: April, 5, 2007
• release of test data: June 15, 2007
• submissions of results: July 15, 2007
• release of results: ~August 1, 2007

Submission of Results

• Contact person
  • Name, Email, Contact address
• Group
  • Group identifier (for presentation of results)
  • Complete name of the group
  • Address
• Run
  • Identifier (for presentation of results)
  • description (200-500 words)
  • file with classification results following the format explained below
  • do you consider this run your primary run?

Submission format

Publications

The most interesting and successful submissions to this task are invited to submit a paper to a special issue in Pattern Recognition Letters :

Automatic annotation of medical images for image retrieval -- ImageCLEF 2007
Guest Editors: Thomas Deselaers, Henning Müller, and Thomas M. Lehmann

Questions & Comments

• Thomas Deselaers for technical questions concerning the data transfer and evaluation.
• Thomas Deserno for general questions concerning the IRMA code and/or IRMA database.

;