MICCAI 2012 Grand Challenge and Workshop on Multi-Atlas Labeling:
--------------------------------------------------------------------------------

CAT12 was used to segment the T1 data and to estimate the registration parameters to transform the probability maps to MNI152NLin2009cAsym space. 

The resulting maximum probability map was slightly modified: (1) remaining holes were filled with median value, (2) an additionally mask with brainmask_T1.nii was applied to remove non-brain areas.
 
Labels in native space were derived from the ''MICCAI 2012 Grand Challenge and Workshop on Multi-Atlas Labeling'' (http://www.neuromorphometrics.com/2012_MICCAI_Challenge_Data.html). These data of 35 subjects were released under the Creative Commons Attribution-NonCommercial (CC BY-NC) with no end date. Users should credit the MRI scans as originating from the OASIS project (http://www.oasis-brains.org/) and the labeled data as "provided by Neuromorphometrics, Inc. (http://Neuromorphometrics.com/) under academic subscription".  These references should be included in all workshop and final publications.


Websites:    
A) http://www.neuromorphometrics.com/2012_MICCAI_Challenge_Data.html
B) http://www.oasis-brains.org/
C) http://Neuromorphometrics.com/


References:



--------------------------------------------------------------------------------

Challenge Details
Please see https://masi.vuse.vanderbilt.edu/workshop2012/index.php/Important_Dates for dates.
Click Watch on the bar above to be notified via e-mail when this page is updated (you must create a free account - see the upper right).

Grand Challenge Rules

Submission Information
See Submission Information for details on how to submit labels.

Contest Data
On approximately April 1, 2012, the following data will be made available after registration (see below). 
Training Atlas Pairs: 15 datasets for distinct human data consisting of a de-faced T1-weighted structural MRI dataset and as associated manually labeled volume with one label per voxel. Each volume (MRI and label) will be stored in a separate 3D NiFTI file. These files will be properly interpreted by the MIPAV software package (freely available). (from the "reliability" OASIS data set: Challenge_Demographics).
Testing Target MRI: 20 T1-weighted structural MRI datasets of 15 distinct subjects. (Challenge_Demographics).
Training and Testing demographics (age, gender handedness) will be available here
A spreadsheet of all considered labels can be found here -- see evaluation procedures for further label information.

Evaluation Procedures
Please see https://masi.vuse.vanderbilt.edu/workshop2012/index.php/Important_Dates for dates.
Proof of paper "preliminary" manuscripts for grand challenge submissions will be accepted. Manuscripts must be in the format indicated below.
After submission of a preliminary manuscript, contestants will be given an access code to upload labeled target MRI datasets through this website. Within 24 hours of upload, the contact author will be e-mailed a PDF of quantitative performance results. The primary metric for the grand challenge contest is the mean Dice similarity coefficient across all brain labels and all subjects in the "testing target" cohort. The list of anatomical labels is here. A discussion of the Statistical Power of the challenge is provided here.

Resubmissions of different label results are permitted within the evaluation window. We understand that technical glitches, format errors, etc. may lead to misleading error metrics. The organizers encourage (and will assist with) multiple submissions to address such errors. The last submission of label results under an access code will be considered the authors final submission.

We strongly advocate against parameter tuning based on the hidden testing labels during the evaluation window. The number of submissions will be recorded on the summary PDF and must be reported in the final publication. Submission of multiple similar algorithms for evaluation is encouraged. Please use a different preliminary manuscript for each algorithm variant. Text may be shared among manuscript submissions.

Additional Evaluation Information 
We only compared against labels that appeared consistently across all of the datasets. As a result, we ignored some of the labels during the calculation of the results. In MATLAB notation, the label numbers that we ignored are:
ignore_labels = [1:3, 5:10, 12:22, 24:29, 33:34, 42:43, 53:54, 63:68, 70, 74, 80:99, 110:111, 126:127, 130:131, 158:159, 188:189];
Anywhere that any of these labels appear on the "true" (manual) labels are automatically set to 0 (background) on both the "true" labels and the estimated labels in order to avoid biasing the results.
After ignoring these labels we are left with 134 labels that count. The resulting DSC values on these labels are what is reported on the results page.
The labels were evaluated in three groups:
1) Cortical Labels: All non-ignored labels >= 100
2) Non-Cortical Labels: All non-ignored labels > 0 and < 100
3) Overall: All non-ignored labels.


Manuscript Preparation
Papers submitted in response to the Grand Challenge should conform to the MICCAI formatting instructions, with the modifications explained below:
Maximum author-generated manuscript length is 4 pages.
The PDF report generated by the evaluation process must be included as a single supplementary final page (for a total of 5 pages).

Submitted in PDF format
Color illustrations in the PDF are not subject to fees.
The preliminary manuscript may be revised up until June 12, 2012.

Validation (Hidden) Data
The hidden true labels will be revealed July 1, 2012 and made available to all authors who submitted a response to the workshop challenge. Additionally, the full source code of the program used to generate the evaluation PDF's will be made public.

Terms of Use for Data Provided During the Contest
The data will be released under the Creative Commons Attribution-NonCommercial (CC BY-NC) with no end date. Users should credit the MRI scans as originating from the OASIS project and the labeled data as "provided by Neuromorphometrics, Inc. (http://Neuromorphometrics.com/) under academic subscription". These references should be included in all workshop and final publications.

Availability of Data after Contest Completion
After the completion of the contest, testing/training data will be available from this website after users agree to the license terms.
After the completion of the contest, please contact Neuromorphometrics (http://neuromorphometrics.com/) for additional details relevant to this dataset.

Eligibility
No individual who has access to the labeled testing datasets is permitted to submit a response to this grand challenge. Specifically, employees, affiliates or contractors for Neuromorphometrics are not permitted to participate in any contest entry.
It is assumed that the majority of entrants will seek to use multi-atlas method to respond to this challenge. However, any method of labeling the testing datasets is permitted as long as the approach is described in a reproducible manner.

Software Sharing
Contestants are encouraged (but not required) to make available the software, tools, and source code for the methods used in response to this challenge. However, software and code sharing are not required for participation.

Contest Results
After the workshop (after October 5, 2012), a summary of all methods will be presented here. We will publish a citable book (with ISBN) with a compilation of all results and a summary of rankings. This book will be available at cost (or no cost, if possible) – neither editors nor their institutions will receive any compensation for this publication. On this website, we will make available copies of all papers, the label masks submitted for each method, the PDF result files for each method, and (if the authors desire) additional resources (software/code/etc.) that a reader may find relevant to each paper.





Challenge Demographics
--------------------------------------------------------------------------------

Training Data
Number	Age	
Total F	10	19	min
Total M	5	23	average
Total	15	34	max
Training Details
Case #	train/test	Training	Subject	Gender	Age
1000	train		OAS1_0061	F	20
1001	train		OAS1_0080	F	25
1002	train		OAS1_0092	M	22
1006	train		OAS1_0145	M	34
1007	train		OAS1_0150	F	20
1008	train		OAS1_0156	F	20
1009	train		OAS1_0191	F	21
1010	train		OAS1_0202	F	23
1011	train		OAS1_0230	F	19
1012	train		OAS1_0236	F	20
1013	train		OAS1_0239	F	29
1014	train		OAS1_0249	F	28
1015	train		OAS1_0285	M	20
1036	train		OAS1_0353	M	22
1017	train		OAS1_0368	M	22
Testing Data
Number	Age	
Total (unique) F	10	18	min
Total (unique) M	5	45.7	average
Total (unique)	16	90	max
Testing Individuals
Case #	train/test	Testing	Subject	Gender	Age	Notes
1003	test		OAS1_0101	M	29	1st Scan
1004	test		OAS1_0111	M	23	1st Scan
1005	test		OAS1_0117	M	25	1st Scan
1018	test		OAS1_0379	F	20	1st Scan
1019	test		OAS1_0395	F	26	1st Scan
1023	test		OAS1_0101	M	29	2nd Scan
1024	test		OAS1_0111	M	23	2nd Scan
1025	test		OAS1_0117	M	25	2nd Scan
1038	test		OAS1_0379	F	20	2nd Scan
1039	test		OAS1_0395	F	26	2nd Scan
1101	test		OAS1_0091	F	18	
1104	test		OAS1_0417	F	30	
1107	test		OAS1_0040	F	38	
1110	test		OAS1_0282	F	45	
1113	test		OAS1_0331	F	54	
1116	test		OAS1_0456	M	61	
1119	test		OAS1_0300	M	68	
1122	test		OAS1_0220	F	75	
1125	test		OAS1_0113	F	83	
1128	test		OAS1_0083	F	90	


--------------------------------------------------------------------------------
$Id$