Skip to main content

The evaluation will be performed using T1/T2-weighted MR images with manual segmentations of WM, GM and CSF. Automatic segmentations will be evaluated against this manual set – a reference standard, by using various measurements, such as Dice Coefficient (DC), Modified Hausdorff Distance (MHD) and Average Surface Distance (ASD).

Besides these overall measurements (DC, MHD and ASD), we will also have local measurements based on major sulci and gyri.

We will only provide a rank list for each metric. We will not provide an overall rank list since we think “Everyone is a winner.

 

—== Validation Results ==—

RANKTEAMSUBMISSION DATEDSC [CSF]MHD [CSF]ASD [CSF] DSC [GM]MHD [GM]ASD [GM] DSC [WM]MHD [WM]ASD [WM]
SmartDSP15/08/20190.93810.3030.1740.896.2720.4430.8677.5710.526
xflz29/08/20190.9588.810.1150.9275.8260.3070.9096.4140.3481
BIG06/08/20190.95110.60.1390.9098.2750.3830.896.7950.47
MA_SmartDSP06/08/20190.9588.9130.1170.9246.1760.320.9056.6590.37
MASI(baseline)29/08/20190.671
14.4050.912
0.53630.8311.3100.64514.6951.561
long12/08/20190.89810.3690.2900.8497.5950.5480.8178.3350.636
tiantian02/09/20190.91210.7600.2370.85410.6980.5300.8259.4140.628
RB01/09/20190.9579.2010.1190.9195.6940.3290.9036.7780.389
PerceptionComputingLab_HIT02/09/20190.9599.0230.1140.9255.6390.3140.9066.5110.363
nic_vicorob14/09/20190.94010.4040.1670.9065.9350.3850.8878.0900.429
Tao_SMU05/09/20190.9579.2860.1200.92612.5270.3150.90311.0680.357
OxfordIBME11/09/20190.9608.5600.1120.9275.4950.3070.9076.7590.353
Brain_Tech05/09/20190.9618.8730.1080.9285.7240.3000.9117.1140.347
lyh05/09/20190.92212.4770.2110.86722.8070.5010.841311.6660.583
VIPSL08/09/20190.9579.2340.1210.9179.3450.3400.8957.0480.397
UBC00108/09/20190.8779.3700.3330.8689.7360.5470.8198.9640.689
CU_SIAT09/09/20190.9549.1910.1250.9246.4000.3240.9046.8470.367
Climb_Mountains09/09/20190.92710.1170.2010.8667.7360.5100.8337.8360.659
FightAutism07/09/20190.9609.2330.1100.9295.6780.3000.9116.6780.341
SJTU-IMR10/09/20190.9568.9950.1230.9226.3180.3280.9026.9250.381
SLHC_MICCAI15/09/20190.9459.6660.1540.9036.6010.3870.8817.2150.456
WorldSeg13/09/20190.8949.2040.341
0.8847.5210.4420.8567.2310.517
Road12/09/20190.9589.0780.1140.9245.8900.3180.9046.7450.370
QL11111112/09/20190.9599.4840.1140.9265.6010.3070.9087.0280.353
WYF2019DUT13/09/20190.9559.4390.12530.92420.93260.32150.9017.7280.365
Guardian62015/09/20190.9599.0710.1120.9186.1660.3350.8946.9780.397
brain_gen
15/09/20190.956
8.9730.1200.9245.9950.3180.9056.9180.371
trung15/09/20190.9529.2610.1330.9145.9600.3430.8976.6110.392
XMU-smartdsp24/09/20190.956
9.262
0.120
0.924
6.805
0.3190.905
7.0350.365
Legand16/05/20200.9029.4630.2630.8696.3390.5090.8487.1110.576

—== The 1st Round Testing Results ==—

RANKTEAMSUBMISSION DATEDSC [CSF]MHD [CSF]ASD [CSF] DSC [GM]MHD [GM]ASD [GM] DSC [WM]MHD [WM]ASD [WM] 
MA_SmartDSP15/08/20190.82311.2050.5460.7958.4620.6040.8308.1780.640Details
long15/08/20190.788 9.503 0.565 0.7359.140 0.6630.7629.6530.828Details
BIG22/08/20190.810
12.8100.6480.7907.7230.5880.8438.6210.630Details
xflz30/08/20190.82612.1270.5560.8258.1300.5580.8617.3080.522Details
RB02/09/20190.79148.6840.8530.8087.7710.5700.8658.5030.531Details
MASI(baseline)05/09/20190.6718.0131.3320.60720.3751.03330.68110.1531.308Details
nic_vicorob05/09/20190.77713.6040.7900.7518.5230.7910.74910.5071.038Details
lyh05/09/20190.79262.1940.6320.70518.7950.7750.69913.6601.110Details
tiantian07/09/20190.796101.1370.8620.71320.3580.8020.64216.1171.288Details
PerceptionComputingLab_HIT08/09/20190.81012.6440.6300.8148.3260.6050.8448.8250.607Details
SmartDSP09/09/20190.81311.5090.5750.7888.2840.6390.8158.5360.682Details
Tao_SMU10/09/20190.83511.9200.5260.83013.0510.5430.8608.1730.514Details
UBC00111/09/20190.6999.8060.8380.6609.8860.9610.63513.8651.693Details
OxfordIBME11/09/20190.80612.1580.6140.7788.6940.6210.8289.5250.644Details
CU_SIAT14/09/20190.82413.6010.6240.8167.8240.5730.8568.2700.584Details
FightAutism12/09/20190.82612.7600.5710.8297.3060.5610.8638.1090.523Details
QL11111114/09/20190.83412.3090.5530.8357.8190.5120.8778.0220.474Details
SJTU-IMT13/09/20190.82111.6780.5520.8127.8700.5820.842
9.2930.6184Details
Road14/09/20190.80312.7220.6580.8018.2960.6590.8269.1310.653Details
brain_gen14/09/20190.82312.4510.5750.8007.8600.5970.8418.3370.624Details
WYF2019DUT14/09/20190.82914.7920.5460.82617.0020.5680.8508.6520.547Details
SLHC_MICCAI15/09/20190.82012.7380.6070.7718.9110.6220.8169.3150.728Details
Brain_Tech15/09/20190.79511.4210.6260.6949.5160.7350.7809.2370.886Details
WorldSeg15/09/20190.80213.6970.7010.8218.3460.5890.8547.5490.540Details
Climb_Mountains15/09/20190.80712.6140.6180.7488.1880.6950.7679.8430.904Details
Guardian62017/09/20190.81411.5570.5870.7788.7590.67160.7838.6920.7019Details
VIPSL20/09/20190.81811.1550.5590.8028.2740.5990.8337.6740.594Details

 

—== The 2nd Round Testing Results ==—

RANKTEAMSUBMISSION DATEDSC [CSF]MHD [CSF]ASD [CSF] DSC [GM]MHD [GM]ASD [GM] DSC [WM]MHD [WM]ASD [WM] 
BIG22/08/20190.81012.8060.6480.7907.7230.5880.843 8.6210.630Details
brain_gen14/09/20190.823 12.4510.5750.8007.8600.5970.8418.3370.624Details
Brain_Tech10/11/20190.81512.9000.618 0.8048.3310.6020.843 8.5030.603Details
Climb_Mountains15/09/20190.807 12.6140.6180.748 8.1880.6950.767 9.8430.904Details
CU_SIAT15/11/20190.829 13.6790.6100.826 7.535 0.5660.8558.639 0.561Details
emc_Brain18/11/20190.765 12.5580.7540.724 8.4500.908 0.643 14.367 1.568Details
FightAutism12/09/20190.82612.7600.5710.829 7.3060.5610.863 8.109 0.523Details
Guardian62017/09/20190.81411.5570.5870.778 8.759 0.6720.783 8.692 0.702Details
IR207/08/20200.77511.8830.6960.7498.3580.7040.7678.7550.804Details
Legand07/07/20200.79811.6270.6110.7798.1310.6550.8068.7520.736Details
long15/08/20190.7889.5030.5650.735 9.1400.6630.762 9.6530.828 Details
lyh05/09/20190.792 62.1940.6320.705 18.7950.7750.699 13.6601.110Details
MA_SmartDSP15/08/20190.823
11.205 0.5460.795 8.462 0.6040.830 8.178 0.640Details
MASI(baseline)14/11/20190.713 13.177 0.7970.709 8.3070.7670.720 9.8021.093Details
nic_vicorob05/09/20190.777 13.6040.7900.751 8.5230.791 0.749 10.507 1.038Details
OxfordIBME11/09/20190.806 12.1580.614 0.778 8.694 0.621 0.828 9.5250.644 Details
PerceptionComputingLab_HIT08/09/20190.810 12.644 0.630 0.814 8.326 0.6050.844 8.825 0.607Details
QL11111114/09/20190.834 12.309 0.5530.835 7.819 0.512 0.877 8.022 0.474Details
RB07/11/20190.807 13.728 0.699 0.814 7.8340.5710.8608.500 0.553Details
Road14/09/20190.803 12.722 0.6580.801 8.297 0.6590.826 9.131 0.653Details
SISE15/11/20190.823 11.541 0.5220.813 8.5770.5690.843 8.5780.564Details
SJTU-IMT13/09/20190.821 11.678 0.552 0.812 7.8700.5820.842 9.2930.618Details
SLHC_MICCAI15/09/20190.820 12.7380.607 0.771 8.911 0.6220.816 9.315 0.728Details
SmartDSP15/11/20190.829 12.387 0.559 0.831 7.898 0.5540.864 8.915 0.522Details
Tao_SMU15/11/20190.834 12.197 0.5400.832 8.143 0.5440.864 8.598 0.508Details
tiantian07/09/20190.796 101.137 0.8620.713 20.358 0.8020.642 16.117 1.288Details
trung15/11/20190.807 13.483 0.684 0.823 7.599 0.5710.860 8.466 0.533Details
UBC00111/09/20190.699 9.806 0.8380.660 9.886 0.961 0.635 13.865 1.693Details
VIPSL20/09/20190.818 11.155 0.559 0.802 8.274 0.5990.832 7.674 0.594Details
WorldSeg15/09/20190.802 13.697 0.7010.821 8.346 0.5890.854 7.549 0.540Details
WYF2019DUT14/09/20190.829 14.792 0.5460.826 17.002 0.5680.850 8.652 0.547Details
xflz30/08/20190.826 12.127 0.5560.825 8.130 0.5580.861 7.308 0.522Details