Colorectal Liver Metastases Survival Prediction

Organized by cpm.organizing.committee - Current server time: Nov. 21, 2017, 2:45 a.m. UTC

First phase

May 15, 2017, midnight UTC


Competition Ends
Aug. 7, 2017, 11:59 p.m. UTC


Clinical Problem

Colorectal cancer is the second leading cause of cancer-related mortality in the United States. More than 50% of patients with colorectal cancer will develop liver metastases in their lifetime with a dismal <10% surviving past three years. However, with combinations of hepatic resection and systemic and regional therapies, up to 25% of patients with colorectal liver metastases (CRLM) can be cured. A major therapeutic problem in this clinical scenario is that no preoperative biomarker prognostic of hepatic recurrence has been established. Preoperative markers prognostic of hepatic recurrence risk after resection of CRLM would identify a subset of patients at low risk for recurrence; selecting patients who may only need surgery alone. High-risk patients could be specifically targeted to receive adjuvant hepatic arterial infusion (HAI) chemotherapy and/or systemic chemotherapy, therapies known to prevent recurrence. Thus validated prognostic biomarkers of hepatic recurrence are of paramount importance to improving patient survival of this deadly disease.

The Challenge

The challenge focuses on the quantitative assessment of CRLM using a consecutive series of 198 patients undergoing hepatic resection at Memorial Sloan Kettering Cancer Center with detailed clinical histories, > 48 months follow-up, and high-quality CT imaging. The aim of this challenge is to:

  • predict hepatic disease-free survival based on predictors derived from contrast-enhanced liver CT scans and patient clinical variables.

Survival Analysis

Survival analysis is used to analyze data in which the time until the event is of interest. A good overview of survival analysis can be found here. Our challenge is inspired by the Prostate DREAM Challenge subchallenge 1, in which overall survival was predicted from clinical variables. An overview paper for subchallenge 1 has been published at Lancet Oncology. You can find the accepted manuscript here.

Our challenge is different as it utilizes imaging data in addition to clinical variables.

Metric Evaluation

Participants will build a prognostic model to predict hepatic disease free survival (HDFS), defined by “HDFS_months” (months from surgery to first liver recurrence, last followup date, or death) and “HDFS_status” (alive and no liver recurrence versus liver recurrence or death) for patients with colorectal liver metastasis. Participants will provide a global risk prediction score as well as the prediction score for three time points (24, 36, and 60 months). For example, in a linear predictor model, risk score is defined as the exponential of the predicted value. Participants have the opportunity to submit models separately optimized for these time points. All information in the training data may be used for analysis. Models will be designed using the training data sets and evaluated with the independent test dataset.

Statistical evaluation will be performed by the Department of Epidemiology and Biostatistics at Memorial Sloan Kettering Cancer Center, led by Mithat Gönen. Two scoring metrics will be used to predict global risk: concordance index and integrated time-dependent area under ROC (iAUC) (from 6 to 60 months). The time specific models (24, 36, and 60 months) will be scored using AUCs computed with Hung and Chiang’s estimator of cumulative AUC (Hung and Chiang, Canadian Journal of Statistics, 2010), available in the timeROC package in R. See the submission instructions page for details on how to submit your models and submission formats. Results will be compared across participants.

Terms and Conditions

The data collection is a limited access data set. By joining the challenge you agree:

  • not to transmit this data to any third parties until after MICCAI 2017,
  • not to utilize the data for research beyond the scope of this challenge without the written permission of Memorial Sloan Kettering Cancer Center, and
to appropriately acknowledge Memorial Sloan Kettering Cancer Center in any publications or presentations of results derived from participation in this challenge.

Submission Instructions


You can submit your abstract to


Comma-delimited (.csv) data files must be uploaded to the evaluation page on this site. A user account is required for submission - if you don't have one, you can create one following the link on the front page. The following guidelines for submitting result file must be followed.


  1. Results must be uploaded as a comma-delimited (.csv) file.
  2. The first column must contain the subject ID with the column heading "SNO", e.g. "1". This ID is the prefix for all of the image filenames. For example, 1 is the ID from 001_recurrence_preop_Liver.mhd image file.
  3. The second column must contain the global predicted risk of recurrence for the patients with the column heading "GlobalRiskScore".
  4. The 3rd, 4th, and 5th columns must contain the risk of recurrence at specified time points 24 months, 36 months, and 60 months with the column headings "RiskScore24", "RiskScore36", and "RiskScore60", respectively (Example A below).
  5. Alternatively, participants may submit two columns containing subject ID number and global risk. Global risk will be interpreted as equal risk and evaluated accordingly (Example B below).
  6. All cells should have finite numeric values (no NaN or inf or blank).
  7. No additional rows or columns may be present in the .csv file.

Example A

SNO GlobalRiskScore RiskScore24 RiskScore36 RiskScore60
1 0.283704 0.06816 0.097678 0.198776
2 0.832001 0.162897 0.228088 0.427757
... ... ... ... ...

Example B

SNO GlobalRiskScore
1 0.283704
2 0.832001
... ...


  • [1] Wolf PS, Park JO, Bao F, Allen PJ, DeMatteo RP, Fong Y, et al. Preoperative chemotherapy and the risk of hepatotoxicity and morbidity after liver resection for metastatic colorectal cancer: a single institution experience. J Am Coll Surg. 2013;216(1):41-9.
  • [2] Poultsides G A., Bao F, Servais EL, Hernandez-Boussard T, DeMatteo RP, Allen PJ, et al. Pathologic Response to Preoperative Chemotherapy in Colorectal Liver Metastases: Fibrosis, not Necrosis, Predicts Outcome. Ann Surg Oncol. 2012;19(9):2797-804.
  • [3] Fong Y, Fortner J, Sun RL, et al. Clinical score for predicting recurrence after hepatic resection for metastatic colorectal cancer: analysis of 1001 consecutive cases. Ann Surg 1999; 230:309-318; discussion 318-321.
No submissions have been made public!


Start: May 15, 2017, midnight


Start: July 20, 2017, midnight

You must be logged in to participate in competitions.

Sign In
No submissions have been made public!
No submissions have been made public!