PANORAMA Banner
Join

Data Splitsย ๐Ÿ—ƒ๏ธ

Data is sampled into three splits, with the following use-cases:

  • Imaging dataย was released via:

  • Hidden Validation and Tuning Cohort (100 cases):
    Used for aย live, public leaderboardย that enables model selection and tuning, during theย Development Phase.

  • Hidden Testing Cohort (>400 cases):
    Used to determine the top 5 AI algorithms at the end of theย Development Phase. Used to benchmark AI, radiologists, and test all hypotheses at the end of theย Testing Phase. Includes internal testing data (unseen cases from seen centers) and external testing data (unseen cases from an unseen center). A subset of 400 cases from this cohort is used to facilitate the PANORAMA: Reader Study.


Imaging and Clinical Data

The PANORAMA study data set consists of ~3000 cases from three dutch centers (Radboud University Medical Center, University Medical Center Groningen,ย Ziekenhuis Groep Twente), one center in Sweden (Karolinska Institutet), and one center in Norway (Haukeland University Hospital). Institutional review boards ofย all centersย have waived the need for informed patient consent, with respect to the retrospectiveย scientific use of anonymized clinical data in this challenge. All images are contrast-enhanced CT (CECT) scans in the portal-venous phase, as this is the most commonly acquired phase in routine abdominal imaging. All exams are from patients undergoing CECT without a history of pancreatic cancer treatment, and without any prior positive PDAC histopathology findings. All cases also contain the clinical parameters: age, sex, study date, and scanner. Missing values are possible if they were not recorded in the original dicom headers.

Reference Standard

The hidden testing cohort has the highest-quality reference standard for all cases, to optimally validate AI and radiologists. As histopathology analysis is the gold standard for PDAC diagnosis confirmation, all positive PDAC cases in the hidden testing cohort will have histopathology ground truth, either through surgical resection or biopsy assessment. For the negative cases, the ground-truth label will be established through histopathology assessment (for cases with non-PDAC pancreatic lesions such as cysts and intraductal mucinous neoplasms) and/or follow-up data. Only patients who do not develop PDAC within 36 months after their initial CT scan will be included as negative cases.

The public training cohort is large and representative to optimally train clinically relevant AI algorithms. Positive cases are confirmed based on histopathology when available, or based on radiology reports and follow-up from clinical routine if no biopsy or resection was performed. Negative cases will be confirmed through clinical reports. For cases deriving from existing publicly available data sets, the corresponding ground-truth will be considered. For cases in the MSD data set, differentiations between PDAC and non-PDAC cases will be made as reported by Suman et al. 2021. All training cases will carry patient/image-level annotations, of which, about half of the cases will also include expert-derived tumor delineations, while the remainder will include AI-derived delineations (based on a re-trained version of the method proposed by Alves, et al. 2022). A subset of 100 scans with the same reference standard of the testing cohort will be used as the hidden validation and tuning cohort.