Automated Quantification of Image-Derived Phenotypes and Integration with the Electronic Health Record at Scale in an Academic Biobank

BACKGROUND: Radiographic images obtained during clinical care can yield a tremendous number of quantitative traits that facilitate translational research. Both liver fat and abdominal adipose mass are examples of quantitative traits that are highly relevant to human health and disease and can be quantitated from medical images such as CT scans. The Penn Medicine Biobank has generated genomic and biomarker data from >50,000 participants with consent to access EHR data. Among these patients, there are greater than 160,000 CT scans, representing over 19,000 patients from which these quantitative traits could be derived. However, manual review of imaging data is time-consuming, costly, and can produce variable results. OBJECTIVE: The goal of this research was to develop a fully automated pipeline to quantify hepatic fat and abdominal adipose mass from CT scans that could be run at scale on patients within the Penn Medicine Biobank. METHODS: We developed a fully automated image filtering and analysis pipeline to analyze CT imaging data. Deep learning networks were trained to identify the presence of IV contrast, delineate the borders of the liver and spleen, and detect visceral and subcutaneous fat. To identify CT studies, we queried our biobank of 52,441 patients for all non-contrast chest, abdomen, and all abdomen/pelvis scans and identified 161,748 CT scans from 19,624 patients. All scans were processed in our deep-learning pipeline in less than 96 hours using parallel processing with cloud-computing resources. From the imaging data, we extracted 12 different image-derived phenotypes that were used in association studies with the electronic health record (N>13000). We also performed genetic association studies (N>5000) on our CT-derived measure of liver fat (LF). Liver fat was defined as the difference in attenuation between the spleen and the liver. Receiver operator characteristic analysis of the liver fat metric was conducted by utilizing 135 patients who had both a CT scan as well as a liver biopsy. Finally, we performed principal component analysis to explore the interrelatedness of the image derived metrics in the context of the phenome. RESULTS: Each component of the algorithm was individually validated for accuracy. The first deep learning network (CNN1) functioned to identify IV contrast, and on a testing set of 400 (half with IV contrast), CNN1 classified 399 scans correctly. CNN2 was tasked with identifying the superior and inferior borders of the abdominal cavity and on a testing set of 100 scans was on average within one slice of the superior (1.01±1.11) and inferior (0.70±0.64) borders. Performance of CNN3, which was tasked with labeling liver, spleen, visceral and subcutaneous fat was assessed by computing region-of-interest area overlap ratios (Dice coefficients). Dice coefficients for a validation set of randomly selected CT scans showed high values for liver (0.95±0.02, n=20), spleen (0.92±0.07, n=20), abdominal compartment (0.98±0.01, n=10), subcutaneous fat (1.00±0.00, n=10), and visceral fat (0.99±0.01, n=10). After extracting the liver fat (LF) metric for all patients, LF had a mean of -6.4 ± 9.1 Hounsfield units. Association studies with billing codes in the electronic health record yielded many known associations for LF including with chronic liver disease, diabetes mellitus, obesity, and hypertension. A genetic association study showed significant associations with variants located in PNPLA3 and TM6SF2. ROC analysis using pathology data yielded an AUC value of 0.81 with a balanced cutoff value of -6 HU for LF. CONCLUSION: This paper presents a fully automated AI-based algorithm for the quantification of traits from medical imaging. This high-throughput algorithm was applied to over 160,000 scans to quantify 12 different traits and the results were integrated with phenome, genome, and pathology data in the Penn Medicine Biobank. This process is scalable, fast, and fault-tolerant and can power translational studies when integrated with clinical and genetic data.