COVID-CXR: An open source explainable deep CNN model for predicting the presence of COVID-1...

Note from the editors: Towards Data Science is a Medium publication primarily based on the study of data science and machine learning. We are not health professionals or epidemiologists, and the opinions of this article should not be interpreted as professional advice. To learn more about the coronavirus pandemic, you can click here .

COVID-CXR

In hopes of contributing to the global response to the COVID-19 pandemic, myself and Blake VanBerlo are releasing an open source explainable machine learning model, covid-cxr , which successfully predicts the presence of COVID-19 from chest x-rays. This work is coming out of the Artificial Intelligence Research and Innovation Lab at the City of London, Canada. This is a prototype model and not yet a diagnostic tool. It builds a foundation and given more data and clinical expertise this model could have a significant impact on the global fight against COVID-19, especially in rural regions of the world where x-rays are more accessible and quicker to turn around than current testing kit infrastructure using RT-PCR.

COVID-CXR is a deep convolutional neural network which allows for both binary and multi-class classification. The binary classifier was trained on approximately 1,000 COVID-19-negative, and 76 COVID-19-positive chest x-rays. We were cautious to integrate more COVID-19-negative images in fear of creating too large of a class imbalance. Given this relatively small set of training data, we achieved encouraging model metrics on the test set, with an AUC of 0.9633 and a sensitivity — or recall — of 0.875. For a more technical deep dive into the model, check out my colleague Blake’s article .

We had noticed some other researchers over the last week exploring this problem as well, but their proposals had significant drawbacks: a) they lacked explainable AI which enables the model to explain its predictions. This is essential in a health care context to build the trust of clinicians, as well as ensuring the model isn’t picking up on meaningless correlations; b) they were only academic explorations whose code was limited in its extensibility, thus constraining the ability of health care institutions to quickly build off the model and deploy something into production in a clinical setting; or c) utilized multiple data sets which in one way or another, leaked the ground truth to the model and resulted in falsely high model metrics. This library mitigates these issues, by being explainable, extensible and well tested. We also tried to ensure it was modular and well documented to increase the speed at which clinicians and data scientists can build off of and contribute to the model. We will continue to add functionality to the model library in the coming weeks.

Future Development of COVID-CXR

Our next steps with the model include:

1. Conducting an exhaustive model architecture hyperparameter search using well resourced cloud compute infrastructure to find an optimal model that isn’t limited by our computer’s GPU.
2. Continuing to improve the explanations of the model with the aid of a radiologist to conduct feature engineering and other model improvements (currently using local interpretable model-agnostic explanations, LIME ).
3. Collaborating with health care practitioners by executing data sharing agreements between them and the City of London to integrate more data in support of improving model metrics.

Calling for Collaboration

We are inviting all data science and health care practitioners to collaborate with us. As mentioned, this model is a prototype and we require more data and more clinical expertise to scale up the model. If you’re interested in collaborating, contact us at the information below:

Matt Ross,
Manager, Artificial Intelligence,
Information Technology Services,
City Manager’s Office,
The Corporation of the City of London
maross@london.ca

Resources:

Note from the editors: Towards Data Science is a Medium publication primarily based on the study of data science and machine learning. We are not health professionals or epidemiologists, and the opinions of this article should not be interpreted as professional advice. To learn more about the coronavirus pandemic, you can click here .