Chest X-rays are currently the best available method for diagnosing pneumonia, playing a crucial role in clinical care and epidemiological studies. Pneumonia is responsible for more than 1 million hospitalizations and 50,000 deaths per year in the US alone.Read our paper
We train CheXNet on the recently released ChestX-ray14 dataset, which contains 112,120 frontal-view chest X-ray images individually labeled with up to 14 different thoracic diseases, including pneumonia. We use dense connections and batch normalization to make the optimization of such a deep network tractable.
The dataset, released by the NIH, contains 112,120 frontal-view X-ray images of 30,805 unique patients, annotated with up to 14 different thoracic pathology labels using NLP methods on radiology reports. We label images that have pneumonia as one of the annotated pathologies as positive examples and label all other images as negative examples for the pneumonia detection task.
We collected a test set of 420 frontal chest X-rays. Annotations were obtained independently from four practicing radiologists at Stanford University, who were asked to label all 14 pathologies, even though . We then evaluate the performance of an individual radiologist by using the majority vote of the other 3 radiologists as ground truth. Similarly, we evaluate CheXNet using the majority vote of 3 of 4 radiologists, repeated four times to cover all groups of 3.
ChexNet is tested against radiologists on sensitivity (which measures the proportion of positives that are correctly identified as such) and specificity (which measures the proportion of negatives that are correctly identified as such). A single radiologist’s performance is represented by an orange marker, while the average is represented by green. CheXNet outputs the probability of detecting pneumonia in a Chest X-ray, and the blue curve is generated by varying the thresholds used for the classification boundary. The sensitivity-specificity point for each radiologist and for the average lie below the blue curve, signifying that CheXNet is able to detect pneumonia at a level matching or exceeding radiologists.