- August 09, 2018
Cardiogram, an Apple Watch app developer, ran a study on 14,011 of its users in conjunction with researchers at UCSF (University of California/San Francisco). After using data on some of the participants to train a deep neural network, called DeepHeart, the team tested the algorithm on results from the remaining cohort of subjects. The best version of the algorithm recorded a c-statistic of 0.85 in diabetes, making it a strong model.
San Francisco-based Cardiogram is able to identify people with diabetes from heart data because of earlier work that spotted a correlation between variability in cardiovascular activity and the condition.
The study suggests the algorithm is pretty good at identifying people previously diagnosed with diabetes. That ability has limited clinical utility, though. The real prize is alerting undiagnosed patients that they may have the condition so they know to get tested and potentially treated. Cardiogram is working to show it can pull off this trickier task.
In this context, the algorithm’s ability to avoid false alarms is as important as the proportion of cases it accurately diagnoses. Each false positive will cause suffering to the individual and result in blood tests, mobile ECGs and other unnecessary healthcare procedures, wiping out many of the benefits of the algorithm’s accurate diagnoses. Cardiogram is aware of the problem; however, thinks the benefits will outweigh the costs once a certain specificity threshold is passed. “With sufficiently high precision the cost of false positives is exceeded by the cost savings to the healthcare system of each true positive,” researchers at Cardiogram and UCSF wrote in the paper published in February.
The question is whether Cardiogram can tweak the algorithm so it can identify a meaningful number of undiagnosed diabetes patients without dialing the specificity down too far. The curves presented in the study show sensitivity rises above 0.8 at the point specificity falls toward 0.8. Ideally, a test would have a sensitivity of 1 and specificity of 1.
REFERENCE: Fierce BioTech; 08 FEB 2018; Nick Paul Taylor