A statistical model illustrated as a nomogram and deployed in an online risk calculator was found to accurately predict the risk of a positive coronavirus disease 19 (COVID-19) test, according to results of a study published in Chest.
Researchers developed a prospective registry of all patients tested for COVID-19 at all Cleveland Clinic locations in Ohio and Florida to develop and validate a statistical prediction model that can predict an individualized risk of a positive COVID-19 test.
Data from 11,672 patients tested before April 2, 2020, including 818 found to have COVID-19, were used to develop the model. A least absolute shrinkage and selection operator logistic regression algorithm was applied to retain the most predictive features. A 10-fold cross-validation method was used to find the regularization parameter lambda, which gave the minimum mean cross-validated concordance. The final model was first internally validated by assessing the discrimination and calibration with 1000 bootstrap resamples. The model was then externally validated in a temporally and geographically distinct cohort of 2295 patients tested (including 290 COVID-19-positive cases) at Cleveland Clinic hospitals in Florida from April 2, 2020, to April 16, 2020. Following the external validation, the statistical prediction model was presented as a nomogram and made available as an online risk calculator.
Model performance in the development and validation cohorts was very good. The bootstrap-corrected concordance index in the development cohort was 0.863 (95% CI, 0.852-0.874), and the index of prediction accuracy (IPA) using a scaled Brier score was 20.9% (95% CI, 18.1-23.7). The concordance index in the validation cohort was 0.839 (95% CI, 0.817-0.861), and the IPA was 18.7% (95% CI, 13.6-23.9).
Decision curve analysis showed that if the threshold of action were ≤1.3%, the model would be no better than simply assuming all individuals being tested are at high risk. However, once the threshold became >1.3%, using the model to determine who is high risk was preferable.
As confirmed in prior literature, the model predicted that risk factors associated with being at higher risk of developing COVID-19 include male sex, advancing age, exposure to a family member with COVID-19, and poor socioeconomic status. However, a critical finding relevant to the model’s performance included lower risk in individuals who had a pneumococcal polysaccharide or influenza vaccine, or who were taking melatonin, paroxetine, or carvedilol.
The researchers note that the model will need to be recalibrated and refit over time to accommodate an ever-increasing COVID-19 prevalence. The study was “not designed to evaluate the very real issue of healthcare disparities which would require a population based approach for the study of healthcare delivery, beyond the scope of the work presented here,” stated the researchers. “Our conclusions are highly dependent on access to testing sites and doctors orders rather than population-based predictors of positive results,” they concluded.
Disclosure: Alex Milinovich, MS, and Michael W. Kattan, PhD, declared affiliations with the pharmaceutical industry. Please see the original reference for a list of their disclosures.
Jehi L, Ji X, Milinovich A, et al. Individualizing risk prediction for positive COVID-19 testing: results from 11,672 patients [published online June 10, 2020]. Chest. doi:10.1016/j.chest.2020.05.580