Skip to main content

Table 4 Performance metrics demonstrating the discriminative capabilities of the XGBoost-based machine learning prediction model by comparison with the random forest model and the standard of care comparator. Metrics used include AUROC, sensitivity, specificity, PPV, and NPV. The performance metrics are reported at operating points chosen to have the same sensitivity of 0.789 for both the ML prediction model and the standard of care comparator. All metrics include a 95% CI

From: Machine learning determination of applied behavioral analysis treatment plan type

Performance metrics

ML prediction model

Random forest model

Standard of care comparator

AUROC (95% CI)

0.895 (0.808–0.959)

0.826 (0.678–0.951)

0.767 (0.629–0.891)

Sensitivity (95% CI)

0.789 (0.673–0.906)

0.750 (0.615–0.885)

0.789 (0.700–0.878)

Specificity (95% CI)

0.808 (0.740–0.876)

0.824 (0.753–0.894)

0.635 (0.571–0.698)

PPV (95% CI)

0.600 (0.478–0.722)

0.600 (0.464–0.736)

0.441 (0.360–0.522)

NPV (95% CI)

0.913 (0.861–0.965)

0.903 (0.846–0.960)

0.892 (0.843–0.940)

  1. AUROC area under the receiver operator characteristic curve; CI confidence interval; PPV positive predictive value; NPV negative predictive value