Review Article| Volume 234, P74-87, August 2021

Assessing opioid overdose risk: a review of clinical prediction models utilizing patient-level data

Published:March 21, 2021DOI:
      Drug, and specifically opioid-related, overdoses remain a major public health problem in the United States. Multiple studies have examined individual risk factors associated with overdose risk, but research developing clinical risk prediction tools for overdose has only emerged in the last few years. We conducted a comprehensive review of the literature on patient-level factors associated with opioid-related overdose risk, with an emphasis on clinical risk prediction models for opioid-related overdose in the United States. Studies that developed and/or validated clinical prediction models were closely reviewed and evaluated to determine the state of the field. We identified 12 studies that reported risk prediction models for opioid-related overdose risk. Published models were developed from a variety of data sources, including Veterans Health Administration data, Medicare data, commercial insurance data, and statewide linked datasets. Studies reported model performance using measures of discrimination, usually at good-to-excellent levels, though they did not always assess calibration. C-statistics were better for models that included clinical predictors (c-statistics: 0.75–0.95) compared to models without them (c-statistics: 0.69–0.82). External validation of models was rare, and we found no studies evaluating implementation of models or risk prediction tools into clinical practice. A common feature of these models was a high rate of false positives, largely because opioid-related overdose is rare in the general population. Thus, efforts to implement prediction models into practice should take into account that published models overestimate overdose risk for many low-risk patients. Future prediction models assessing overdose risk should employ external validation and address model calibration. In order to translate findings from prediction models into clinical public health benefit, future studies should focus on developing clinical prediction tools based on prediction models, implementing these tools into clinical practice, and evaluating the impact of these models on treatment decisions, patient outcomes, and, ultimately, opioid overdose rates.


      CDC (Centers for Disease Control and Prevention), CMS (Center for Medicare and Medicaid Services), ICD-9 (International Classification of Diseases, 9th edition), ICD-10 (International Classification of Diseases, 10th edition), LASSO (least absolute shrinkage and selection operator), PDMP (prescription drug monitoring program), STORM (Stratification Tool for Opioid Risk Management), VHA (Veterans’ Health Administration)
      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to Translational Research
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • Liu S
        • Scholl L
        • Hoots B
        • Seth P.
        Nonfatal drug and polydrug overdoses treated in emergency departments - 29 States, 2018-2019.
        MMWR Morb Mortal Wkly Rep. 2020; 69: 1149-1155
        • Wilson N
        • Kariisa M
        • Seth P
        • Ht Smith
        • Davis NL.
        Drug and opioid-involved overdose deaths - United States, 2017-2018.
        MMWR Morb Mortal Wkly Rep. 2020; 69: 290-297
        • Centers for Disease Control and Prevention
        Vital signs: overdoses of prescription opioid pain relievers—United States,1999–2008.
        MMWR Morb Mortal Wkly Rep. 2011; 60: 1487-1492
        • Guy Jr., GP
        • Zhang K
        • Bohm MK
        • et al.
        Vital signs: changes in opioid prescribing in the United States, 2006-2015.
        MMWR Morb Mortal Wkly Rep. 2017; 66: 697-704
        • Rudd RA
        • Paulozzi LJ
        • Bauer MJ
        • et al.
        Increases in heroin overdose deaths - 28 States, 2010 to 2012.
        MMWR Morb Mortal Wkly Rep. 2014; 63: 849-854
        • Gladden RM
        • Martinez P
        • Seth P.
        Fentanyl law enforcement submissions and increases in synthetic opioid-involved overdose deaths - 27 States, 2013-2014.
        MMWR Morb Mortal Wkly Rep. 2016; 65: 837-843
        • O'Donnell JK
        • Gladden RM
        • Seth P.
        Trends in deaths involving heroin and synthetic opioids excluding methadone, and law enforcement drug product reports, by census region - United States, 2006-2015.
        MMWR Morb Mortal Wkly Rep. 2017; 66: 897-903
        • Gladden RM
        • O'Donnell J
        • Mattson CL
        • Seth P.
        Changes in opioid-involved overdose deaths by opioid type and presence of benzodiazepines, cocaine, and methamphetamine - 25 States, July-December 2017 to January-June 2018.
        MMWR Morb Mortal Wkly Rep. 2019; 68: 737-744
      1. Centers for Disease Control and Prevention. CDC's Opioid Overdose Indicator Support Toolkit: Guidance for building and reporting on opioid-related mortality, morbidity, and PDMP indicators (Versions 3.0). 2018.

        • Seth P
        • Rudd RA
        • Noonan RK
        • Haegerich TM.
        Quantifying the epidemic of prescription opioid overdose deaths.
        Am J Public Health. 2018 Apr; 108: 500-502
        • Dunn KM
        • Saunders KW
        • Rutter CM
        • et al.
        Opioid prescriptions for chronic pain and overdose: a cohort study.
        Ann Intern Med. 2010; 152: 85-92
        • Green CA
        • Perrin NA
        • Janoff SL
        • et al.
        Assessing the accuracy of opioid overdose and poisoning codes in diagnostic information from electronic health records, claims data, and death records.
        Pharmacoepidemiol Drug Saf. 2017; 26: 509-517
        • Friedman J
        • Kim D
        • Schneberk T
        • et al.
        Assessment of racial/ethnic and income disparities in the prescription of opioids and other controlled medications in California.
        JAMA Intern. Med. 2019; 179: 469-476
        • Cerdá M
        • Gaidus A
        • Keyes KM
        • et al.
        Prescription opioid poisoning across urban and rural areas: identifying vulnerable groups and geographic areas.
        Addiction. 2017; 112: 103-112
        • Altekruse SF
        • Cosgrove CM
        • Altekruse WC
        • Jenkins RA
        • Blanco C.
        Socioeconomic risk factors for fatal opioid overdoses in the United States: Findings from the Mortality Disparities in American Communities Study (MDAC).
        PLoS One. 2020; 15e0227966
        • Bohnert AS
        • Valenstein M
        • Bair MJ
        • et al.
        Association between opioid prescribing patterns and opioid overdose-related deaths.
        JAMA. 2011; 305: 1315-1321
        • Campbell CI
        • Bahorik AL
        • VanVeldhuisen P
        • et al.
        Use of a prescription opioid registry to examine opioid misuse and overdose in an integrated health system.
        Prev Med. 2018; 110: 31-37
        • Cho J
        • Spence MM
        • Niu F
        • et al.
        Risk of overdose with exposure to prescription opioids, benzodiazepines, and non-benzodiazepine sedative-hypnotics in adults: a retrospective cohort study.
        J Gen Intern Med. 2020; 35: 696-703
        • Cochran G
        • Gordon AJ
        • Lo-Ciganic WH
        • et al.
        An examination of claims-based predictors of overdose from a large medicaid program.
        Med Care. 2017; 55: 291-298
        • Garg RK
        • Fulton-Kehoe D
        • Franklin GM.
        Patterns of opioid use and risk of opioid overdose death among medicaid patients.
        Med Care. 2017; 55: 661-668
        • Krawczyk N
        • Eisenberg M
        • Schneider KE
        • et al.
        Predictors of overdose death among high-risk emergency department patients with substance-related encounters: a data linkage cohort study.
        Ann Emerg Med. 2020; 75: 1-12
        • Nam YH
        • Bilker WB
        • DeMayo FJ
        • Neuman MD
        • Hennessy S.
        Incidence rates of and risk factors for opioid overdose in new users of prescription opioids among US medicaid enrollees: A cohort study.
        Pharmacoepidemiol Drug Saf. 2020; 29: 931-938
        • Nechuta SJ
        • Tyndall BD
        • Mukhopadhyay S
        • McPheeters ML.
        Sociodemographic factors, prescription history and opioid overdose deaths: a statewide analysis using linked PDMP and mortality data.
        Drug Alcohol Depend. 2018; 190: 62-71
        • Zedler B
        • Xie L
        • Wang L
        • et al.
        Risk factors for serious prescription opioid-related toxicity or overdose among Veterans Health Administration patients.
        Pain Med. 2014; 15: 1911-1929
        • Burke LG
        • Zhou X
        • Boyle KL
        • et al.
        Trends in opioid use disorder and overdose among opioid-naive individuals receiving an opioid prescription in Massachusetts from 2011 to 2014.
        Addiction. 2020; 115: 493-504
        • Fine DR
        • Yu L
        • Triant VA
        • Baggett TP
        • Metlay JP.
        Baseline factors associated with mortality in patients who engaged in buprenorphine treatment for opioid use disorder: a cohort study.
        J Gen Intern Med. 2020; 35: 2375-2382
        • Hernandez I
        • He M
        • Brooks MM
        • Zhang Y.
        Exposure-response association between concurrent opioid and benzodiazepine use and risk of opioid-related overdose in medicare part D beneficiaries.
        JAMA Netw Open. 2018; 1e180919
        • Kuo YF
        • Raji MA
        • Goodwin JS.
        Association of disability with mortality from opioid overdose among US Medicare adults.
        JAMA Netw Open. 2019; 2e1915638
        • Yang Z
        • Wilsey B
        • Bohm M
        • et al.
        Defining risk of prescription opioid overdose: pharmacy shopping and overlapping prescriptions among long-term opioid users in medicaid.
        J Pain. 2015; 16: 445-453
        • Yamamoto A
        • Needleman J
        • Gelberg L
        • et al.
        Association between homelessness and opioid overdose and opioid-related hospital admissions/emergency department visits.
        Soc Sci Med. 2019; 242112585
        • Chua KP
        • Brummett CM
        • Conti RM
        • Bohnert A.
        Association of opioid prescribing patterns with prescription opioid overdose in adolescents and young adults.
        JAMA Pediatr. 2020; 174: 141-148
        • Dilokthornsakul P
        • Moore G
        • Campbell JD
        • et al.
        Risk factors of prescription opioid overdose among Colorado Medicaid beneficiaries.
        J Pain. 2016; 17: 436-443
        • Glanz JM
        • Binswanger IA
        • Shetterly SM
        • Narwaney KJ
        • Xu S.
        Association between opioid dose variability and opioid overdose among adults prescribed long-term opioid therapy.
        JAMA Netw Open. 2019; 2e192613
        • Groenewald CB
        • Zhou C
        • Palermo TM
        • Van Cleve WC.
        Associations between opioid prescribing patterns and overdose among privately insured adolescents.
        Pediatrics. 2019; 144
        • Jairam V
        • Yang DX
        • Yu JB
        • Park HS.
        Emergency department visits for opioid overdoses among patients with cancer.
        J Natl Cancer Inst. 2020; 112: 938-943
        • John WS
        • Wu LT.
        Chronic non-cancer pain among adults with substance use disorders: prevalence, characteristics, and association with opioid overdose and healthcare utilization.
        Drug Alcohol Depend. 2020; 209107902
        • Karmali RN
        • Ray GT
        • Rubinstein AL
        • et al.
        The role of substance use disorders in experiencing a repeat opioid overdose, and substance use treatment patterns among patients with a non-fatal opioid overdose.
        Drug Alcohol Depend. 2020; 209107923
        • Bohnert AS
        • Logan JE
        • Ganoczy D
        • Dowell D.
        A detailed exploration into the association of prescribed opioid dosage and overdose deaths among patients with chronic pain.
        Med Care. 2016; 54: 435-441
        • Dasgupta N
        • Funk MJ
        • Proescholdbell S
        • et al.
        Cohort study of the impact of high-dose opioid analgesics on overdose mortality.
        Pain Med. 2016; 17: 85-98
        • Dunn KM
        • Saunders KW
        • Rutter CM
        • et al.
        Opioid prescriptions for chronic pain and overdose: a cohort study.
        Ann Intern Med. 2010; 152: 85-92
        • Gwira Baumblatt JA
        • Wiedeman C
        • Dunn JR
        • et al.
        High-risk use by patients prescribed opioids for pain and its role in overdose deaths.
        JAMA Intern. Med. 2014; 174: 796-801
        • Larochelle MR
        • Bernstein R
        • Bernson D
        • et al.
        Touchpoints - opportunities to predict and prevent opioid overdose: A cohort study.
        Drug Alcohol Depend. 2019; 204107537
        • Larochelle MR
        • Liebschutz JM
        • Zhang F
        • Ross-Degnan D
        • Wharam JF.
        Opioid prescribing after nonfatal overdose and association with repeated overdose: a cohort study.
        Ann Intern Med. 2016; 164: 1-9
        • Li Y
        • Delcher C
        • Wei YJ
        • et al.
        Risk of opioid overdose associated with concomitant use of opioids and skeletal muscle relaxants: a population-based cohort study.
        Clin Pharmacol Ther. 2020; 108: 81-89
        • Carey CM
        • Jena AB
        • Barnett ML.
        Patterns of potential opioid misuse and subsequent adverse outcomes in Medicare, 2008 to 2012.
        Ann Intern Med. 2018; 168: 837-845
        • James JR
        • Scott JM
        • Klein JW
        • et al.
        Mortality after discontinuation of primary care-based chronic opioid therapy for pain: a retrospective cohort study.
        J Gen Intern Med. 2019; 34: 2749-2755
        • Young JC
        • Lund JL
        • Dasgupta N
        • Jonsson Funk M
        Opioid tolerance and clinically recognized opioid poisoning among patients prescribed extended-release long-acting opioids.
        Pharmacoepidemiol Drug Saf. 2019; 28: 39-47
        • Hallvik SE
        • Geissert P
        • Wakeland W
        • et al.
        Opioid-prescribing continuity and risky opioid prescriptions.
        Ann Fam Med. 2018; 16: 440-442
        • Rose AJ
        • Bernson D
        • Chui KKH
        • et al.
        Potentially inappropriate opioid prescribing, overdose, and mortality in Massachusetts, 2011-2015.
        J Gen Intern Med. 2018; 33: 1512-1519
        • Hartung DM
        • Johnston KA
        • Hallvik S
        • et al.
        Prescription opioid dispensing patterns prior to heroin overdose in a State Medicaid Program: a case-control study.
        J Gen Intern Med. 2020;
        • Khan NF
        • Bateman BT
        • Landon JE
        • Gagne JJ.
        Association of opioid overdose with opioid prescriptions to family members.
        JAMA Intern. Med. 2019; 179: 1186-1192
        • Nguyen AP
        • Glanz JM
        • Narwaney KJ
        • Binswanger IA.
        Association of opioids prescribed to family members with opioid overdose among adolescents and young adults.
        JAMA Netw Open. 2020; 3e201018
        • Calcaterra SL
        • Severtson SG
        • Bau GE
        • et al.
        Trends in intentional abuse or misuse of benzodiazepines and opioid analgesics and the associated mortality reported to poison centers across the United States from 2000 to 2014.
        Clin Toxicol (Phila). 2018; 56: 1107-1114
        • Hawkins EJ
        • Goldberg SB
        • Malte CA
        • Saxon AJ.
        New coprescription of opioids and benzodiazepines and mortality among veterans affairs patients with posttraumatic stress disorder.
        J Clin Psychiatry. 2019; 80
        • Park TW
        • Larochelle MR
        • Saitz R
        • et al.
        Associations between prescribed benzodiazepines, overdose death and buprenorphine discontinuation among people receiving buprenorphine.
        Addiction. 2020; 115: 924-932
        • Boscarino JA
        • Kirchner HL
        • Pitcavage JM
        • et al.
        Factors associated with opioid overdose: a 10-year retrospective study of patients in a large integrated health care system.
        Subst Abuse Rehabil. 2016; 7: 131-141
        • Moons KGM
        • Altman DG
        • Reitsma JB
        • et al.
        Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD): explanation and elaboration.
        Ann Intern Med. 2015; 162: W1-W73
        • Steyerberg EW.
        Clinical prediction models.
        Springer, 2009
        • Chang HY
        • Ferris L
        • Eisenberg M
        • et al.
        The impact of various risk assessment time frames on the performance of opioid overdose forecasting models.
        Med Care. 2020; 58: 1013-1021
        • Chang HY
        • Krawczyk N
        • Schneider KE
        • et al.
        A predictive risk model for nonfatal opioid overdose in a statewide population of buprenorphine patients.
        Drug Alcohol Depend. 2019; 201: 127-133
        • Dong X
        • Rashidian S
        • Wang Y
        • et al.
        Machine learning based opioid overdose prediction using electronic health records.
        AMIA Annu Symp Proc. 2019; 2019: 389-398
        • Ferris LM
        • Saloner B
        • Krawczyk N
        • et al.
        Predicting opioid overdose deaths using prescription drug monitoring program data.
        Am J Prev Med. 2019; 57: e211-e217
        • Geissert P
        • Hallvik S
        • Van Otterloo J
        • et al.
        High-risk prescribing and opioid overdose: prospects for prescription drug monitoring program-based proactive alerts.
        Pain. 2018; 159: 150-156
        • Glanz JM
        • Narwaney KJ
        • Mueller SR
        • et al.
        Prediction model for two-year risk of opioid overdose among patients prescribed chronic opioid therapy.
        J Gen Intern Med. 2018; 33: 1646-1653
        • Lo-Ciganic WH
        • Huang JL
        • Zhang HH
        • et al.
        Evaluation of machine-learning algorithms for predicting opioid overdose risk among Medicare beneficiaries with opioid prescriptions.
        JAMA Netw Open. 2019; 2e190968
        • Metcalfe L
        • Murrelle EL
        • Vu L
        • et al.
        Independent validation in a large privately insured population of the risk index for serious prescription opioid-induced respiratory depression or overdose.
        Pain Med. 2020; 21: 2219-2228
        • Saloner B
        • Chang HY
        • Krawczyk N
        • et al.
        Predictive modeling of opioid overdose using linked statewide medical and criminal justice data.
        JAMA psychiatry. 2020;
        • Sun JW
        • Franklin JM
        • Rough K
        • et al.
        Predicting overdose among individuals prescribed opioids using routinely collected healthcare utilization data.
        PLoS One. 2020; 15e0241083
        • Zedler B
        • Xie L
        • Wang L
        • et al.
        Development of a risk index for serious prescription opioid-induced respiratory depression or overdose in Veterans' Health Administration patients.
        Pain Med. 2015; 16: 1566-1579
        • Zedler BK
        • Saunders WB
        • Joyce AR
        • Vick CC
        • Murrelle EL.
        Validation of a screening risk index for serious prescription opioid-induced respiratory depression or overdose in a US commercial health plan claims database.
        Pain Med. 2018; 19: 68-78
        • Dowell D
        • Haegerich TM
        • Chou R.
        CDC guideline for prescribing opioids for chronic pain–United States, 2016.
        JAMA. 2016; 315: 1624-1645
        • Ochalek TA
        • Cumpston KL
        • Wills BK
        • Gal TS
        • Moeller FG.
        Nonfatal opioid overdoses at an Urban Emergency Department during the COVID-19 pandemic.
        JAMA. 2020;
        • Slavova S
        • Rock P
        • Bush HM
        • Quesinberry D
        • Walsh SL.
        Signal of increased opioid overdose during COVID-19 from emergency medical services data.
        Drug Alcohol Depend. 2020; 214108176
        • Van Calster B
        • McLernon DJ
        • van Smeden M
        • Wynants L
        • Steyerberg EW.
        Calibration: the Achilles heel of predictive analytics.
        BMC Med. 2019; 17: 230
      2. United States Census Bureau. QuickFacts: Kentucky. 2020 [cited 2020 September]; Available from: 2021

      3. United States Census Bureau. QuickFacts: California. 2020 [cited 2020 September]; Available from: 2021

        • National Center for Injury Prevention and Control (CDC)
        Opioid prescribing rate maps.
        Centers for Disease Control and Prevention. 2019; ([cited 2020 September]; Available from)
      4. National Center for Health Statistics (CDC). Multiple Cause of Death 1999-2018. 2020 [cited 2020 September]; Available from: 2021

        • Castillo-Carniglia A
        • Gonzalez-Santa Cruz A
        • Cerdá M
        • et al.
        Changes in opioid prescribing after implementation of mandatory registration and proactive reports within California's prescription drug monitoring program.
        Drug Alcohol Depend. 2020; 221108606
        • Vickers AJ
        • Elkin EB.
        Decision curve analysis: a novel method for evaluating prediction models.
        Med Decis Making. 2006; 26: 565-574
        • Oliva EM
        • Bowe T
        • Tavakoli S
        • et al.
        Development and applications of the Veterans Health Administration's Stratification Tool for Opioid Risk Mitigation (STORM) to improve opioid safetyand prevent overdose and suicide.
        Psychol Serv. 2017; 14: 34-49
      5. General Court of the Commonwealth of Massachusetts. Chapter 55: An Act Requiring Certain Reports for Opiate Overdoses. 2015 [cited 2020 December]; Available from: 2021

        • Ash JS
        • Sittig DF
        • Campbell EM
        • Guappone KP
        • Dykstra RH.
        Some unintended consequences of clinical decision support systems.
        AMIA Annu Symp Proc. 2007; 2007: 26-30
        • Minegishi T
        • Garrido MM
        • Pizer SD
        • Frakt AB.
        Effectiveness of policy and risk targeting for opioid-related risk mitigation: a randomised programme evaluation with stepped-wedge design.
        BMJ open. 2018; 8e020097
        • Chinman M
        • Gellad WF
        • McCarthy S
        • et al.
        Protocol for evaluating the nationwide implementation of the VA Stratification Tool for Opioid Risk Management (STORM).
        Implement Sci. 2019; 14: 5