Christian Rose, MD

All Publications

• Learning from the EHR to implement AI in healthcare. NPJ digital medicine Rose, C., Chen, J. H. 2024; 7 (1): 330

  View details for DOI 10.1038/s41746-024-01340-0

  View details for PubMedID 39567723

  View details for PubMedCentralID 1380189

• The Role of Large Language Models in Transforming Emergency Medicine: Scoping Review. JMIR medical informatics Preiksaitis, C., Ashenburg, N., Bunney, G., Chu, A., Kabeer, R., Riley, F., Ribeira, R., Rose, C. 2024; 12: e53787

  Abstract

  Artificial intelligence (AI), more specifically large language models (LLMs), holds significant potential in revolutionizing emergency care delivery by optimizing clinical workflows and enhancing the quality of decision-making. Although enthusiasm for integrating LLMs into emergency medicine (EM) is growing, the existing literature is characterized by a disparate collection of individual studies, conceptual analyses, and preliminary implementations. Given these complexities and gaps in understanding, a cohesive framework is needed to comprehend the existing body of knowledge on the application of LLMs in EM.Given the absence of a comprehensive framework for exploring the roles of LLMs in EM, this scoping review aims to systematically map the existing literature on LLMs' potential applications within EM and identify directions for future research. Addressing this gap will allow for informed advancements in the field.Using PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) criteria, we searched Ovid MEDLINE, Embase, Web of Science, and Google Scholar for papers published between January 2018 and August 2023 that discussed LLMs' use in EM. We excluded other forms of AI. A total of 1994 unique titles and abstracts were screened, and each full-text paper was independently reviewed by 2 authors. Data were abstracted independently, and 5 authors performed a collaborative quantitative and qualitative synthesis of the data.A total of 43 papers were included. Studies were predominantly from 2022 to 2023 and conducted in the United States and China. We uncovered four major themes: (1) clinical decision-making and support was highlighted as a pivotal area, with LLMs playing a substantial role in enhancing patient care, notably through their application in real-time triage, allowing early recognition of patient urgency; (2) efficiency, workflow, and information management demonstrated the capacity of LLMs to significantly boost operational efficiency, particularly through the automation of patient record synthesis, which could reduce administrative burden and enhance patient-centric care; (3) risks, ethics, and transparency were identified as areas of concern, especially regarding the reliability of LLMs' outputs, and specific studies highlighted the challenges of ensuring unbiased decision-making amidst potentially flawed training data sets, stressing the importance of thorough validation and ethical oversight; and (4) education and communication possibilities included LLMs' capacity to enrich medical training, such as through using simulated patient interactions that enhance communication skills.LLMs have the potential to fundamentally transform EM, enhancing clinical decision-making, optimizing workflows, and improving patient outcomes. This review sets the stage for future advancements by identifying key research areas: prospective validation of LLM applications, establishing standards for responsible use, understanding provider and patient perceptions, and improving physicians' AI literacy. Effective integration of LLMs into EM will require collaborative efforts and thorough evaluation to ensure these technologies can be safely and effectively applied.

  View details for DOI 10.2196/53787

  View details for PubMedID 38728687

• Diagnostic Dilemma: ChatGPT Can't Tell You What You Don't Already Know. Annals of emergency medicine Preiksaitis, C., Rose, C. 2024; 83 (3): 286-287

  View details for DOI 10.1016/j.annemergmed.2023.09.024

  View details for PubMedID 38388084

• A Conference (Missingness in Action) to Address Missingness in Data and AI in Health Care: Qualitative Thematic Analysis. Journal of medical Internet research Rose, C., Barber, R., Preiksaitis, C., Kim, I., Mishra, N., Kayser, K., Brown, I., Gisondi, M. 2023; 25: e49314

  Abstract

  BACKGROUND: Missingness in health care data poses significant challenges in the development and implementation of artificial intelligence (AI) and machine learning solutions. Identifying and addressing these challenges is critical to ensuring the continued growth and accuracy of these models as well as their equitable and effective use in health care settings.OBJECTIVE: This study aims to explore the challenges, opportunities, and potential solutions related to missingness in health care data for AI applications through the conduct of a digital conference and thematic analysis of conference proceedings.METHODS: A digital conference was held in September 2022, attracting 861 registered participants, with 164 (19%) attending the live event. The conference featured presentations and panel discussions by experts in AI, machine learning, and health care. Transcripts of the event were analyzed using the stepwise framework of Braun and Clark to identify key themes related to missingness in health care data.RESULTS: Three principal themes-data quality and bias, human input in model development, and trust and privacy-emerged from the analysis. Topics included the accuracy of predictive models, lack of inclusion of underrepresented communities, partnership with physicians and other populations, challenges with sensitive health care data, and fostering trust with patients and the health care community.CONCLUSIONS: Addressing the challenges of data quality, human input, and trust is vital when devising and using machine learning algorithms in health care. Recommendations include expanding data collection efforts to reduce gaps and biases, involving medical professionals in the development and implementation of AI models, and developing clear ethical guidelines to safeguard patient privacy. Further research and ongoing discussions are needed to ensure these conclusions remain relevant as health care and AI continue to evolve.

  View details for DOI 10.2196/49314

  View details for PubMedID 37995113

• Opportunities, Challenges, and Future Directions of Generative Artificial Intelligence in Medical Education: Scoping Review. JMIR medical education Preiksaitis, C., Rose, C. 2023; 9: e48785

  Abstract

  Generative artificial intelligence (AI) technologies are increasingly being utilized across various fields, with considerable interest and concern regarding their potential application in medical education. These technologies, such as Chat GPT and Bard, can generate new content and have a wide range of possible applications.This study aimed to synthesize the potential opportunities and limitations of generative AI in medical education. It sought to identify prevalent themes within recent literature regarding potential applications and challenges of generative AI in medical education and use these to guide future areas for exploration.We conducted a scoping review, following the framework by Arksey and O'Malley, of English language articles published from 2022 onward that discussed generative AI in the context of medical education. A literature search was performed using PubMed, Web of Science, and Google Scholar databases. We screened articles for inclusion, extracted data from relevant studies, and completed a quantitative and qualitative synthesis of the data.Thematic analysis revealed diverse potential applications for generative AI in medical education, including self-directed learning, simulation scenarios, and writing assistance. However, the literature also highlighted significant challenges, such as issues with academic integrity, data accuracy, and potential detriments to learning. Based on these themes and the current state of the literature, we propose the following 3 key areas for investigation: developing learners' skills to evaluate AI critically, rethinking assessment methodology, and studying human-AI interactions.The integration of generative AI in medical education presents exciting opportunities, alongside considerable challenges. There is a need to develop new skills and competencies related to AI as well as thoughtful, nuanced approaches to examine the growing use of generative AI in medical education.

  View details for DOI 10.2196/48785

  View details for PubMedID 37862079

• ChatGPT is not the solution to physicians' documentation burden. Nature medicine Preiksaitis, C., Sinsky, C. A., Rose, C. 2023

  View details for DOI 10.1038/s41591-023-02341-4

  View details for PubMedID 37169865

  View details for PubMedCentralID 7043175

• Team is brain: leveraging EHR audit log data for new insights into acute care processes. Journal of the American Medical Informatics Association : JAMIA Rose, C., Thombley, R., Noshad, M., Lu, Y., Clancy, H. A., Schlessinger, D., Li, R. C., Liu, V. X., Chen, J. H., Adler-Milstein, J. 2022

  Abstract

  OBJECTIVE: To determine whether novel measures of contextual factors from multi-site electronic health record (EHR) audit log data can explain variation in clinical process outcomes.MATERIALS AND METHODS: We selected one widely-used process outcome: emergency department (ED)-based team time to deliver tissue plasminogen activator (tPA) to patients with acute ischemic stroke (AIS). We evaluated Epic audit log data (that tracks EHR user-interactions) for 3052 AIS patients aged 18+ who received tPA after presenting to an ED at three Northern California health systems (Stanford Health Care, UCSF Health, and Kaiser Permanente Northern California). Our primary outcome was door-to-needle time (DNT) and we assessed bivariate and multivariate relationships with six audit log-derived measures of treatment team busyness and prior team experience.RESULTS: Prior team experience was consistently associated with shorter DNT; teams with greater prior experience specifically on AIS cases had shorter DNT (minutes) across all sites: (Site 1: -94.73, 95% CI: -129.53 to 59.92; Site 2: -80.93, 95% CI: -130.43 to 31.43; Site 3: -42.95, 95% CI: -62.73 to 23.17). Teams with greater prior experience across all types of cases also had shorter DNT at two sites: (Site 1: -6.96, 95% CI: -14.56 to 0.65; Site 2: -19.16, 95% CI: -36.15 to 2.16; Site 3: -11.07, 95% CI: -17.39 to 4.74). Team busyness was not consistently associated with DNT across study sites.CONCLUSIONS: EHR audit log data offers a novel, scalable approach to measure key contextual factors relevant to clinical process outcomes across multiple sites. Audit log-based measures of team experience were associated with better process outcomes for AIS care, suggesting opportunities to study underlying mechanisms and improve care through deliberate training, team-building, and scheduling to maximize team experience.

  View details for DOI 10.1093/jamia/ocac201

  View details for PubMedID 36303451

• Invisibility, cloaksand daggers: Balancing clinical hazards in the age of artificial intelligence. Journal of evaluation in clinical practice Rose, C., Chan, T. 2022

  View details for DOI 10.1111/jep.13758

  View details for PubMedID 36071693

• Addressing Medicine's Dark Matter. Interactive journal of medical research Rose, C., Diaz, M., Diaz, T. 2022; 11 (2): e37584

  Abstract

  In the 20th century, the models used to predict the motion of heavenly bodies did not match observation. Investigating this incongruity led to the discovery of dark matter-the most abundant substance in the universe. In medicine, despite years of using a data-hungry approach, our models have been limited in their ability to predict population health outcomes-that is, our observations also do not meet our expectations. We believe this phenomenon represents medicine's "dark matter"- the features with have a tremendous effect on clinical outcomes that we cannot directly observe yet. Advancing the information science of health care systems will thus require unique solutions and a humble approach that acknowledges its limitations. Dark matter changed the way the scientific community understood the universe; what might medicine learn from what it cannot yet see?

  View details for DOI 10.2196/37584

  View details for PubMedID 35976194

• Signal from the Noise: A Mixed Graphical and Quantitative Process Mining Approach to Evaluate Care Pathways Applied to Emergency Stroke Care. Journal of biomedical informatics Noshad, M., Rose, C. C., Chen, J. H. 1800: 104004

  Abstract

  OBJECTIVE: Mapping real-world practice patterns vs. deviations from intended guidelines and protocols is necessary to identify and improve the quality of care for emergent medical conditions like acute ischemic stroke. Most status-quo process identification relies on expert opinion or direct observation, which can be biased or limited in scalability. We propose a mixed graphical and quantitative process mining approach to Electronic Health Record (EHR) event log data as a unique opportunity not only to more easily identify practice patterns, but also to compare real-world care processes and measure their conformance or variability.MATERIALS: Data was obtained from the event log of a major EHR vendor (Epic) for Stanford Health Care Hospital patients aged 18 years and older presenting to the ED from January 1, 2010 through December 31, 2018 and receiving tPA (tissue plasminogen activator) within 4.5 hours of presentation.METHODS: We developed an unsupervised process-mining algorithm to create a process map from clinical event logs. The method first identifies the most common events across the cohort. Then, all possible ordered events are recorded, and a summarized vector of nodes (events) and edges (events occurring in series) are mapped by their timing and probability. The highest probability ordered pairs are used to identify the most common path. We define measures for individual pathways conformity and average conformity across all encounters.RESULTS: Automatically generated process mining graphs, and specifically it's the most common path, mimicked our institutions recommended "code stroke" clinical pathway. The average conformity score for our cohort was 0.36 (i.e. paths had an average of 36% overlap with all possible paths), with a range from high of 0.64 and low of 0.20.DISCUSSION: This method allows for unsupervised visualization of the current state of common processes as well as their most common path, which can then be used to calculate the conformity of individual pathways through this process. These results may be used to evaluate the consistency of quality care at a given institution. It may also be extended to other common processes like sepsis or myocardial infarction care or even those which currently lack standardized clinical pathways.CONCLUSION: Our mixed graphical and quantitative process mining approach represents an essential data analysis step to improve complex care processes by automatically generating qualitative and quantitative process measures from existing event log data which can then be used to target quality improvement initiatives.

  View details for DOI 10.1016/j.jbi.2022.104004

  View details for PubMedID 35085813

• Trial by fire: How physicians responding to the COVID-19 pandemic illuminated the need for digital credentials. Digital health Brogan, J., Goodier, H., Nijjar, M., Rose, C. 2022; 8: 20552076221084462

  Abstract

  The current credentialing process for physicians struggled to accommodate fluctuating regional demands for providers during the severe acute respiratory syndrome coronavirus 2 pandemic. This hurdle highlighted existing inefficiencies and difficulties facing healthcare systems across the world and led us to explore how credentialing can be improved using digital technologies. We explain how this is a critical moment to make the shift from physical to digital credentials by specifying how a digital credentialing system could simplify onboarding for providers, enable secure expansion of telehealth services, and enhance information exchange.

  View details for DOI 10.1177/20552076221084462

  View details for PubMedID 35309389

  View details for PubMedCentralID PMC8922044

• Developing machine learning models to personalize care levels among emergency room patients for hospital admission. Journal of the American Medical Informatics Association : JAMIA Nguyen, M., Corbin, C. K., Eulalio, T., Ostberg, N. P., Machiraju, G., Marafino, B. J., Baiocchi, M., Rose, C., Chen, J. H. 2021

  Abstract

  OBJECTIVE: To develop prediction models for intensive care unit (ICU) vs non-ICU level-of-care need within 24 hours of inpatient admission for emergency department (ED) patients using electronic health record data.MATERIALS AND METHODS: Using records of 41 654 ED visits to a tertiary academic center from 2015 to 2019, we tested 4 algorithms-feed-forward neural networks, regularized regression, random forests, and gradient-boosted trees-to predict ICU vs non-ICU level-of-care within 24 hours and at the 24th hour following admission. Simple-feature models included patient demographics, Emergency Severity Index (ESI), and vital sign summary. Complex-feature models added all vital signs, lab results, and counts of diagnosis, imaging, procedures, medications, and lab orders.RESULTS: The best-performing model, a gradient-boosted tree using a full feature set, achieved an AUROC of 0.88 (95%CI: 0.87-0.89) and AUPRC of 0.65 (95%CI: 0.63-0.68) for predicting ICU care need within 24 hours of admission. The logistic regression model using ESI achieved an AUROC of 0.67 (95%CI: 0.65-0.70) and AUPRC of 0.37 (95%CI: 0.35-0.40). Using a discrimination threshold, such as 0.6, the positive predictive value, negative predictive value, sensitivity, and specificity were 85%, 89%, 30%, and 99%, respectively. Vital signs were the most important predictors.DISCUSSION AND CONCLUSIONS: Undertriaging admitted ED patients who subsequently require ICU care is common and associated with poorer outcomes. Machine learning models using readily available electronic health record data predict subsequent need for ICU admission with good discrimination, substantially better than the benchmarking ESI system. The results could be used in a multitiered clinical decision-support system to improve ED triage.

  View details for DOI 10.1093/jamia/ocab118

  View details for PubMedID 34402507

• ALiEM Connect: Large-Scale, Interactive Virtual Residency Programming in Response to COVID-19. Academic medicine : journal of the Association of American Medical Colleges Rose, C. C., Haas, M. R., Yilmaz, Y., Alvarez, A., Mott, S. E., Landry, A. I., Gisondi, M. A., Ankel, F., Lin, M., Chan, T. M. 2021

  Abstract

  PROBLEM: The COVID-19 pandemic restricted in-person gatherings, including residency conferences. The pressure to quickly reorganize educational conferences and convert content to a remote format overwhelmed many programs. This article describes the pilot event of a large-scale, interactive virtual educational conference model designed and implemented by Academic Life in Emergency Medicine (ALiEM), called ALiEM Connect.APPROACH: The pilot ALiEM Connect event was conceptualized and implemented within a 2-week period in March 2020. The pilot was livestreamed via a combination of Zoom and YouTube and was archived by YouTube. Slack was used as a backchannel to allow interaction with other participants and engagement with the speakers (via moderators who posed questions from the backchannel to the speakers live during the videoconference).OUTCOMES: The RE-AIM (Reach, Effectiveness, Adoption, Implementation, Maintenance) framework was used for program evaluation, showing that 64 U.S. Accreditation Council for Graduate Medical Education-accredited emergency medicine residency programs participated in the pilot event, with 1,178 unique users during the event (reach). For effectiveness, 93% (139/149) of trainees reported the pilot as enjoyable and 85% (126/149) reported it was equivalent to or better than their usual academic proceedings. Adoption for ALiEM Connect was fairly good with 64/237 (27%) of invited residency programs registering and participating in the pilot event. Implementation was demonstrated by nearly half of the livestream viewers (47%, 553/1,178) interacting in the backchannel discussion, sending a total of 4,128 messages in the first 4 hours.NEXT STEPS: The final component of the RE-AIM framework, maintenance, will take more time to evaluate. Further study is required to measure the educational impact of events like the ALiEM Connect pilot. The ALiEM Connect model could potentially be used to replace educational conferences that have been cancelled or to implement and/or augment a large-scale, shared curriculum among residency programs in the future.

  View details for DOI 10.1097/ACM.0000000000004122

  View details for PubMedID 33883400

• Am I Part of the Cure or Am I Part of the Disease? Keeping Coronavirus Out When a Doctor Comes Home. The New England journal of medicine Rose, C. 2020

  View details for DOI 10.1056/NEJMp2004768

  View details for PubMedID 32187461

• Context is Key: Using the Audit Log to Capture Contextual Factors Affecting Stroke Care Processes. AMIA ... Annual Symposium proceedings. AMIA Symposium Noshad, M., Rose, C. C., Thombley, R., Chiang, J., Corbin, C. K., Nguyen, M., Liu, V. X., Adler-Milstein, J., Chen, J. H. 2020; 2020: 953–62

  Abstract

  High quality patient care through timely, precise and efficacious management depends not only on the clinical presentation of a patient, but the context of the care environment to which they present. Understanding and improving factors that affect streamlined workflow, such as provider or department busyness or experience, are essential to improving these care processes, but have been difficult to measure with traditional approaches and clinical data sources. In this exploratory data analysis, we aim to determine whether such contextual factors can be captured for important clinical processes by taking advantage of non-traditional data sources like EHR audit logs which passively track the electronic behavior of clinical teams. Our results illustrate the potential of defining multiple measures of contextual factors and their correlation with key care processes. We illustrate this using thrombolytic (tPA) treatment for ischemic stroke as an example process, but the measurement approaches can be generalized to multiple scenarios.

  View details for PubMedID 33936471

• Stanford Emergency Medicine Partnership Program: a novel approach to streamlining the evaluation and implementation of emerging health technologies through academic-industry partnerships BMJ INNOVATIONS Dayton, J., Yiadom, M. B., Shen, S., Strehlow, M. C., Rose, C., Bunney, G., Ribeira, R. 2024

  View details for DOI 10.1136/bmjinnov-2023-001154

  View details for Web of Science ID 001251125300001

• Real-Time Electronic Patient Portal Use Among Emergency Department Patients. JAMA network open Turer, R. W., McDonald, S. A., Lehmann, C. U., Thakur, B., Dutta, S., Taylor, R. A., Rose, C. C., Frisch, A., Feterik, K., Norquist, C., Baker, C. K., Nielson, J. A., Cha, D., Kwan, B., Dameff, C., Killeen, J. P., Hall, M. K., Doerning, R. C., Rosenbloom, S. T., Distaso, C., Steitz, B. D. 2024; 7 (5): e249831

  Abstract

  Patients with inequitable access to patient portals frequently present to emergency departments (EDs) for care. Little is known about portal use patterns among ED patients.To describe real-time patient portal usage trends among ED patients and compare demographic and clinical characteristics between portal users and nonusers.In this cross-sectional study of 12 teaching and 24 academic-affiliated EDs from 8 health systems in California, Connecticut, Massachusetts, Ohio, Tennessee, Texas, and Washington, patient portal access and usage data were evaluated for all ED patients 18 years or older between April 5, 2021, and April 4, 2022.Use of the patient portal during ED visit.The primary outcomes were the weekly proportions of ED patients who logged into the portal, viewed test results, and viewed clinical notes in real time. Pooled random-effects models were used to evaluate temporal trends and demographic and clinical characteristics associated with real-time portal use.The study included 1 280 924 unique patient encounters (53.5% female; 0.6% American Indian or Alaska Native, 3.7% Asian, 18.0% Black, 10.7% Hispanic, 0.4% Native Hawaiian or Pacific Islander, 66.5% White, 10.0% other race, and 4.0% with missing race or ethnicity; 91.2% English-speaking patients; mean [SD] age, 51.9 [19.2] years). During the study, 17.4% of patients logged into the portal while in the ED, whereas 14.1% viewed test results and 2.5% viewed clinical notes. The odds of accessing the portal (odds ratio [OR], 1.36; 95% CI, 1.19-1.56), viewing test results (OR, 1.63; 95% CI, 1.30-2.04), and viewing clinical notes (OR, 1.60; 95% CI, 1.19-2.15) were higher at the end of the study vs the beginning. Patients with active portal accounts at ED arrival had a higher odds of logging into the portal (OR, 17.73; 95% CI, 9.37-33.56), viewing test results (OR, 18.50; 95% CI, 9.62-35.57), and viewing clinical notes (OR, 18.40; 95% CI, 10.31-32.86). Patients who were male, Black, or without commercial insurance had lower odds of logging into the portal, viewing results, and viewing clinical notes.These findings suggest that real-time patient portal use during ED encounters has increased over time, but disparities exist in portal access that mirror trends in portal usage more generally. Given emergency medicine's role in caring for medically underserved patients, there are opportunities for EDs to enroll and train patients in using patient portals to promote engagement during and after their visits.

  View details for DOI 10.1001/jamanetworkopen.2024.9831

  View details for PubMedID 38700859

• Thick Data Analytics (TDA): An Iterative and Inductive Framework for Algorithmic Improvement. The American statistician Nguyen, M., Eulalio, T., Marafino, B. J., Rose, C., Chen, J. H., Baiocchi, M. 2024; 78 (4): 456-464

  Abstract

  A gap remains between developing risk prediction models and deploying models to support real-world decision making, especially in high-stakes situations. Human-experts' reasoning abilities remain critical in identifying potential improvements and ensuring safety. We propose a thick data analytics (TDA) framework for eliciting and combining expert-human insight into the evaluation of models. The insight is threefold: (1) statistical methods are limited to using joint distributions of observable quantities for predictions but often there is more information available in a real-world than what is usable for algorithms, (2) domain experts can access more information (e.g., patient files) than an algorithm and bring additional knowledge into their assessments through leveraging insights and experiences, and (3) experts can re-frame and re-evaluate prediction problems to suit real-world situations. Here, we revisit an example of predicting temporal risk for intensive care admission within 24 hours of hospitalization. We propose a sampling procedure for identifying informative cases for deeper inspection. Expert feedback is used to understand sources of information to improve model development and deployment. We recommend model assessment based on objective evaluation metrics derived from subjective evaluations of the problem formulation. TDA insights facilitate iterative model development towards safer, actionable, and acceptable risk predictions.

  View details for DOI 10.1080/00031305.2024.2327535

  View details for PubMedID 39524529

  View details for PubMedCentralID PMC11545316

• Thick Data Analytics (TDA): An Iterative and Inductive Framework for Algorithmic Improvement The American Statistician Nguyen, M., Eulalio, T., Marafino, B. J., Rose, C., Chen, J. H., Baiocchi, M. 2024

  View details for DOI 10.1080/00031305.2024.2327535

• Ghost in the inbox: AI may help alleviate the burden of patient messages. Evidence-based nursing Rose, C., Preiksaitis, C. 2023

  View details for DOI 10.1136/ebnurs-2023-103770

  View details for PubMedID 37657886

• Mobile application adjunct to the WHO basic emergency care course: a mixed methods study. BMJ open Tenner, A. G., Greenberg, A. L., Nicholaus, P., Rose, C. C., Addo, N., Shari, C. R., Friedman, A., George, U. N., Losak, M. J., Mfinanga, J. A., Sawe, H. R. 2022; 12 (7): e056763

  Abstract

  OBJECTIVES: The WHO developed a 5-day basic emergency care (BEC) course using the traditional lecture format. However, adult learning theory suggests that lecture-based courses alone may not promote long-term knowledge retention. We assessed whether a mobile application adjunct (BEC app) can have positive impact on knowledge acquisition and retention compared with the BEC course alone and evaluated perceptions, acceptability and barriers to adoption of such a tool.DESIGN: Mixed-methods prospective cohort study.PARTICIPANTS: Adult healthcare workers in six health facilities in Tanzania who enrolled in the BEC course and were divided into the control arm (BEC course) or the intervention arm (BEC course plus BEC app).MAIN OUTCOME MEASURES: Changes in knowledge assessment scores, self-efficacy and perceptions of BEC app.RESULTS: 92 enrolees, 46 (50%) in each arm, completed the BEC course. 71 (77%) returned for the 4-month follow-up. Mean test scores were not different between the two arms at any time period. Both arms had significantly improved test scores from enrolment (prior to distribution of materials) to day 1 of the BEC course and from day 1 of BEC course to immediately after BEC course completion. The drop-off in mean scores from immediately after BEC course completion to 4months after course completion was not significant for either arm. No differences were observed between the two arms for any self-efficacy question at any time point. Focus groups revealed five major themes related to BEC app adoption: educational utility, clinical utility, user experience, barriers to access and barriers to use.CONCLUSION: The BEC app was well received, but no differences in knowledge retention and self-efficacy were observed between the two arms and only a very small number of participants reported using the app. Technologic-based, linguistic-based and content-based barriers likely limited its impact.

  View details for DOI 10.1136/bmjopen-2021-056763

  View details for PubMedID 35798522

• Journal update monthly top five. Emergency medicine journal : EMJ Saxena, M., Altamirano, J., Rose, C., Bennett, C., Govindarajan, P., Lumba-Brown, A., Hirst, R. 2022; 39 (7): 561-562

  View details for DOI 10.1136/emermed-2022-212603

  View details for PubMedID 35732304

• Congress Should Provide Student Debt Relief To Frontline Health Care Workers Losak, M., Gowda, V., Rose, C. Health Affairs Forefront. 2022
• Professional development during a pandemic: a live virtual conference for emergency medicine chief residents. CJEM Zaver, F., Battaglioli, N., Rose, C. C., Montrief, T., Alvarez, A., Lin, M. 2021

  Abstract

  Limited professional development training exists for chief residents. The available training uses in-person lectures and workshops at annual national conferences. The COVID-19 pandemic prevented most in-person gatherings in 2020, including pivotal onboarding and training events for new chief residents. However, for the last five years, Academic Life in Emergency Medicine's Chief Resident Incubator conducted year-long remote training programs, creating virtual communities of practice for chief residents in emergency medicine (EM). As prior leaders and alumni from the Incubator, we sought to respond to the limitations presented by the pandemic and create an onboarding event to provide foundational knowledge for incoming chief residents. We developed a half-day virtual conference, whereupon 219 EM chief residents enrolled. An effective professional development experience is feasible and scalable using online videoconferencing technologies, especially if constructed with content expertise, psychological safety, and production design in mind.

  View details for DOI 10.1007/s43678-021-00146-3

  View details for PubMedID 34264507

• Reaching further: Lessons from the implementation of the WHO Basic Emergency Care Course Companion App in Tanzania. African journal of emergency medicine : Revue africaine de la medecine d'urgence Greenberg, A. L., Rose, C. C., Nicholaus, P., Mfinanga, J. A., Sawe, H. R., Tenner, A. G. 2021; 11 (2): 325-330

  Abstract

  Introduction: The World Health Organization's (WHO's) Basic Emergency Care (BEC) course was developed to address training gaps in low- and middle-income countries (LMICs). Simultaneously, LMICs have experienced an unprecedented increase in the number of cell phone and internet users. We developed a mobile application adjunct to the BEC course (BEC app) and sought to assess the reach of the BEC app.Methods: Forty-six BEC course participants, made up of doctors and nurses from three hospital sites in Tanzania, were given access to the BEC app with download instructions. Moderators tracked mobile access characteristics and barriers. This is a descriptive study outlining the implementation of the BEC app and associated findings from the process.Results: Fewer than 10% of participants were able to independently download and use the application. The download process revealed three key barrier areas: accessibility (no smartphone, smartphone without charge, no access to data/WiFi to download app, increased cost of data), technical (outdated operating system, inconsistent access to data/WiFi to run the app, insufficient phone storage), and participant-related characteristics (variability in smartphone literary, language discordance, smartphone turnover). Of the 46 participants, 29 (63%) were able to download and use the BEC app successfully with moderator support.Conclusions: There is potential utility of mobile health in LMICs. However, barriers still exist to reaching the largest possible audience for these initiatives. The importance of app compatibility with a broad range of operating systems and limitation of the amount of data needed to download and use the app was underscored by our study. Moreover, creative solutions are needed to facilitate large-scale roll-outs of mobile health interventions, such as a distribution model that relies on super user and peer support rather than an individual moderator. Additional local perspectives on the download process and the utilisation and acceptance of the application post-implementation are needed.

  View details for DOI 10.1016/j.afjem.2021.04.001

  View details for PubMedID 34012767

• Utilizing Lean software development strategies to improve global eHealth initiatives: viewpoint from a basic emergency care app. JMIR formative research Rose, C., Nichols, T., Hackner, D., Chang, J., Straube, S., Jooste, W., Sawe, H., Tenner, A. 2021

  Abstract

  BACKGROUND: Health systems in low- and middle-income countries (LMICs) face considerable challenges in providing high-quality, accessible care. eHealth has had mounting interest as a possible solution given the unprecedented growth in cell phone and internet technologies in these locations. However, few apps or software programs have as of yet gone beyond the testing phase, most downloads are never opened, and consistent use is extremely rare. This is believed to be due to a failure to engage and meet local stakeholder needs as well as high costs of software development.OBJECTIVE: Feedback from World Health Organization (WHO) Basic Emergency Care (BEC) course participants requested a mobile, point-of-care adjunct to learning the complexities of the primary survey. Our team undertook the task of developing this solution through a community-based participatory model in an effort to meet trainees' reported needs and avoid some of the above failings. e aimed to use the well-described Lean software development strategy - owing to familiarity with its elements and ubiquitous use in medicine, global health and software development - to complete this task efficiently and with maximal stakeholder involvement.METHODS: From September 2016 through January 2017, the BEC App was roadmapped and developed at UCSF in California. When a prototype was complete, it was piloted in Cape Town, South Africa and Dar es Salaam, Tanzania - WHO BEC partner sites. Feedback from this pilot shaped continuous amendments to the app before subsequent user testing and study of the effect of use of the app on trainee retention of BEC course material.RESULTS: Our user-centered mobile app was developed relatively quickly and with high acceptance - 95% of BEC Course participants felt it was useful. Our solution had minimal direct costs and resulted in a robust infrastructure for subsequent assessment and maintenance which is familiar to the global health and medical communities and allows for efficient feedback and expansion.CONCLUSIONS: We believe that utilizing Lean software development strategies may help global health advocates and researchers build eHealth solutions utilizing a familiar process with buy-in across stakeholders that is responsive, rapid to deploy and sustainable.CLINICALTRIAL:

  View details for DOI 10.2196/14851

  View details for PubMedID 33882013

• Novel educational adjuncts for the World Health Organization Basic Emergency Care Course: A prospective cohort study AFRICAN JOURNAL OF EMERGENCY MEDICINE Straube, S., Chang-Bullick, J., Nicholaus, P., Mfinanga, J., Rose, C., Nichols, T., Hackner, D., Murphy, S., Sawe, H., Tenner, A. 2020; 10 (1): 30–34

  Abstract

  The World Health Organization's (WHO) Basic Emergency Care Course (BEC) is a five day, in-person course covering basic assessment and life-saving interventions. We developed two novel adjuncts for the WHO BEC: a suite of clinical cases (BEC-Cases) to simulate patient care and a mobile phone application (BEC-App) for reference. The purpose was to determine whether the use of these educational adjuncts in a flipped classroom approach improves knowledge acquisition and retention among healthcare workers in a low-resource setting.We conducted a prospective, cohort study from October 2017 through February 2018 at two district hospitals in the Pwani Region of Tanzania. Descriptive statistics, Fisher's exact t-tests, and Wilcoxon ranked-sum tests were used to examine whether the use of these adjuncts resulted in improved learner knowledge. Participants were enrolled based on location into two arms; Arm 1 received the BEC course and Arm 2 received the BEC-Cases and BEC-App in addition to the BEC course. Both Arms were tested before and after the BEC course, as well as a 7-month follow-up exam. All participants were invited to focus groups on the course and adjuncts.A total of 24 participants were included, 12 (50%) of whom were followed to completion. Mean pre-test scores in Arm 1 (50%) were similar to Arm 2 (53%) (p=0.52). Both arms had improved test scores after the BEC Course Arm 1 (74%) and Arm 2 (87%), (p=0.03). At 7-month follow-up, though with significant participant loss to follow up, Arm 1 had a mean follow-up exam score of 66%, and Arm 2, 74%.Implementation of flipped classroom educational adjuncts for the WHO BEC course is feasible and may improve healthcare worker learning in low resource settings. Our focus- group feedback suggest that the course and adjuncts are user friendly and culturally appropriate.

  View details for DOI 10.1016/j.afjem.2019.11.003

  View details for Web of Science ID 000519198800007

  View details for PubMedID 32161709

  View details for PubMedCentralID PMC7058880

• Physically Distant, Educationally Connected: Interactive Conferencing in the Era of COVID-19. Medical education Rose, C. n., Mott, S. n., Alvarez, A. n., Lin, M. n. 2020

  Abstract

  During the coronavirus outbreak, physical distancing restrictions led to the cancellation of live, large-group events worldwide. This included weekly educational conferences required of Emergency Medicine (EM) residency programs in the United States. Specifically, the Residency Review Committee in EM under the Accreditation Council for Graduate Medical Education has mandated that there be at least four hours per week of synchronous conference didactics.

  View details for DOI 10.1111/medu.14192

  View details for PubMedID 32324933

• Spokes for Our Folks: Public Health Bike Tour. AEM education and training Rose, C., Chang, B., Brown, J. 2019; 3 (4): 393–95

  Abstract

  Nearly half of medical care in the United States is managed through the emergency department, a large portion of which could be managed by "lateral" health services provided by public health facilities like human immunodeficiency virus (HIV) prophylaxis, alcohol and drug treatment programs, emergency psychiatric resources, and medical respite or rehabilitation centers. These options may be underutilized due to lack of knowledge of their services and demographics by patients and health care workers alike. We aimed to educate all levels of emergency medicine trainees and staff to citywide services via bike tour. Participants reported an improved understanding of health services as well as a sense of "camaraderie" toward lateral health services and other providers on the rides.

  View details for DOI 10.1002/aet2.10371

  View details for PubMedID 31637357

• Strategies to Enhance Wellness in Emergency Medicine Residency Training Programs ANNALS OF EMERGENCY MEDICINE Ross, S., Liu, E., Rose, C., Chou, A., Battaglioli, N. 2017; 70 (6): 891–97

  View details for DOI 10.1016/j.annemergmed.2017.07.007

  View details for Web of Science ID 000418311700028

  View details for PubMedID 28826752

• Toward Precision Diagnostics ACADEMIC EMERGENCY MEDICINE Rose, C. C., Rodriguez, R. M. 2017; 24 (5): 644–46

  View details for DOI 10.1111/acem.13163

  View details for Web of Science ID 000401165800013

  View details for PubMedID 28145094