Carla Pugh, MD, PhD

All Publications

• MS-TCRNet: Multi-Stage Temporal Convolutional Recurrent Networks for action segmentation using sensor-augmented kinematics PATTERN RECOGNITION Goldbraikh, A., Shubi, O., Rubin, O., Pugh, C. M., Laufer, S. 2024; 156

  View details for DOI 10.1016/j.patcog.2024.110778

  View details for Web of Science ID 001276182900001

• MS-TCRNet: Multi-Stage Temporal Convolutional Recurrent Networks for Action Segmentation Using Sensor-Augmented Kinematics. Pattern recognition Goldbraikh, A., Shubi, O., Rubin, O., Pugh, C. M., Laufer, S. 2024; 156

  Abstract

  Action segmentation is a challenging task in high-level process analysis, typically performed on video or kinematic data obtained from various sensors. This work presents two contributions related to action segmentation on kinematic data. Firstly, we introduce two versions of Multi-Stage Temporal Convolutional Recurrent Networks (MS-TCRNet), specifically designed for kinematic data. The architectures consist of a prediction generator with intra-stage regularization and Bidirectional LSTM or GRU-based refinement stages. Secondly, we propose two new data augmentation techniques, World Frame Rotation and Hand Inversion, which utilize the strong geometric structure of kinematic data to improve algorithm performance and robustness. We evaluate our models on three datasets of surgical suturing tasks: the Variable Tissue Simulation (VTS) Dataset and the newly introduced Bowel Repair Simulation (BRS) Dataset, both of which are open surgery simulation datasets collected by us, as well as the JHU-ISI Gesture and Skill Assessment Working Set (JIGSAWS), a well-known benchmark in robotic surgery. Our methods achieved state-of-the-art performance. code: https://github.com/AdamGoldbraikh/MS-TCRNet.

  View details for DOI 10.1016/j.patcog.2024.110778

  View details for PubMedID 39494221

  View details for PubMedCentralID PMC11526485

• Key Issues in Surgical Residency Education: Recommendations of the Blue Ribbon II Committee Residency Education Subcommittee. Annals of surgery Mellinger, J. D., Brasel, K., Elster, E., Fried, G., Hashimoto, D. A., Jarman, B., Joshi, A. R., Kelz, R. R., Lindeman, B., Pugh, C., Reznick, R. 2024

  Abstract

  In September 2022, a summit was convened by the American Board of Surgery (ABS) to discuss competency-based reform in surgical education. A key output of that summit was the recommendation that the prior work of the Blue Ribbon I Committee convened 20 years earlier be revived. With leadership from the American College of Surgeons (ACS) and the American Surgical Association (ASA) , the Blue Ribbon Committee (BRC) II was subsequently convened. This paper describes the output of the Residency Education Subcommittee of the BRC II Committee. The Subcommittee organized its work around prioritized themes including curriculum, assessment, and transition to practice. Top recommendations, time-based action steps, potential barriers, and required resources were detailed and vetted through group discussion, broader Committee review and critique, and subsequent refinement. Primary concluding emphases included transitioning to a competency-based training model, facilitating dynamically capable curricular reform emphasizing the digital transformation of surgical care, using predictive analytic assessment strategies to optimize training effectiveness and efficiency, and creating mentorship strategies to govern the transition from training to independent practice in an outcomes-accountable fashion. It was recognized that coordinated efforts across existing organizational structures will be required, informed by dataset integration strategies that meaningfully measure educational and related patient outcomes.

  View details for DOI 10.1097/SLA.0000000000006434

  View details for PubMedID 38946537

• Opportunities and Applications of Educational Technologies in Surgical Education and Assessment. Annals of surgery Fried, G. M., Varas, J., Telem, D. A., Greenberg, C. C., Hashimoto, D. A., Paige, J. T., Pugh, C. 2024

  Abstract

  Surgical education is challenged by continuously increasing clinical content, greater subspecialization, and public scrutiny of access to high quality surgical care. Since the last Blue Ribbon Committee on surgical education, novel technologies have been developed including artificial intelligence and telecommunication.The goals of this Blue Ribbon Sub-Committee were to describe the latest technological advances and construct a framework for applying these technologies to improve the effectiveness and efficiency of surgical education and assessment. An additional goal was to identify implementation frameworks and strategies for centers with different resources and access. All sub-committee recommendations were included in a Delphi consensus process with the entire Blue Ribbon Committee (N=67).Our sub-committee found several new technologies and opportunities that are well poised to improve the effectiveness and efficiency of surgical education and assessment (see Tables 1-3). Our top recommendation was that a Multidisciplinary Surgical Educational Council be established to serve as an oversight body to develop consensus, facilitate implementation, and establish best practices for technology implementation and assessment. This recommendation achieved 93% consensus during the first round of the Delphi process.Advances in technology-based assessment, data analytics, and behavioral analysis now allow us to create personalized educational programs based on individual preferences and learning styles. If implemented properly, education technology has the promise of improving the quality and efficiency of surgical education and decreasing the demands on clinical faculty.

  View details for DOI 10.1097/SLA.0000000000006367

  View details for PubMedID 38810267

• White paper: requirements for routine data recording in the operating room ARTIFICIAL INTELLIGENCE SURGERY Schnelldorfer, T., Gumbs, A. A., Tolkoff, J., Choksi, S., Stockheim, J., Madani, A., Pugh, C. M., Ishizawa, T., Speidel, S., Swanstroem, L. L., Rau, B. M., Szold, A., Ausania, F., Filicori, F., Croner, R., Grasso, S. 2024; 4 (1): 7-22

  View details for DOI 10.20517/ais.2023.34

  View details for Web of Science ID 001318112600002

• Precision Education: The Future of Lifelong Learning in Medicine. Academic medicine : journal of the Association of American Medical Colleges Desai, S. V., Burk-Rafel, J., Lomis, K. D., Caverzagie, K., Richardson, J., O'Brien, C. L., Andrews, J., Heckman, K., Henderson, D., Prober, C. G., Pugh, C. M., Stern, S. D., Triola, M. M., Santen, S. A. 2024

  Abstract

  The goal of medical education is to produce a physician workforce capable of delivering high-quality equitable care to diverse patient populations and communities. To achieve this aim amidst explosive growth in medical knowledge and increasingly complex medical care, a system of personalized and continuous learning, assessment, and feedback for trainees and practicing physicians is urgently needed. In this perspective, the authors build on prior work to advance a conceptual framework for such a system: precision education (PE).PE is a system that uses data and technology to transform lifelong learning by improving personalization, efficiency, and agency at the individual, program, and organization levels. PE "cycles" start with data inputs proactively gathered from new and existing sources, including assessments, educational activities, electronic medical records, patient care outcomes, and clinical practice patterns. Through technology-enabled analytics, insights are generated to drive precision interventions. At the individual level, such interventions include personalized just-in-time educational programming. Coaching is essential to provide feedback and increase learner participation and personalization. Outcomes are measured using assessment and evaluation of interventions at the individual, program, and organizational level, with ongoing adjustment for repeated cycles of improvement. PE is rooted in patient, health system, and population data; promotes value-based care and health equity; and generates an adaptive learning culture.The authors suggest fundamental principles for PE, including promoting equity in structures and processes, learner agency, and integration with workflow (harmonization). Finally, the authors explore the immediate need to develop consensus-driven standards: rules of engagement between people, products, and entities that interact in these systems to ensure interoperability, data sharing, replicability, and scale of PE innovations.

  View details for DOI 10.1097/ACM.0000000000005601

  View details for PubMedID 38277444

• Sensor-Based Discovery of Search and Palpation Modes in the Clinical Breast Examination. Academic medicine : journal of the Association of American Medical Colleges Laufer, S., Klatzky, R. L., Pugh, C. M. 2024

  Abstract

  Successful implementation of precision education systems requires widespread adoption and seamless integration of new technologies with unique data streams that facilitate real-time performance feedback. This paper explores the use of sensor technology to quantify hands-on clinical skills. The goal is to shorten the learning curve through objective and actionable feedback.A sensor-enabled clinical breast examination (CBE) simulator was used to capture force and video data from practicing clinicians (N = 152). Force-by-time markers from the sensor data and a machine learning algorithm were used to parse physicians' CBE performance into periods of search and palpation and then these were used to investigate distinguishing characteristics of successful vs unsuccessful attempts to identify masses in CBEs.Mastery performance from successful physicians showed stable levels of speed and force across the entire CBE and a 15% increase in force when in palpation mode compared to search mode. Unsuccessful physicians failed to search with sufficient force to detect deep masses F(5,146) = 4.24, P = .001. While similar proportions of male and female physicians reached the highest performance level, males used more force as noted by higher palpation to search force ratios t(63) = 2.52, P = .014.Sensor technology can serve as a useful pathway to assess hands-on clinical skills and provide data-driven feedback. When using a sensor-enabled simulator, the authors found specific haptic approaches that were associated with successful CBE outcomes. Given this study's findings, continued exploration of sensor technology in support of precision education for hands-on clinical skills is warranted.

  View details for DOI 10.1097/ACM.0000000000005614

  View details for PubMedID 38207081

• Foreword: The Next Era of Assessment and Precision Education. Academic medicine : journal of the Association of American Medical Colleges Schumacher, D. J., Santen, S. A., Pugh, C. M., Burk-Rafel, J. 2023

  View details for DOI 10.1097/ACM.0000000000005609

  View details for PubMedID 38109655

• Haptics: The Science of Touch as a Foundational Pathway to Precision Education and Assessment. Academic medicine : journal of the Association of American Medical Colleges Perrone, K., Abdelaal, A. E., Pugh, C., Okamura, A. 2023

  Abstract

  Clinical touch is the cornerstone of the doctor-patient relationship and can impact patient experience and outcomes. In the current era, driven by an ever-increasing infusion of point of care technologies, physical exam skills have become undervalued. Moreover, touch and hands-on skills have been difficult to teach due to inaccurate assessments and difficulty with learning transfer through observation. In this article, the authors argue that haptics, the science of touch, provides a unique opportunity to explore new pathways to facilitate touch training. Furthermore, haptics can dramatically increase the density of touch-based assessments without increasing human rater burden-essential for realizing precision assessment. The science of haptics is reviewed, including the benefits of using haptics-informed language for objective structured clinical examinations. The authors describe how haptic devices and haptic language have and can be used to facilitate learning, communication, documentation and a much-needed reinvigoration of physical examination and touch excellence at the point of care. The synergy of haptic devices, artificial intelligence, and virtual reality environments are discussed. The authors conclude with challenges of scaling haptic technology in medical education, such as cost and translational needs, and opportunities to achieve wider adoption of this transformative approach to precision education.

  View details for DOI 10.1097/ACM.0000000000005607

  View details for PubMedID 38109654

• Surgical Education. Annals of surgery Farmer, D. L., O'Connell, P. R., Pugh, C. M., Lang, H., Greenberg, C. C., Borel-Rinkes, I. H., Mellinger, J. D., Pinto-Marques, H. 2023; 278 (5): 642-646

  Abstract

  This paper summarizes the proceedings of the joint European Surgical Association ESA/American Surgical Association symposium on Surgical Education that took place in Bordeaux, France, as part of the celebrations for 30 years of ESA scientific meetings. Three presentations on the use of quantitative metrics to understand technical decisions, coaching during training and beyond, and entrustable professional activities were presented by American Surgical Association members and discussed by ESA members in a symposium attended by members of both associations.

  View details for DOI 10.1097/SLA.0000000000006078

  View details for PubMedID 37796749

• Surgical Instant Replay-A National Video-Based Performance Assessment Toolbox. JAMA surgery Yule, S., Dearani, J. A., Pugh, C. 2023

  Abstract

  This article discusses the widespread implementation of surgical video replay to improve technical and nontechnical performance of surgeons.

  View details for DOI 10.1001/jamasurg.2023.1803

  View details for PubMedID 37755836

• SAGES consensus recommendations on surgical video data use, structure, and exploration (for research in artificial intelligence, clinical quality improvement, and surgical education). Surgical endoscopy Eckhoff, J. A., Rosman, G., Altieri, M. S., Speidel, S., Stoyanov, D., Anvari, M., Meier-Hein, L., Marz, K., Jannin, P., Pugh, C., Wagner, M., Witkowski, E., Shaw, P., Madani, A., Ban, Y., Ward, T., Filicori, F., Padoy, N., Talamini, M., Meireles, O. R. 2023

  Abstract

  BACKGROUND: Surgery generates a vast amount of data from each procedure. Particularly video data provides significant value for surgical research, clinical outcome assessment, quality control, and education. The data lifecycle is influenced by various factors, including data structure, acquisition, storage, and sharing; data use and exploration, and finally data governance, which encompasses all ethical and legal regulations associated with the data. There is a universal need among stakeholders in surgical data science to establish standardized frameworks that address all aspects of this lifecycle to ensure data quality and purpose.METHODS: Working groups were formed, among 48 representatives from academia and industry, including clinicians, computer scientists and industry representatives. These working groups focused on: Data Use, Data Structure, Data Exploration, and Data Governance. After working group and panel discussions, a modified Delphi process was conducted.RESULTS: The resulting Delphi consensus provides conceptualized and structured recommendations for each domain related to surgical video data. We identified the key stakeholders within the data lifecycle and formulated comprehensive, easily understandable, and widely applicable guidelines for data utilization. Standardization of data structure should encompass format and quality, data sources, documentation, metadata, and account for biases within the data. To foster scientific data exploration, datasets should reflect diversity and remain adaptable to future applications. Data governance must be transparent to all stakeholders, addressing legal and ethical considerations surrounding the data.CONCLUSION: This consensus presents essential recommendations around the generation of standardized and diverse surgical video databanks, accounting for multiple stakeholders involved in data generation and use throughout its lifecycle. Following the SAGES annotation framework, we lay the foundation for standardization of data use, structure, and exploration. A detailed exploration of requirements for adequate data governance will follow.

  View details for DOI 10.1007/s00464-023-10288-3

  View details for PubMedID 37516693

• Allyship in action: The critical, missing link to crossing the quality chasm in healthcare. American journal of surgery Pugh, C. M. 2023

  View details for DOI 10.1016/j.amjsurg.2023.06.002

  View details for PubMedID 37331909

• Laying the Groundwork for Optimized Surgical Feedback. JAMA network open Shkolyar, E., Pugh, C., Liao, J. C. 2023; 6 (6): e2320465

  View details for DOI 10.1001/jamanetworkopen.2023.20465

  View details for PubMedID 37378988

• Promoting breast health awareness: Can a sensor-enabled training system for patient education help? American journal of surgery Kearse, L., Goll, C., Wise, B., Yang, S., Mohamadipanah, H., Witt, A., Ratliff, P., Pugh, C. 2023

  Abstract

  According to a 2009 study published in the Journal of Clinical Oncology, 79% of women (N = 222) diagnosed with breast cancer reported that they identified their cancers through breast self-exam (BSE). However, the U.S. Preventative Services Task Force does not require clinicians to teach women how to perform BSE.To address this grave challenge, our team at the Technology Enabled Clinical Improvement (TECI) Center has developed a mobile, sensor-enabled haptic training system to teach women proper BSE technique. To validate the efficacy of the training system, our team deployed a data collection at the 2019 Breast Cancer and African Americans (BCAA) event where survey, sensor, and anecdotal data were collected from 61 participants. The custom-built breast model used in this study had a single, hard mass embedded in it.Participants at the BCAA event were able to successfully identify the mass 65% of the time and used an average force of 7.2 N. When looking at participants' confidence in their abilities to perform BSE, only 10% of respondents answered "very confident" pre-training whereas post-training, the reporting for "very confident" jumped to 66% (p < 0.01).By comparison, our previous work revealed that practitioners who use less than 10 N of force are 70% more likely to miss a lesion. The integration of sensors into the BSE haptic training system allowed for objective, evidence-based assessment of hands-on skill. In addition to teaching women proper BSE technique, this training empowered women to be informed advocates in their breast health journey. Future community-based training/feedback sessions will allow for continuous advancement of the training system.

  View details for DOI 10.1016/j.amjsurg.2023.05.003

  View details for PubMedID 37258320

• The Quantified Surgeon: A Glimpse Into the Future of Surgical Metrics and Outcomes. The American surgeon Pugh, C. M. 2023: 31348231168315

  Abstract

  This paper summarizes key points of the 2023 Southeastern Surgical Congress Laws Lecture. The focus of the presentation was on the use of advanced engineering technology to quantify surgical mastery. New concepts relating to the visual-haptic loop, mastery and perception, and mastery and technical decisions were introduced and shown in an empirical fashion to have relevance in procedural outcomes in a simulated setting. The major takeaway point is that surgical mastery can be quantified using advanced engineering technology, and this process will help to shorten the learning curve.

  View details for DOI 10.1177/00031348231168315

  View details for PubMedID 37002209

• Response to: Comment on The AI and I: A Collaboration on Competence. Annals of surgery open : perspectives of surgical history, education, and clinical approaches Funk, L. M., Pugh, C. M. 2023; 4 (1): e272

  View details for DOI 10.1097/AS9.0000000000000272

  View details for PubMedID 37600895

  View details for PubMedCentralID PMC10431507

• Reclaiming the Calendar: Time Management for the Clinician Educator. Journal of graduate medical education Pitre, C. J., Pugh, C. M. 2023; 15 (1): 117-118

  View details for DOI 10.4300/JGME-D-22-00939.1

  View details for PubMedID 36817525

• An American Board of Surgery Pilot of Video Assessment of Surgeon Technical Performance in Surgery. Annals of surgery Pryor, A. D., Lendvay, T., Jones, A., Ibáñez, B., Pugh, C. 2023

  Abstract

  The American Board of Surgery sought to investigate the suitability of Video-based Assessment (VBA) as an adjunct to certification for assessing technical skills.Board certification is based on successful completion of a residency program coupled with knowledge and reasoning assessments. Video-based Assessment is a new modality for evaluating operative skills that has been shown to correlate with patient outcomes following surgery.Diplomates of the ABS were initially assessed for background knowledge and interest in VBA. Surgeons were then solicited to participate in the pilot. Three commercially available VBA platforms were identified and used for the pilot assessment. All participants served as reviewers and reviewees for videos. Following the interaction, participants were surveyed regarding their experiences and recommendations to the ABS.4,853/25,715 diplomates responded to the initial survey. The majority were not familiar with VBA, nor the tools used for operative assessments. 274 surgeons actively engaged in the subsequent pilot. 169 surgeons completed the post pilot survey. Most participants found the process straightforward. 74% of participants felt that the feedback would help their surgical practice. The majority (81%) remain interested in VBA for CME credits. 70% felt that using VBA in continuous certification (CC) could improve surgeon skills. Two-thirds of participants felt VBA could help identify and remediate underperforming surgeons. Identified barriers to VBA included limitations for open surgery, privacy issues and technical concerns.VBA is promising for an adjunct to the current board certification process and should be further considered by the ABS.

  View details for DOI 10.1097/SLA.0000000000005804

  View details for PubMedID 36645875

• Addressing the Surgical Workplace: An Opportunity to Create a Culture of Belonging. Annals of surgery Pugh, C. M., Kirton, O. C., Tuttle, J. E., Maier, R. V., Hu, Y., Stewart, J. H., Freischlag, J. A., Sosa, J. A., Vickers, S. M., Hawn, M. T., Eberlein, T. J., Farmer, D. L., Higgins, R. S., Pellegrini, C. A., Roman, S. A., Crandall, M. L., De Virgilio, C. M., Tsung, A., Britt, L. D. 2022

  View details for DOI 10.1097/SLA.0000000000005773

  View details for PubMedID 36575980

• Generating Rare Surgical Events Using CycleGAN: Addressing Lack of Data for Artificial Intelligence Event Recognition. The Journal of surgical research Mohamadipanah, H., Kearse, L., Wise, B., Backhus, L., Pugh, C. 2022; 283: 594-605

  Abstract

  Artificial Intelligence (AI) has shown promise in facilitating surgical video review through automatic recognition of surgical activities/events. There are few public video data sources that demonstrate critical yet rare events which are insufficient to train AI for reliable video event recognition. We suggest that a generative AI algorithm can create artificial massive bleeding images for minimally invasive lobectomy that can be used to augment the current lack of data in this field.A generative adversarial network (GAN) algorithm was used (CycleGAN) to generate artificial massive bleeding event images. To train CycleGAN, six videos of minimally invasive lobectomies were utilized from which 1819 frames of nonbleeding instances and 3178 frames of massive bleeding instances were used.The performance of the CycleGAN algorithm was tested on a new video that was not used during the training process. The trained CycleGAN was able to alter the laparoscopic lobectomy images according to their corresponding massive bleeding images, where the contents of the original images were preserved (e.g., location of tools in the scene) and the style of each image is changed to massive bleeding (i.e., blood automatically added to appropriate locations on the images).The result could suggest a promising approach to supplement the lack of data for the rare massive bleeding event that can occur during minimally invasive lobectomy. Future work could be dedicated to developing AI algorithms to identify surgical strategies and actions that potentially lead to massive bleeding and warn surgeons prior to this event occurrence.

  View details for DOI 10.1016/j.jss.2022.11.008

  View details for PubMedID 36442259

• AI-Based Video Segmentation: Procedural Steps or Basic Maneuvers? The Journal of surgical research Perumalla, C., Kearse, L., Peven, M., Laufer, S., Goll, C., Wise, B., Yang, S., Pugh, C. 2022; 283: 500-506

  Abstract

  INTRODUCTION: Video-based review of surgical procedures has proven to be useful in training by enabling efficiency in the qualitative assessment of surgical skill and intraoperative decision-making. Current video segmentation protocols focus largely on procedural steps. Although some operations are more complex than others, many of the steps in any given procedure involve an intricate choreography of basic maneuvers such as suturing, knot tying, and cutting. The use of these maneuvers at certain procedural steps can convey information that aids in the assessment of the complexity of the procedure, surgical preference, and skill. Our study aims to develop and evaluate an algorithm to identify these maneuvers.METHODS: A standard deep learning architecture was used to differentiate between suture throws, knot ties, and suture cutting on a data set comprised of videos from practicing clinicians (N=52) who participated in a simulated enterotomy repair. Perception of the added value to traditional artificial intelligence segmentation was explored by qualitatively examining the utility of identifying maneuvers in a subset of steps for an open colon resection.RESULTS: An accuracy of 84% was reached in differentiating maneuvers. The precision in detecting the basic maneuvers was 87.9%, 60%, and 90.9% for suture throws, knot ties, and suture cutting, respectively. The qualitative concept mapping confirmed realistic scenarios that could benefit from basic maneuver identification.CONCLUSIONS: Basic maneuvers can indicate error management activity or safety measures and allow for the assessment of skill. Our deep learning algorithm identified basic maneuvers with reasonable accuracy. Such models can aid in artificial intelligence-assisted video review by providing additional information that can complement traditional video segmentation protocols.

  View details for DOI 10.1016/j.jss.2022.10.069

  View details for PubMedID 36436286

• Annals of Surgery Open Access: Where is the Value, and What does the Future Hold? Annals of surgery Funk, L. M., Barr, J., Johnston, F. M., Smith, B. K., Cooper, Z., Pugh, C., Dimick, J. B., Clavien, P., Read, T. E., Wong, S. L. 2022

  View details for DOI 10.1097/SLA.0000000000005723

  View details for PubMedID 36218310

• Society of Black Academic Surgeons (SBAS) diversity, equity, and inclusion series: Microaggressions - Lessons Learned from Black Academic Surgeons. American journal of surgery Butler, P. D., Wexner, S. D., Alimi, Y. R., Dent, D. L., Fayanju, O. M., Gantt, N. L., Johnston, F. M., Pugh, C. M. 2022

  Abstract

  BACKGROUND: Microaggressions can target individuals based on a variety of differences and these can include sexual orientation, nationality, gender, or personal traits and are often disruptors in the healthcare setting.METHODS: To address this issue, The Society of Black Academic Surgeons (SBAS) convened a series of presentations and a panel discussion by leaders from SBAS regarding the issue of microaggressions in the surgical workplace. This program was part of a monthly diversity, equity, and inclusion series produced by the Advances in Surgery Channel in alliance with the American College of Surgeons. Dr. Yewande Alimi addresses microaggressions in surgical training, Dr. Fabian Johnston talks about microaggressions in the black male physician, Dr. Lola Fayanju speaks to microaggressions and the black female surgeon, Dr. Carla Pugh discusses microaggressions in the surgical workplace, and Dr. Paris Butler presents on allyship, policies, and real solutions.RESULTS: Specifically, through the lens of the Black surgeon experience, SBAS leaders candidly articulate and elaborate on microaggressions' pervasiveness and the deleterious impact on the profession. Authentic opinions are rendered and constructive techniques to mitigate this challenge are provided. The concept of majority allyship is also introduced, and recommendations on how this can be operationalized is also examined.CONCLUSIONS: There are a lot of experiences that contribute to our understanding of microaggressions. We look forward to finding new ways to partner with our allies and continuing the conversation.

  View details for DOI 10.1016/j.amjsurg.2022.09.001

  View details for PubMedID 36155676

• Do Individual Surgeon Preferences Affect Procedural Outcomes? Annals of surgery Mohamadipanah, H., Perumalla, C. A., Kearse, L. E., Yang, S., Wise, B. J., Goll, C. K., Witt, A. K., Korndorffer, J. R., Pugh, C. M. 2022

  Abstract

  OBJECTIVES: Surgeon preferences such as instrument and suture selection and idiosyncratic approaches to individual procedure steps have been largely viewed as minor differences in the surgical workflow. We hypothesized that idiosyncratic approaches could be quantified and shown to have measurable effects on procedural outcomes.METHODS: At the ACS Clinical Congress, experienced surgeons volunteered to wear motion tracking sensors and be videotaped while evaluating a loop of porcine intestines to identify and repair two pre-configured, standardized enterotomies. Video annotation was used to identify individual surgeon preferences and motion data was used to quantify surgical actions. Chi-square analysis was used to determine whether surgical preferences were associated with procedure outcomes (bowel leak).RESULTS: Surgeons' (N=255) preferences were categorized into four technical decisions. Three out of the four technical decisions (repaired injuries together, double layer closure, corner-stitches versus no corner-stitches) played a significant role in outcomes, P<0.05. Running versus interrupted did not affect outcomes. Motion analysis revealed significant differences in average operative times (leak-6.67 min vs. no leak-8.88 min, P=0.0004) and work effort (leak-path length=36.86cm vs. no leak-path length=49.99cm, P=0.001). Surgeons who took the riskiest path but did not leak had better bimanual dexterity (leak=0.21/1.0 vs. no leak=0.33/1.0, P=0.047) and placed more sutures during the repair (leak=4.69 sutures vs. no leak=6.09 sutures, P=0.03).CONCLUSION: Our results show that individual preferences affect technical decisions and play a significant role in procedural outcomes. Future analysis in more complex procedures may make major contributions to our understanding of contributors to procedure outcomes.

  View details for DOI 10.1097/SLA.0000000000005595

  View details for PubMedID 35861074

• Open surgery tool classification and hand utilization using a multi-camera system. International journal of computer assisted radiology and surgery Basiev, K., Goldbraikh, A., Pugh, C. M., Laufer, S. 2022

  Abstract

  The goal of this work is to use multi-camera video to classify open surgery tools as well as identify which tool is held in each hand. Multi-camera systems help prevent occlusions in open surgery video data. Furthermore, combining multiple views such as a top-view camera covering the full operative field and a close-up camera focusing on hand motion and anatomy may provide a more comprehensive view of the surgical workflow. However, multi-camera data fusion poses a new challenge: A tool may be visible in one camera and not the other. Thus, we defined the global ground truth as the tools being used regardless their visibility. Therefore, tools that are out of the image should be remembered for extensive periods of time while the system responds quickly to changes visible in the video.Participants (n = 48) performed a simulated open bowel repair. A top-view and a close-up cameras were used. YOLOv5 was used for tool and hand detection. A high-frequency LSTM with a 1-second window at 30 frames per second (fps) and a low-frequency LSTM with a 40-second window at 3 fps were used for spatial, temporal, and multi-camera integration.The accuracy and F1 of the six systems were: top-view (0.88/0.88), close-up (0.81,0.83), both cameras (0.9/0.9), high-fps LSTM (0.92/0.93), low-fps LSTM (0.9/0.91), and our final architecture the multi-camera classifier(0.93/0.94).Since each camera in a multi-camera system may have a partial view of the procedure, we defined a 'global ground truth.' Defining this at the data labeling phase emphasized this requirement at the learning phase, eliminating the need for any heuristic decisions. By combining a system with a high fps and a low fps from the multiple camera array, we improved the classification abilities of the global ground truth.

  View details for DOI 10.1007/s11548-022-02691-3

  View details for PubMedID 35759176

• Artificial intelligence in surgery: A research team perspective. Current problems in surgery Mohamadipanah, H., Perumalla, C., Yang, S., Wise, B., Kearse, L., Goll, C., Witt, A., Korndorffer, J. R., Pugh, C. 2022; 59 (6): 101125

  View details for DOI 10.1016/j.cpsurg.2022.101125

  View details for PubMedID 35690434

• Artificial Intelligence Methods and Artificial Intelligence-Enabled Metrics for Surgical Education: A Multidisciplinary Consensus. Journal of the American College of Surgeons Vedula, S. S., Ghazi, A., Collins, J. W., Pugh, C., Stefanidis, D., Meireles, O., Hung, A. J., Schwaitzberg, S., Levy, J. S., Sachdeva, A. K., and the Collaborative for Advanced Assessment of Robotic Surgical Skills, Stoyanov, D., Chen, C. C., Hernandez, E., Athanasiadis, D. I., Martino, M. A., Feins, R., Satava, R. 2022; 234 (6): 1181-1192

  Abstract

  BACKGROUND: Artificial intelligence (AI) methods and AI-enabled metrics hold tremendous potential to advance surgical education. Our objective was to generate consensus guidance on specific needs for AI methods and AI-enabled metrics for surgical education.STUDY DESIGN: The study included a systematic literature search, a virtual conference, and a 3-round Delphi survey of 40 representative multidisciplinary stakeholders with domain expertise selected through purposeful sampling. The accelerated Delphi process was completed within 10 days. The survey covered overall utility, anticipated future (10-year time horizon), and applications for surgical training, assessment, and feedback. Consensus was agreement among 80% or more respondents. We coded survey questions into 11 themes and descriptively analyzed the responses.RESULTS: The respondents included surgeons (40%), engineers (15%), affiliates of industry (27.5%), professional societies (7.5%), regulatory agencies (7.5%), and a lawyer (2.5%). The survey included 155 questions; consensus was achieved on 136 (87.7%). The panel listed 6 deliverables each for AI-enhanced learning curve analytics and surgical skill assessment. For feedback, the panel identified 10 priority deliverables spanning 2-year (n = 2), 5-year (n = 4), and 10-year (n = 4) timeframes. Within 2 years, the panel expects development of methods to recognize anatomy in images of the surgical field and to provide surgeons with performance feedback immediately after an operation. The panel also identified 5 essential that should be included in operative performance reports for surgeons.CONCLUSIONS: The Delphi panel consensus provides a specific, bold, and forward-looking roadmap for AI methods and AI-enabled metrics for surgical education.

  View details for DOI 10.1097/XCS.0000000000000190

  View details for PubMedID 35703817

• In Brief. Current problems in surgery Mohamadipanah, H., Perumalla, C., Yang, S., Wise, B., Kearse, L., Goll, C., Witt, A., Korndorffer, J. R., Pugh, C. 2022; 59 (6): 101127

  View details for DOI 10.1016/j.cpsurg.2022.101127

  View details for PubMedID 35690433

• Using open surgery simulation kinematic data for tool and gesture recognition. International journal of computer assisted radiology and surgery Goldbraikh, A., Volk, T., Pugh, C. M., Laufer, S. 2022

  Abstract

  PURPOSE: The use of motion sensors is emerging as a means for measuring surgical performance. Motion sensors are typically used for calculating performance metrics and assessing skill. The aim of this study was to identify surgical gestures and tools used during an open surgery suturing simulation based on motion sensor data.METHODS: Twenty-five participants performed a suturing task on a variable tissue simulator. Electromagnetic motion sensors were used to measure their performance. The current study compares GRU and LSTM networks, which are known to perform well on other kinematic datasets, as well as MS-TCN++, which was developed for video data and was adapted in this work for motion sensors data. Finally, we extended all architectures for multi-tasking.RESULTS: In the gesture recognition task the MS-TCN++ has the highest performance with accuracy of [Formula: see text] and F1-Macro of [Formula: see text], edit distance of [Formula: see text] and F1@10 of [Formula: see text] In the tool usage recognition task for the right hand, MS-TCN++ performs the best in most metrics with an accuracy score of [Formula: see text], F1-Macro of [Formula: see text], F1@10 of [Formula: see text], and F1@25 of [Formula: see text]. The multi-task GRU performs best in all metrics in the left-hand case, with an accuracy of [Formula: see text], edit distance of [Formula: see text], F1-Macro of [Formula: see text], F1@10 of [Formula: see text], and F1@25 of [Formula: see text].CONCLUSION: In this study, using motion sensor data, we automatically identified the surgical gestures and the tools used during an open surgery suturing simulation. Our methods may be used for computing more detailed performance metrics and assisting in automatic workflow analysis. MS-TCN++ performed better in gesture recognition as well as right-hand tool recognition, while the multi-task GRU provided better results in the left-hand case. It should be noted that our multi-task GRU network is significantly smaller and has achieved competitive results in the rest of the tasks as well.

  View details for DOI 10.1007/s11548-022-02615-1

  View details for PubMedID 35419721

• A disturbing trend: An analysis of the decline in surgical critical care (SCC) fellowship training of Black and Hispanic surgeons. The journal of trauma and acute care surgery Hambrecht, A., Berry, C., DiMaggio, C., Chiu, W., Inaba, K., Frangos, S., Krowsoski, L., Greene, W. R., Issa, N., Pugh, C., Bukur, M. 2022

  Abstract

  Underrepresented minorities in medicine (URiM) are disproportionally represented in surgery training programs. Rates of URiM applying to and completing General Surgery residency remain low. We hypothesized that the patterns of URiM disparities would persist into Surgical Critical Care (SCC) fellowship applicants, matriculants and graduates.We performed a retrospective analysis of SCC applicants, matriculants and graduates from 2005-2020 using the Graduate Medical Education (GME) resident survey and analyzed applicant characteristics using the Surgical critical care and Acute care surgery Fellowship Application Service (SAFAS) from 2018-2020. The data were stratified by race/ethnicity and gender. Indicator variables were created for Asian, Hispanic, White and Black trainees. Yearly proportions for each race/ethnicity and gender categories completing or enrolling in a program were calculated and plotted over time with Loess smoothing lines and overlying 95% confidence bands. The yearly rate and statistical significance of change over time were tested with linear regression models with race/ethnicity and gender proportion as the dependent variables and year as the explanatory variable.From 2005-2020, there were a total of 2,481 graduates. Black men accounted for 4.7% of male graduates with a significant decline of 0.3% per year for the study period of those completing the fellowship (p = 0.02). Black women comprised 6.4% of female graduates and had a 0.6% decline each year (p < 0.01). A similar trend was seen with Hispanic men, who comprised 3.2% of male graduates and had a 0.3% annual decline (p = 0.02). White men had a significant increase in both matriculation to and graduation from SCC fellowships during the same interval. Similarly, Black and Hispanic applicants declined from 2019 to 2020, while the percentage of White applicants increased.Disparities in URiM representation remain omnipresent in surgery and extend from residency training to SCC fellowship. Efforts to enhance the recruitment and retention of URiM in SCC training are warranted.Level IV - Therapeutic/Care Management.

  View details for DOI 10.1097/TA.0000000000003621

  View details for PubMedID 35343928

• Diversity, equity, and inclusion in presidential leadership of academic medical and surgical societies. American journal of surgery Kearse, L. E., Goll, C. K., Jensen, R. M., Wise, B. J., Witt, A. K., Huemer, K., Korndorffer, J. R., Pugh, C. M. 2022

  Abstract

  BACKGROUND: Our aim was to identify gender and racial disparities in presidential leadership for national medical and surgical organizations.METHODS: We located publicly sourced information on national medical organizations. Years between or since the first diverse presidents were analyzed using descriptive statistics and Mann Whitney U tests.RESULTS: Sixty-seven national medical and surgical organizations were surveyed. 70.8% (n=34) diversified via gender first (White-female), whereas 26.1% (n=14) had racial diversity first. Organizations with gender diversity first followed with an African American male president sooner than organizations who first diversified by race (14.7±11.8 v. 27.6±11.3 years, p=0.018). No significant difference was observed for the third tier of diversification.CONCLUSIONS: Significant gender and racial leadership disparities in national medical organizations are still present. It is notable that organizations with female leaders had a shorter timeline to racial diversity. These findings help to inform strategies to promote and increase diversity, equity, and inclusion in national leadership.

  View details for DOI 10.1016/j.amjsurg.2022.03.028

  View details for PubMedID 35369971

• Invited Commentary: Dissecting the Mentorship Needs of Women and Ethnic Minorities in Surgery: New Opportunities Revealed. Journal of the American College of Surgeons Pugh, C. 2022; 234 (3): 261-262

  View details for DOI 10.1097/XCS.0000000000000048

  View details for PubMedID 35213487

• Examination of Intersectionality and the Pipeline for Black Academic Surgeons. JAMA surgery Keshinro, A., Butler, P., Fayanju, O., Khabele, D., Newman, E., Greene, W., Ude Welcome, A., Joseph, K., Stallion, A., Backhus, L., Frangos, S., DiMaggio, C., Berman, R., Hasson, R., Rodriguez, L. M., Stain, S., Bukur, M., Klein, M. J., Henry-Tillman, R., Barry, L., Oseni, T., Martin, C., Johnson-Mann, C., Smith, R., Karpeh, M., White, C., Turner, P., Pugh, C., Hayes-Jordan, A., Berry, C. 2022

  Abstract

  Importance: The lack of underrepresented in medicine physicians within US academic surgery continues, with Black surgeons representing a disproportionately low number.Objective: To evaluate the trend of general surgery residency application, matriculation, and graduation rates for Black trainees compared with their racial and ethnic counterparts over time.Design, Setting, and Participants: In this nationwide multicenter study, data from the Electronic Residency Application Service (ERAS) for the general surgery residency match and Graduate Medical Education (GME) surveys of graduating general surgery residents were retrospectively reviewed and stratified by race, ethnicity, and sex. Analyses consisted of descriptive statistics, time series plots, and simple linear regression for the rate of change over time. Medical students and general surgery residency trainees of Asian, Black, Hispanic or Latino of Spanish origin, White, and other races were included. Data for non-US citizens or nonpermanent residents were excluded. Data were collected from 2005 to 2018, and data were analyzed in March 2021.Main Outcomes and Measures: Primary outcomes included the rates of application, matriculation, and graduation from general surgery residency programs.Results: Over the study period, there were 71 687 applicants, 26 237 first-year matriculants, and 24 893 graduates. Of 71 687 applicants, 24 618 (34.3%) were women, 16 602 (23.2%) were Asian, 5968 (8.3%) were Black, 2455 (3.4%) were Latino, and 31 197 (43.5%) were White. Women applicants and graduates increased from 29.4% (1178 of 4003) to 37.1% (2293 of 6181) and 23.5% (463 of 1967) to 33.5% (719 of 2147), respectively. When stratified by race and ethnicity, applications from Black women increased from 2.2% (87 of 4003) to 3.5% (215 of 6181) (P<.001) while applications from Black men remained unchanged (3.7% [150 of 4003] to 4.6% [284 of 6181]). While the matriculation rate for Black women remained unchanged (2.4% [46 of 1919] to 2.3% [52 of 2264]), the matriculation rate for Black men significantly decreased (3.0% [57 of 1919] to 2.4% [54 of 2264]; P=.04). Among Black graduates, there was a significant decline in graduation for men (4.3% [85 of 1967] to 2.7% [57 of 2147]; P=.03) with the rate among women remaining unchanged (1.7% [33 of 1967] to 2.2% [47 of 2147]).Conclusions and Relevance: Findings of this study show that the underrepresentation of Black physicians at every stage in surgical training pipeline persists. Black men are especially affected. Identifying factors that address intersectionality and contribute to the successful recruitment and retention of Black trainees in general surgery residency is critical for achieving racial and ethnic as well as gender equity.

  View details for DOI 10.1001/jamasurg.2021.7430

  View details for PubMedID 35138327

• Video-based fully automatic assessment of open surgery suturing skills. International journal of computer assisted radiology and surgery Goldbraikh, A., D'Angelo, A., Pugh, C. M., Laufer, S. 1800

  Abstract

  PURPOSE: The goal of this study was to develop a new reliable open surgery suturing simulation system for training medical students in situations where resources are limited or in the domestic setup. Namely, we developed an algorithm for tools and hands localization as well as identifying the interactions between them based on simple webcam video data, calculating motion metrics for assessment of surgical skill.METHODS: Twenty-five participants performed multiple suturing tasks using our simulator. The YOLO network was modified to a multi-task network for the purpose of tool localization and tool-hand interaction detection. This was accomplished by splitting the YOLO detection heads so that they supported both tasks with minimal addition to computer run-time. Furthermore, based on the outcome of the system, motion metrics were calculated. These metrics included traditional metrics such as time and path length as well as new metrics assessing the technique participants use for holding the tools.RESULTS: The dual-task network performance was similar to that of two networks, while computational load was only slightly bigger than one network. In addition, the motion metrics showed significant differences between experts and novices.CONCLUSION: While video capture is an essential part of minimal invasive surgery, it is not an integral component of open surgery. Thus, new algorithms, focusing on the unique challenges open surgery videos present, are required. In this study, a dual-task network was developed to solve both a localization task and a hand-tool interaction task. The dual network may be easily expanded to a multi-task network, which may be useful for images with multiple layers and for evaluating the interaction between these different layers.

  View details for DOI 10.1007/s11548-022-02559-6

  View details for PubMedID 35103921

• Developing a longitudinal database of surgical skills performance for practicing surgeons: A formal feasibility and acceptance inquiry. American journal of surgery Applewhite, M. K., Kearse, L. E., Mohamadipanah, H., Witt, A., Goll, C., Wise, B., Korndorffer, J. R., Pugh, C. M. 1800

  Abstract

  BACKGROUND: We explored the feasibility and surgeons' perceptions of the utility of a longitudinal skills performance database.METHODS: A 10-station surgical skills assessment center was established at a national scientific meeting. Skills assessment volunteers (n=189) completed a survey including opinions on practicing surgeons' skills evaluation, ethics, and interest in a longitudinal database. A subset (n=23) participated in a survey-related interview.RESULTS: Nearly all participants reported interest in a longitudinal database and most believed there is an ethical obligation for such assessments to protect the public. Several interviewees specified a critical role for both formal and informal evaluation is to first create a safe and supportive environment.CONCLUSIONS: Participants support the construction of longitudinal skills databases that allow information sharing and establishment of professional standards. In a constructive environment, structured peer feedback was deemed acceptable to enhance and diversify surgeon skills. Large scale skills testing is feasible and scientific meetings may be the ideal location.

  View details for DOI 10.1016/j.amjsurg.2021.12.035

  View details for PubMedID 34998521

• Response to the Comments on "Situating Artificial Intelligence in Surgery, a Focus on Disease Severity'' Reply ANNALS OF SURGERY Pugh, C. M., Wolf, T., Korndorffer, J. R. 2021; 274 (6): E892-E893

  View details for Web of Science ID 000718532600290

• Response to the Comment on "Situating Artificial Intelligence in Surgery: A Focus on Disease Severity". Annals of surgery Pugh, C. M. 2021; 274 (6): e925-e926

  View details for DOI 10.1097/SLA.0000000000005139

  View details for PubMedID 34784677

• Response to the Comment on "Situating Artificial Intelligence in Surgery: A Focus on Disease Severity'' ANNALS OF SURGERY Pugh, C. M. 2021; 274 (6): E925-E926

  View details for Web of Science ID 000718532600329

• Surgical data science - from concepts toward clinical translation. Medical image analysis Maier-Hein, L., Eisenmann, M., Sarikaya, D., Marz, K., Collins, T., Malpani, A., Fallert, J., Feussner, H., Giannarou, S., Mascagni, P., Nakawala, H., Park, A., Pugh, C., Stoyanov, D., Vedula, S. S., Cleary, K., Fichtinger, G., Forestier, G., Gibaud, B., Grantcharov, T., Hashizume, M., Heckmann-Notzel, D., Kenngott, H. G., Kikinis, R., Mundermann, L., Navab, N., Onogur, S., RoSS, T., Sznitman, R., Taylor, R. H., Tizabi, M. D., Wagner, M., Hager, G. D., Neumuth, T., Padoy, N., Collins, J., Gockel, I., Goedeke, J., Hashimoto, D. A., Joyeux, L., Lam, K., Leff, D. R., Madani, A., Marcus, H. J., Meireles, O., Seitel, A., Teber, D., Uckert, F., Muller-Stich, B. P., Jannin, P., Speidel, S. 2021; 76: 102306

  Abstract

  Recent developments in data science in general and machine learning in particular have transformed the way experts envision the future of surgery. Surgical Data Science (SDS) is a new research field that aims to improve the quality of interventional healthcare through the capture, organization, analysis and modeling of data. While an increasing number of data-driven approaches and clinical applications have been studied in the fields of radiological and clinical data science, translational success stories are still lacking in surgery. In this publication, we shed light on the underlying reasons and provide a roadmap for future advances in the field. Based on an international workshop involving leading researchers in the field of SDS, we review current practice, key achievements and initiatives as well as available standards and tools for a number of topics relevant to the field, namely (1) infrastructure for data acquisition, storage and access in the presence of regulatory constraints, (2) data annotation and sharing and (3) data analytics. We further complement this technical perspective with (4) a review of currently available SDS products and the translational progress from academia and (5) a roadmap for faster clinical translation and exploitation of the full potential of SDS, based on an international multi-round Delphi process.

  View details for DOI 10.1016/j.media.2021.102306

  View details for PubMedID 34879287

• The Experienced Surgeon and New Tricks-It's Time for Full Adoption and Support of Automated Performance Metrics and Databases. JAMA surgery Pugh, C. M. 2021

  View details for DOI 10.1001/jamasurg.2021.4531

  View details for PubMedID 34524403

• Performance assessment using sensor technology. Journal of surgical oncology Mohamadipanah, H., Wise, B., Witt, A., Goll, C., Yang, S., Perumalla, C., Huemer, K., Kearse, L., Pugh, C. 2021; 124 (2): 200-215

  Abstract

  Over the past 30 years, there have been numerous, noteworthy successes in the development, validation, and implementation of clinical skills assessments. Despite this progress, the medical profession has barely scratched the surface towards developing assessments that capture the true complexity of hands-on skills in procedural medicine. This paper highlights the development implementation and new discoveries in performance metrics when using sensor technology to assess cognitive and technical aspects of hands-on skills.

  View details for DOI 10.1002/jso.26519

  View details for PubMedID 34245582

• SAGES consensus recommendations on an annotation framework for surgical video. Surgical endoscopy Meireles, O. R., Rosman, G., Altieri, M. S., Carin, L., Hager, G., Madani, A., Padoy, N., Pugh, C. M., Sylla, P., Ward, T. M., Hashimoto, D. A., SAGES Video Annotation for AI Working Groups 2021

  Abstract

  BACKGROUND: The growing interest in analysis of surgical video through machine learning has led to increased research efforts; however, common methods of annotating video data are lacking. There is a need to establish recommendations on the annotation of surgical video data to enable assessment of algorithms and multi-institutional collaboration.METHODS: Four working groups were formed from a pool of participants that included clinicians, engineers, and data scientists. The working groups were focused on four themes: (1) temporal models, (2) actions and tasks, (3) tissue characteristics and general anatomy, and (4) software and data structure. A modified Delphi process was utilized to create a consensus survey based on suggested recommendations from each of the working groups.RESULTS: After three Delphi rounds, consensus was reached on recommendations for annotation within each of these domains. A hierarchy for annotation of temporal events in surgery was established.CONCLUSIONS: While additional work remains to achieve accepted standards for video annotation in surgery, the consensus recommendations on a general framework for annotation presented here lay the foundation for standardization. This type of framework is critical to enabling diverse datasets, performance benchmarks, and collaboration.

  View details for DOI 10.1007/s00464-021-08578-9

  View details for PubMedID 34231065

• From Listening to Action: Academic Surgcial Departmental Response to Social Injustice Through Curricular Development. Annals of surgery Korndorffer, J. R., Wren, S. M., Pugh, C. M., Hawn, M. T. 2021

  Abstract

  OBJECTIVES: To describe the development and evaluation of a structured department wide cultural competency curriculum.SUMMARY BACKGROUND DATA: Despite numerous organizational policies and statements, social injustice and bias still exists. Our department committed to assist individuals of the entire department to develop foundational knowledge and skills to combat implicit bias and systemic racism through the creation of a cultural competency curriculum. The purpose of this manuscript is to detail our curriculum and the evaluation of its effectiveness.METHODS: Using a well-established curriculum development framework, a cultural competency curriculum was developed focusing on knowledge, skills and attitudes at the individual level, for all members of the department. The curriculum was implemented through six, hour-long sessions over a nine-week period. Effectiveness was assessed through a post curriculum survey.RESULTS: 20% of the respondents had experienced bias based on race, ethnicity or sexual orientation in the past 12 months while 30% had experienced bias based on gender. 71% independently explored related topics. The curriculum was overall well received and generally achieved the goals and objectives.CONCLUSIONS: Using a standard curriculum development framework, an effective department-wide cultural competency curriculum can be developed and implemented.

  View details for DOI 10.1097/SLA.0000000000004891

  View details for PubMedID 33856378

• Response to: Comments on "Situating Artificial Intelligence in Surgery, a Focus on Disease Severity". Annals of surgery Pugh, C. M., Wolf, T., Korndorffer, J. R. 2021

  View details for DOI 10.1097/SLA.0000000000004820

  View details for PubMedID 33630431

• Reassessing career pathways of surgical leaders: An examination of surgical leaders' early accomplishments. American journal of surgery Meer, E. n., Hughes, B. D., Martin, C. A., Rios-Diaz, A. J., Patel, V. n., Pugh, C. M., Berry, C. n., Stain, S. C., Britt, L. D., Stein, S. L., Butler, P. D. 2021

  Abstract

  The American College of Surgeon (ACS), American Surgical Association (ASA), Association of Women Surgeons (AWS), and Society of Black Academic Surgeons (SBAS) partnered to gain insight into whether inequities found in surgical society presidents may be present earlier.ACS, ASA, AWS, and SBAS presidents' CVs were assessed for demographics and scholastic achievements at the time of first faculty appointment. Regression analyses controlling for age were performed to determine relative differences across societies.66 of the 68 presidents' CVs were received and assessed (97% response rate). 50% of AWS future presidents were hired as Instructors rather than Assistant professors, compared to 29.4% of SBAS, 25% of ASA and 29.4% of ACS. The future ACS, ASA, and SBAS presidents had more total publications than the AWS presidents, but similar numbers of 1st and Sr. author publications.Gender inequities in academic surgeon hiring practices and perceived scholastic success may be present at first hire.

  View details for DOI 10.1016/j.amjsurg.2021.04.012

  View details for PubMedID 33894978

• Can Deep Learning Algorithms Help Identify Surgical Workflow and Techniques? The Journal of surgical research Mohamadipanah, H., Kearse, L., Witt, A., Wise, B., Yang, S., Goll, C., Pugh, C. 2021; 268: 318-325

  Abstract

  Surgical videos are now being used for performance review and educational purposes; however, broad use is still limited due to time constraints. To make video review more efficient, we implemented Artificial Intelligence (AI) algorithms to detect surgical workflow and technical approaches.Participants (N = 200) performed a simulated open bowel repair. The operation included two major phases: (1) Injury Identification and (2) Suture Repair. Accordingly, a phase detection algorithm (MobileNetV2+GRU) was implemented to automatically detect the two phases using video data. In addition, participants were noted to use three different technical approaches when running the bowel: (1) use of both hands, (2) use of one hand and one tool, or (3) use of two tools. To discern the three technical approaches, an object detection (YOLOv3) algorithm was implemented to recognize objects that were commonly used during the Injury Identification phase (hands versus tools).The phase detection algorithm achieved high precision (recall) when segmenting the two phases: Injury Identification (86 ± 9% [81 ± 12%]) and Suture Repair (81 ± 6% [81 ± 16%]). When evaluating three technical approaches in running the bowel, the object detection algorithm achieved high average precisions (Hands [99.32%] and Tools [94.47%]). The three technical approaches showed no difference in execution time (Kruskal-Wallis Test: P= 0.062) or injury identification (not missing an injury) (Chi-squared: P= 0.998).The AI algorithms showed high precision when segmenting surgical workflow and identifying technical approaches. Automation of these techniques for surgical video databases has great potential to facilitate efficient performance review.

  View details for DOI 10.1016/j.jss.2021.07.003

  View details for PubMedID 34399354

• How Wearable Technology Can Facilitate AI Analysis of Surgical Videos. Annals of surgery open : perspectives of surgical history, education, and clinical approaches Pugh, C. M., Ghazi, A., Stefanidis, D., Schwaitzberg, S. D., Martino, M. A., Levy, J. S. 2020; 1 (2): e011

  Abstract

  Operative video has great potential to enable instant replays of critical surgical decisions for training and quality review. Recently, artificial intelligence (AI) has shown early promise as a method of enabling efficient video review, analysis, and segmentation. Despite the progress with AI analysis of surgical videos, more work needs to be done to improve the accuracy and efficiency of AI-driven video analysis. At a recent consensus conference held on July 10-11, 2020, 8 research teams shared their work using AI for surgical video analysis. Four of the teams showcased the utility of wearable technology in providing objective surgical metrics. Data from these technologies were shown to pinpoint important cognitive and motor actions during operative tasks and procedures. The results support the utility of wearable technology to facilitate efficient and accurate video analysis and segmentation.

  View details for DOI 10.1097/AS9.0000000000000011

  View details for PubMedID 37637444

  View details for PubMedCentralID PMC10455149

• Situating Artificial Intelligence in Surgery A Focus on Disease Severity ANNALS OF SURGERY Korndorffer, J. R., Hawn, M. T., Spain, D. A., Knowlton, L. M., Azagury, D. E., Nassar, A. K., Lau, J. N., Arnow, K. D., Trickey, A. W., Pugh, C. M. 2020; 272 (3): 523–28

  View details for DOI 10.1097/SLA.0000000000004207

  View details for Web of Science ID 000589824900056

• Situating Artificial Intelligence in Surgery: A Focus on Disease Severity. Annals of surgery Korndorffer, J. R., Hawn, M. T., Spain, D. A., Knowlton, L. M., Azagury, D. E., Nassar, A. K., Lau, J. N., Arnow, K. D., Trickey, A. W., Pugh, C. M. 2020; 272 (3): 523-528

  Abstract

  Artificial intelligence (AI) has numerous applications in surgical quality assurance. We assessed AI accuracy in evaluating the critical view of safety (CVS) and intraoperative events during laparoscopic cholecystectomy. We hypothesized that AI accuracy and intraoperative events are associated with disease severity.One thousand fifty-one laparoscopic cholecystectomy videos were annotated by AI for disease severity (Parkland Scale), CVS achievement (Strasberg Criteria), and intraoperative events. Surgeons performed focused video review on procedures with ≥1 intraoperative events (n = 335). AI versus surgeon annotation of CVS components and intraoperative events were compared. For all cases (n = 1051), intraoperative-event association with CVS achievement and severity was examined using ordinal logistic regression.Using AI annotation, surgeons reviewed 50 videos/hr. CVS was achieved in ≤10% of cases. Hepatocystic triangle and cystic plate visualization was achieved more often in low-severity cases (P < 0.03). AI-surgeon agreement for all CVS components exceeded 75%, with higher agreement in high-severity cases (P < 0.03). Surgeons agreed with 99% of AI-annotated intraoperative events. AI-annotated intraoperative events were associated with both disease severity and number of CVS components not achieved. Intraoperative events occurred more frequently in high-severity versus low-severity cases (0.98 vs 0.40 events/case, P < 0.001).AI annotation allows for efficient video review and is a promising quality assurance tool. Disease severity may limit its use and surgeon oversight is still required, especially in complex cases. Continued refinement may improve AI applicability and allow for automated assessment.

  View details for DOI 10.1097/SLA.0000000000004207

  View details for PubMedID 33759839

• Quantifying Performance Decline in the Operating Room Using fNIRS. Annals of surgery Pugh, C. M. 2020

  View details for DOI 10.1097/SLA.0000000000004196

  View details for PubMedID 32657931

• Evaluating how residents talk and what it means for surgical performance in the simulation lab AMERICAN JOURNAL OF SURGERY D'angelo, A. D., Ruis, A. R., Collier, W., Shaffer, D., Pugh, C. M. 2020; 220 (1): 37–43

  Abstract

  This paper explores a method for assessing intraoperative performance by modeling how surgeons integrate skills and knowledge through discourse.Senior residents (N = 11) were recorded while performing a simulated laparoscopic ventral hernia (LVH) repair. Audio transcripts were coded for five discourse elements related to knowledge, skills, and operative independence. Epistemic network analysis was used to model the ordered integration of the five discourse elements.Participants with poorer hernia repair outcomes had stronger connections between the discourse elements operative planning and asking for information or advice (Operative planning), while participants with better hernia repair outcomes had stronger connections between the discourse elements giving assistant instructions and identifying errors (Operative management): (p = .006; Cohen's d = 2.79).Participants with better hernia repair outcomes engaged in more operative management communication during the simulated procedure. This ability to integrate multiple operative steps and verbally communicate them significantly correlated with better operative outcomes.

  View details for DOI 10.1016/j.amjsurg.2020.02.016

  View details for Web of Science ID 000545562900002

  View details for PubMedID 32093868

• The what? How? And who? Of video based assessment. American journal of surgery Pugh, C. M., Hashimoto, D. A., Korndorffer, J. R. 2020

  Abstract

  BACKGROUND: Currently, there is significant variability in the development, implementation and overarching goals of video review for assessment of surgical performance.METHODS: This paper evaluates the current methods in which video review is used for evaluation of surgical performance and identifies which processes are critical for successful, widespread implementation of video-based assessment.RESULTS: Despite the advances in video capture technology and growing interest in video-based assessment, there is a notable gap in the implementation and longitudinal use of formative and summative assessment using video.CONCLUSION: Validity, scalability and discoverability are current but removable barriers to video-based assessment.

  View details for DOI 10.1016/j.amjsurg.2020.06.027

  View details for PubMedID 32665080

• Does the location of short-arm cast univalve effect pressure of the three-point mould? Journal of children's orthopaedics Montgomery, B. K., Perrone, K. H., Yang, S., Segovia, N. A., Rinsky, L., Pugh, C. M., Frick, S. L. 2020; 14 (3): 236–40

  Abstract

  Purpose: Forearm and distal radius fractures are among the most common fractures in children. Many fractures are definitively treated with closed reduction and casting, however, the risk for re-displacement is high (7% to 39%). Proper cast application and the three-point moulding technique are modifiable factors that improve the ability of a cast to maintain the fracture reduction. Many providers univalve the cast to accommodate swelling. This study describes how the location of the univalve cut impacts the pressure at three-point mould sites for a typical dorsally displaced distal radius fracture.Methods: We placed nine force-sensing resistors on an arm model to collect pressure data at the three-point mould sites. Sensory inputs were sampled at 15 Hz. Cast padding and a three-point moulded short arm fibreglass cast was applied. The cast was then univalved on the dorsal, volar, radial or ulnar aspect. Pressure recordings were obtained throughout the procedure.Results: A total of 24 casts were analyzed. Casts univalved in the sagittal plane (dorsal or volar surface) retained up to 16% more pressure across the three moulding sites compared with casts univalved in the coronal plane (radial or ulnar border).Conclusion: Maintaining pressure at the three-point mould prevents loss of reduction at the fracture site. This study shows that univalving the cast dorsally or volarly results in less pressure loss at moulding sites. This should improve the chances of maintaining fracture reductions when compared with radial or ulnar cuts in the cast. Sagittal plane univalving of forearm casts is recommended.

  View details for DOI 10.1302/1863-2548.14.200034

  View details for PubMedID 32582392

• Sensors and Psychomotor Metrics: A Unique Opportunity to Close the Gap on Surgical Processes and Outcomes. ACS biomaterials science & engineering Mohamadipanah, H., Perrone, K. H., Peterson, K., Nathwani, J., Huang, F., Garren, A., Garren, M., Witt, A., Pugh, C. 2020; 6 (5): 2630-2640

  Abstract

  The surgical process remains elusive to many. This paper presents two independent empirical investigations where psychomotor skill metrics were used to quantify elements of the surgical process in a procedural context during surgical tasks in a simulated environment. The overarching goal of both investigations was to address the following hypothesis: Basic motion metrics can be used to quantify specific aspects of the surgical process including instrument autonomy, psychomotor efficiency, procedural readiness, and clinical errors. Electromagnetic motion tracking sensors were secured to surgical trainees' (N = 64) hands for both studies, and several motion metrics were investigated as a measure of surgical skill. The first study assessed performance during a bowel repair and laparoscopic ventral hernia (LVH) repair in comparison to a suturing board task. The second study assessed performance in a VR task in comparison to placement of a subclavian central line. The findings of the first study support our subhypothesis that motion metrics have a generalizable application to surgical skill by showing significant correlations in instrument autonomy and psychomotor efficiency during the suturing task and bowel repair (idle time: r = 0.46, p < 0.05; average velocity: r = 0.57, p < 0.05) and the suturing task and LVH repair (jerk magnitude: r = 0.36, p < 0.05; bimanual dexterity: r = 0.35, p < 0.05). In the second study, performance in VR (steering and jerkiness) correlated to clinical errors (r = 0.58, p < 0.05) and insertion time (r = 0.55, p < 0.05) in placement of a subclavian central line. Both gross (dexterity) and fine motor skills (steering) were found to be important as well as efficiency (i.e., idle time, duration, velocity) when seeking to understand the quality of surgical performance. Both studies support our hypotheses that basic motion metrics can be used to quantify specific aspects of the surgical process and that the use of different technologies and metrics are important for comprehensive investigations of surgical skill.

  View details for DOI 10.1021/acsbiomaterials.9b01019

  View details for PubMedID 33463275

• Sensors and Psychomotor Metrics: A Unique Opportunity to Close the Gap on Surgical Processes and Outcomes ACS BIOMATERIALS SCIENCE & ENGINEERING Mohamadipanah, H., Perrone, K. H., Peterson, K., Nathwani, J., Huang, F., Garren, A., Garren, M., Witt, A., Pugh, C. 2020; 6 (5): 2630–40

  View details for DOI 10.1021/acsbiomaterials.9b01019

  View details for Web of Science ID 000535188500014

• Translating motion tracking data into resident feedback: An opportunity for streamlined video coaching Perrone, K. H., Yang, S., Mohamadipanah, H., Wise, B., Witt, A., Goll, C., Pugh, C. EXCERPTA MEDICA INC-ELSEVIER SCIENCE INC. 2020: 552–56

  Abstract

  We hypothesized that differences in motion data during a simulated laparoscopic ventral hernia repair (LVH) can be used to stratify top and lower tier performers and streamline video review.Surgical residents (N = 94) performed a simulated partial LVH repair while wearing motion tracking sensors. We identified the top ten and lower ten performers based on a final product quality score (FPQS) of the repair. Two blinded raters independently reviewed motion plots to identify patterns and stratify top and lower tier performers.Top performers had significantly higher FPQS (23.3 ± 1.2 vs 5.7 ± 1.6 p < 0.01). Raters identified patterns and stratified top performers from lower tier performers (Rater 1 χ2 = 3.2 p = 0.07 and Rater 2 χ2 = 2.0 p = 0.16). During video review, we correlated motion plots with the relevant portion of the procedure.Differences in motion data can identify learning needs and enable rapid review of surgical videos for coaching.

  View details for DOI 10.1016/j.amjsurg.2020.01.032

  View details for Web of Science ID 000525802700004

  View details for PubMedID 32014295

• A Call to Action: Black/African American Women Surgeon Scientists, Where are They? Annals of surgery Berry, C., Khabele, D., Johnson-Mann, C., Henry-Tillman, R., Joseph, K., Turner, P., Pugh, C., Fayanju, O. M., Backhus, L., Sweeting, R., Newman, E. A., Oseni, T., Hasson, R. M., White, C., Cobb, A., Johnston, F. M., Stallion, A., Karpeh, M., Nwariaku, F., Rodriguez, L. M., Jordan, A. H. 2020

  Abstract

  OBJECTIVE: To determine the representation of Black/AA women surgeons in academic medicine among U.S. medical school faculty and to assess the number of NIH grants awarded to Black/AA women surgeon-scientists over the past 2 decades.SUMMARY OF BACKGROUND DATA: Despite increasing ethnic/racial and sex diversity in U.S. medical schools and residencies, Black/AA women have historically been underrepresented in academic surgery.METHODS: A retrospective review of the Association of American Medical Colleges 2017 Faculty Roster was performed and the number of grants awarded to surgeons from the NIH (1998-2017) was obtained. Data from the Association of American Medical Colleges included the total number of medical school surgery faculty, academic rank, tenure status, and department Chair roles. Descriptive statistics were performed.RESULTS: Of the 15,671 U.S. medical school surgical faculty, 123 (0.79%) were Black/AA women surgeons with only 11 (0.54%) being tenured faculty. When stratified by academic rank, 15 (12%) Black/AA women surgeons were instructors, 73 (59%) were assistant professors, 19 (15%) were associate professors, and 10 (8%) were full professors of surgery. Of the 372 U.S. department Chairs of surgery, none were Black/AA women. Of the 9139 NIH grants awarded to academic surgeons from 1998 and 2017, 31 (0.34%) grants were awarded to fewer than 12 Black/AA women surgeons.CONCLUSION: A significant disparity in the number of Black/AA women in academic surgery exists with few attaining promotion to the rank of professor with tenure and none ascending to the role of department Chair of surgery. Identifying and removing structural barriers to promotion, NIH grant funding, and academic advancement of Black/AA women as leaders and surgeon-scientists is needed.

  View details for DOI 10.1097/SLA.0000000000003786

  View details for PubMedID 32209893

• The Society of Black Academic Surgeons CV benchmarking initiative: Early career trends of academic surgical leaders. American journal of surgery Hughes, B. D., Butler, P. D., Edwards, M. A., Pugh, C. M., Martin, C. A. 2020

  Abstract

  BACKGROUND: Surgeons from under-represented backgrounds are less likely to receive academic tenure and obtain leadership positions. Our objective was to query the curriculum vitaes (CVs) of SBAS leadership to develop a benchmarking tool to promote and guide careers in academic surgery.METHODS: CVs from academic leaders were reviewed for academic productivity at early career stages-the first 5-and 10-years. Variables queried: peer-reviewed publications, grant funding, surgical societal involvement, invited lectureships and visiting professorships.RESULTS: Of 20 CVs, 41 leadership positions including 13 SBAS Presidents were identified. At 5- and 10-years, respectively, the academic productivity increased: 20.6 and 52.3 publications; 4.7 and 9.7 grants; 18 and 42.6 lectures/professorships.CONCLUSION: The CV benchmarking tool may be a useful framework for aspiring academic surgeons to track their progress relative to successful SBAS members. Creative strategies like these, paired with faculty mentorship and sponsorship are necessary to improve the ethnic diversity in academic surgery.

  View details for DOI 10.1016/j.amjsurg.2020.01.047

  View details for PubMedID 32147021

• Benchmarking Accomplishments of Leaders in American Surgery and Justification for Enhancing Diversity and Inclusion. Annals of surgery Butler, P. D., Pugh, C. M., Meer, E. n., Lett, L. A., Tilahun, E. D., Sanfey, H. A., Berry, C. n., Stain, S. C., DeMatteo, R. P., Vickers, S. M., Britt, L. D., Martin, C. A. 2020

  Abstract

  To comprehensively assess the level of achievement and demographics of national surgical society presidents.Data on the accomplishments needed to rise to positions of national surgical leadership is scarce and merit alone does not always yield such opportunities. Recognizing the shortcomings of sex and ethnic diversity within academic surgical leadership, the American College of Surgeon (ACS), American Surgical Association (ASA), Association of Women Surgeons (AWS), and the Society of Black Academic Surgeons (SBAS) partnered to address these challenges by performing a comprehensive assessment of their presidents over the last 16 years.ACS, ASA, AWS, and SBAS presidents' CVs, at the time of their presidential term, were assessed for demographics and scholastic achievements. Regression analyses controlling for age were performed to determine relative differences across societies.A total of 62 of the 64 presidents' CVs were received and assessed (97% response rate). There was a large discrepancy in the average age in years of ACS (70) and ASA (66) presidents compared to the AWS (51) and SBAS (53) presidents. For the ACS and ASA cohort, 87% were male and 83% were White, collectively. After controlling for age (52), the AWS and SBAS presidents' scholastic achievements were comparable to the ACS (and ASA) cohort in 9 and 12 of the 15 accessed metrics, respectively.The ACS and ASA presidents' CVs displayed unsurpassed scholastic achievement, and although not equivalent, both the AWS and the SBAS presidents had comparable attainment. These findings further substantiate that women and ethnic minority surgeons are deserving of additional national leadership consideration as organized medicine pursues a more diverse and reflective physician workforce.

  View details for DOI 10.1097/SLA.0000000000004151

  View details for PubMedID 32649466

• The Role of Race and Gender in the Career Experiences of Black/African-American Academic Surgeons: A Survey of the Society of Black Academic Surgeons and a Call to Action. Annals of surgery Crown, A. n., Berry, C. n., Khabele, D. n., Fayanju, O. M., Cobb, A. n., Backhus, L. n., Smith, R. n., Sweeting, R. n., Hasson, R. n., Johnson-Mann, C. n., Oseni, T. n., Newman, E. A., Turner, P. n., Karpeh, M. n., Pugh, C. n., Jordan, A. H., Henry-Tillman, R. n., Joseph, K. A. 2020

  Abstract

  To determine the role of race and gender in the career experience of Black/AA academic surgeons and to quantify the prevalence of experience with racial and gender bias stratified by gender.Compared to their male counterparts, Black/African American (AA) women remain significantly underrepresented among senior surgical faculty and department leadership. The impact of racial and gender bias on the academic and professional trajectory of Black/AA women surgeons has not been well-studied.A cross-sectional survey regarding demographics, employment, and perceived barriers to career advancement was distributed via email to faculty surgeon members of the Society of Black American Surgeons (SBAS) in September 2019.Of 181 faculty members, 53 responded (29%), including 31 women (58%) and 22 men (42%). Academic positions as a first job were common (men 95% vs women 77%, p = 0.06). Men were more likely to attain the rank of full professor (men 45% vs women 7%, p = 0.01). Reports of racial bias in the workplace were similar (women 84% vs men 86%, NS); however, reports of gender bias (women 97% vs men 27%, p < 0.001) and perception of salary inequities (women 89% vs 63%, p = 0.02) were more common among women.Despite efforts to increase diversity, high rates of racial bias persist in the workplace. Black/AA women also report experiencing a high rate of gender bias and challenges in academic promotion.

  View details for DOI 10.1097/SLA.0000000000004502

  View details for PubMedID 32941287

• Multi-Modal Cardiopulmonary Bypass Skills Assessment within a High-Fidelity Simulation Environment. The Annals of thoracic surgery Hermsen, J. L., Mohamadipanah, H. n., Yang, S. n., Wise, B. n., Fiedler, A. n., DiMusto, P. n., Pugh, C. n. 2020

  Abstract

  A high fidelity simulator that utilizes a perfused porcine heart, cannulae, and tubing has demonstrated to be a useful training adjunct. We hypothesized that multi-modal assessment of cardiopulmonary bypass skills within this high fidelity simulated environment could discern expert from trainee performance.Three traditional fellows (PGY 6-8) and three attending surgeons each performed three aortic cannulations. The third sequence included venous cannulation, commencement of cardiopulmonary bypass (CPB) and placement of a cardioplegia catheter and aortic crossclamp (XC). Performance across 20 cognitive and 21 technical domains were evaluated. Surgeon and assistant hand movements and economy of motion were assessed by electromagnetic motion sensors worn under sterile gloves.Analysis showed a significant difference in cognitive (6.7±2.3 vs 4.6±2.7, p=0.03) but not technical (6.2±2.5 vs 5.8±2.2, p=0.7) scores favoring the experts. In addition, experts showed higher efficiency by spending 64±14s to construct a non-pledgeted aortic pursestring suture and secure it with a Rummel while trainees spent 82±30s to complete this task (p=0.03). Motion analysis revealed non-different path lengths between experts and trainees for cannulation and CPB, but significantly shorter path for experts in XC (47.5±15.5m vs 91.9±20.3m, p=0.04).Multi-modal assessment using cognitive, technical and motion analysis of basic cardiopulmonary bypass tasks using a high-fidelity simulation environment is a valid system to measure performance and discriminate experts from trainees. This construct may allow for development of 'competence thresholds' with important implications for training and certification in cardiothoracic surgery.

  View details for DOI 10.1016/j.athoracsur.2020.07.022

  View details for PubMedID 32971063

• Use of sensors to quantify procedural idle time: Validity evidence for a new mastery metric. Surgery Perrone, K. H., Yang, S., Wise, B., Witt, A., Goll, C., Dawn, S., Eichhorn, W., Mohamadipanah, H., Pugh, C. 2019

  Abstract

  BACKGROUND: Quantification of mastery is the first step in using objective metrics for teaching. We hypothesized that during orotracheal intubation, top tier performers have less idle time compared to lower tier performers.METHODS: At the Anesthesiology 2018 Annual Meeting, 82 participants intubated a normal airway simulator and a burnt airway simulator. The movements of the participant's laryngoscope were quantified using electromagnetic motion sensors. Top tier performers were defined as participants who intubated both simulators successfully in less than the median time for each simulator. Idle time was defined as the duration of time when the laryngoscope was not moving.RESULTS: Top performers showed less Idle Time when intubating the normal airway compared to lower tier performers (14.5 ± 9.8 seconds vs 34.0 ± 52.0 seconds, respectively P < .01). Likewise, top performers showed less Idle Time when intubating the burnt airway compared to lower tier performers (18.6 ± 15.2 seconds vs 63.4 ± 59.11 seconds; P < .01). Comparing performance on the burnt airway to the normal airway, there was a difference for lower tier performers (63.4 ± 59.1 seconds vs 34.0 ± 52.0 seconds; P < .01) but not for top tier performers (18.6 ± 15.2 seconds vs 14.5 ± 9.8 seconds; P= .07).CONCLUSION: Similar to our previous findings with other procedures, Idle Time was shown to have known group validity evidence when comparing top performers with lower tier performers. Further, Idle Time was correlated with procedure difficulty in our prior work. We observed statistically significant differences in Idle Times for lower tier performers when comparing the normal airway to the burnt airway but not for top tier performers. Our findings support the continued exploration of Idle Time for development of objective assessment and curricula.

  View details for DOI 10.1016/j.surg.2019.09.016

  View details for PubMedID 31708084

• In Search of Characterizing Surgical Skill JOURNAL OF SURGICAL EDUCATION Azari, D., Greenberg, C., Pugh, C., Wiegmann, D., Radwin, R. 2019; 76 (5): 1348–63

  View details for DOI 10.1016/j.jsurg.2019.02.010

  View details for Web of Science ID 000487574300022

• Screening surgical residents' laparoscopic skills using virtual realitytasks: Who needs more time in the sim lab? Surgery Mohamadipanah, H., Perrone, K. H., Nathwani, J., Parthiban, C., Peterson, K., Wise, B., Garren, A., Pugh, C. 2019

  Abstract

  BACKGROUND: This study investigated the possibility of using virtual reality perceptual-motor tasks as a screening tool for laparoscopic ability. We hypothesized that perceptual-motor skills assessed using virtual reality will correlate with the quality of simulated laparoscopic ventral hernia repair.MATERIALS AND METHODS: Surgical residents (N= 37), performed 2 virtual reality perceptual-motor tasks: (1) force matching and (2) target tracking. Participants also performed a laparoscopic ventral hernia repair on a simulator and final product quality score, and endoscopic visualization errors were calculated. Correlational analysis was performed to assess the relationship between performance on virtual reality tasks and laparoscopic ventral hernia repair.RESULTS: Residents with poor performance on force matching in virtual reality-"peak deflection" (r= -0.34, P < .05) and "summation distance" (r= -0.36, P < .05)-had lower final product quality scores. Likewise, poor performance in virtual reality-based target tracking-"path length" (r= -0.49, P < .05) and "maximum distance" (r= -0.37, P < .05)-correlated with a lower final product quality score.CONCLUSION: Our findings support the notion that virtual reality could be used as a screening tool for perceptual-motor skill. Trainees identified as having poor perceptual-motor skill can benefit from focused curricula, allowing them to hone personal areas of weakness and maximize technical skill.

  View details for DOI 10.1016/j.surg.2019.04.013

  View details for PubMedID 31229312

• Teaching practicing surgeons what not to do: An analysis of instruction fluidity during a simulation-based continuing medical education course Godfrey, M., Rosser, A. A., Pugh, C. M., Shaffer, D., Sachdeva, A. K., Jung, S. A. MOSBY-ELSEVIER. 2019: 1082–87

  View details for DOI 10.1016/j.surg.2019.01.016

  View details for Web of Science ID 000472984500005

• Advanced Volumetric 3-Dimensional Visualization of Surgical Anatomy-Are We There Yet? JAMA surgery Pugh, C. M. 2019

  View details for DOI 10.1001/jamasurg.2019.1169

  View details for PubMedID 31141145

• In Search of Characterizing Surgical Skill. Journal of surgical education Azari, D., Greenberg, C., Pugh, C., Wiegmann, D., Radwin, R. 2019

  Abstract

  OBJECTIVE: This paper provides a literature review and detailed discussion of surgical skill terminology. Culminating in a novel model that proposes a set of unique definitions, this review is designed to facilitate shared understanding to study and develop metrics quantifying surgical skill.DESIGN: Objective surgical skill analysis depends on consistent definitions and shared understanding of terms like performance, expertise, experience, aptitude, ability, competency, and proficiency.STRUCTURE: Each term is discussed in turn, drawing from existing literature and colloquial uses.IMPLICATIONS: A new model of definitions is proposed to cement a common and consistent lexicon for future skills analysis, and to quantitatively describe a surgeon's performance throughout their career.

  View details for PubMedID 30890315

• Teaching practicing surgeons what not to do: An analysis of instruction fluidity during a simulation-based continuing medical education course. Surgery Godfrey, M., Rosser, A. A., Pugh, C. M., Shaffer, D. W., Sachdeva, A. K., Jung, S. A. 2019

  Abstract

  BACKGROUND: Interest is growing in simulation-based continuing medical education courses for practicing surgeons. However, little research has explored the instruction employed during these courses. This study examines instruction practices used during an annual simulation-based continuing medical education course.METHODS: Audio-video data were collected from surgeon instructors (n= 12) who taught a simulated laparoscopic hernia repair continuing medical education course across 2 years. Surgeon learners (n= 58) were grouped by their self-reported laparoscopic and hernia repair experience. Instructors' transcribed dialogue was automatically coded for 5 types of responses to the following questions: anecdotes, confirming, correcting, guidance, and what not to do. Differences in these responses were measured against the progress of the simulations and across learners with different experience levels. Postcourse interviews with instructors were conducted for additional qualitative validation.RESULTS: Performing t tests of instructor responses revealed that they were significantly more likely to answer in forms coded as anecdotes when responding to relative experts and in forms coded as what not to do when responding to novices. Linear regressions of each code against normalized progressions of each simulation revealed a significant relationship between progression through a simulation and frequency of the what not to do code for less-experienced learners. Postcourse interviews revealed that instructors continuously assess participants throughout a session and modify their teaching strategies.CONCLUSION: Instructors significantly modified the focus of their teaching as a function both of their learners' self-reported experience levels, their assessment of learner needs, and learner progression through the training sessions.

  View details for PubMedID 30876670

• Electronic health records, physician workflows and system change: defining a pathway to better healthcare. Annals of translational medicine Pugh, C. M. 2019; 7 (Suppl 1): S27

  View details for DOI 10.21037/atm.2019.01.83

  View details for PubMedID 31032307

  View details for PubMedCentralID PMC6462584

• Electronic health records, physician workflows and system change: defining a pathway to better healthcare ANNALS OF TRANSLATIONAL MEDICINE Pugh, C. M. 2019; 7

  View details for DOI 10.21037/atm.2019.01.83

  View details for Web of Science ID 000462507800027

• What do you want to know? Operative experience predicts the type of questions practicing surgeons ask during a CME laparoscopic hernia repair course AMERICAN JOURNAL OF SURGERY Godfrey, M., Rosser, A. A., Pugh, C. M., Sachdeva, A. K., Sullivan, S. 2019; 217 (2): 382-386

  View details for DOI 10.1016/j.amjsurg.2018.11.027

  View details for Web of Science ID 000454787500032

• Surgical procedural map scoring for decision-making in laparoscopic cholecystectomy AMERICAN JOURNAL OF SURGERY Hashimoto, D. A., Axelsson, C., Jones, C. B., Phitayakorn, R., Petrusa, E., McKinley, S. K., Gee, D., Pugh, C. 2019; 217 (2): 356-361

  View details for DOI 10.1016/j.amjsurg.2018.11.011

  View details for Web of Science ID 000454787500028

• Can VR Be Used to Track Skills Decay During the Research Years? The Journal of surgical research Mohamadipanah, H. n., Perrone, K. n., Peterson, K. n., Garren, M. n., Parthiban, C. n., Sunkara, A. n., Zinn, M. n., Pugh, C. n. 2019

  Abstract

  Time away from surgical practice can lead to skills decay. Research residents are thought to be prone to skills decay, given their limited experience and reduced exposure to clinical activities during their research training years. This study takes a cross-sectional approach to assess differences in residents' skills at the beginning and end of their research years using virtual reality. We hypothesized that research residents will have measurable decay in psychomotor skills when evaluated using virtual reality.Surgical residents (n = 28) were divided into two groups; the first group was just beginning their research time (clinical residents: n = 19) and the second group (research residents: n = 9) had just finished at least 2 y of research. All participants were asked to perform a target-tracking task using a haptic device, and their performance was compared using Welch's t-test.Research residents showed a higher level of "tracking error" (1.69 ± 0.44 cm versus 1.40 ± 0.19 cm; P = 0.04) and a similar level of "path length" (62.5 ± 10.5 cm versus 62.1 ± 5.2 cm; P = 0.92) when compared with clinical residents.The increased "tracking error" among residents at the end of their research time suggests fine psychomotor skills decay in residents who spend time away from clinical duties during laboratory time. This decay demonstrates the need for research residents to regularly participate in clinical activities, simulation, or assessments to minimize and monitor skills decay while away from clinical practice. Additional longitudinal studies may help better map learning and decay curves for residents who spend time away from clinical practice.

  View details for DOI 10.1016/j.jss.2019.10.030

  View details for PubMedID 31776024

• Dynamic Visual Feedback During Junctional Tourniquet Training JOURNAL OF SURGICAL RESEARCH Xu, J., Kwan, C., Sunkara, A., Mohamadipanah, H., Bell, K., Tizale, M., Pugh, C. M. 2019; 233: 444-452

  View details for DOI 10.1016/j.jss.2018.08.044

  View details for Web of Science ID 000451335500062

• Simulation and High-Stakes Assessment CLINICAL SIMULATION: EDUCATION, OPERATIONS AND ENGINEERING, 2ND EDITION Lau, J. N., Korndorffer, J. R., Pugh, C. M., Chiniara, G. 2019: 879-888

  View details for DOI 10.1016/B978-0-12-815657-5.00059-0

  View details for Web of Science ID 000796816800059

• Use of error management theory to quantify and characterize residents' error recovery strategies. American journal of surgery Pugh, C. M., Law, K. E., Cohen, E. R., D'Angelo, A. D., Greenberg, J. A., Greenberg, C. C., Wiegmann, D. A. 2019

  Abstract

  Traditional checklist metrics for surgical performance can miss key intraoperative decisions that impact procedural outcomes. Error-based assessments may help identify important metrics for evaluating operative performance and resident readiness for independent practice.This study utilized human factors error analysis and error management theory to investigate a previously collected video database of resident performance during a simulated laparoscopic ventral hernia (LVH) repair on a table-top simulator using standard laparoscopic tools and mesh. Errors were deconstructed and coded using a structured observation tool and video analysis software. Error detection events and error recovery events were categorized for each operative step of the ventral hernia repair.Residents made a total of 314 errors (M = 15.7, SD = 4.96). There were more technical errors (63%) than cognitive errors (37%) and more commission errors (69%) than omission errors (30%). Almost half (47%) of all errors went completely undetected by the residents for the entire LVH repair. Of the errors that residents attempted to recover (n = 136), 86.0% were successfully recovered. Technical errors were four times more likely to be successfully recovered than cognitive errors (p = .020).Our results revealed specific details regarding residents' error management strategies and provides validity evidence for the use of human factors error frameworks in surgical performance assessments. Practice in simulation-based learning environments may improve resident decision-making and error management opportunities by providing a structured experience where errors are explicitly characterized and used for training and feedback. Error management training may play a major role in equipping residents and junior faculty with the skills required for independent, high-quality operative performance.

  View details for DOI 10.1016/j.amjsurg.2019.11.013

  View details for PubMedID 31806167

• Combining metrics from clinical simulators and sensorimotor tasks can reveal the training background of surgeons. IEEE transactions on bio-medical engineering Huang, F. C., Mohamadipanah, H. n., Mussa-Ivaldi, F. n., Pugh, C. n. 2019

  Abstract

  Skill assessment in surgery traditionally has relied on expert observation and qualitative scoring. Our novel study design demonstrates how analysis of performance in sensorimotor tasks and bench-top surgical simulators can provide inferences about the technical proficiency as well as the training history of surgeons.Our unique study design examined metrics for basic sensorimotor tasks in a virtual reality interface as well as motion metrics in clinical scenario simulations. As indicators of training level, we considered survey responses from surgery residents, including the number of years post-graduation (PGY, four levels), research years (RY, three levels), and clinical years (CY, three levels). Next, we performed a linear discriminant analysis with cross-validation (90% training, 10% testing) to relate the training levels to the selected metrics.Using combined metrics from all stations, we found greater than chance predictions for each survey category, with an overall accuracy of 43.4±2.9% for identifying the level for post-graduate years, 79.1±1.0% accuracy for research training years, and 64.2±1.0% for clinical training years. Our main finding was that combining metrics from all stations resulted in more accurate predictions than using only sensorimotor or clinical scenario tasks. In addition, our analysis indicates that metrics related to the ability to cope with changes in the task environment were the most important predictors of training level.These results suggest that each simulator type provided crucial information for evaluating surgical proficiency. The methods developed in this study could improve evaluations of a surgeon's clinical proficiency as well as training potential in terms of basic sensorimotor ability.

  View details for DOI 10.1109/TBME.2019.2892342

  View details for PubMedID 30629489

• Dynamic Visual Feedback During Junctional Tourniquet Training. The Journal of surgical research Xu, J., Kwan, C., Sunkara, A., Mohamadipanah, H., Bell, K., Tizale, M., Pugh, C. M. 2019; 233: 444–52

  Abstract

  BACKGROUND: This project involved the development and evaluation of a new visual bleeding feedback (VBF) system for tourniquet training. We hypothesized that dynamic VBF during junctional tourniquet training would be helpful and well received by trainees.MATERIALS AND METHODS: We designed the VBF to simulate femoral bleeding. Medical students (n=15) and emergency medical service (EMS) members (n=4) were randomized in a single-blind, crossover study to the VBF or without feedback groups. Poststudy surveys assessing VBF usefulness and recommendations were conducted along with participants' reported confidence using a 7-point Likert scale. Data from the different groups were compared using Wilcoxon signed-rank and rank-sum tests.RESULTS: Participants rated the helpfulness of the VBF highly (6.53/7.00) and indicated they were very likely to recommend the VBF simulator to others (6.80/7.00). Pre- and post-VBF confidence were not statistically different (P=0.59). Likewise, tourniquet application times for VBF and without feedback before crossover were not statistically different (P=0.63). Although participant confidence did not change significantly from beginning to end of the study (P=0.46), application time was significantly reduced (P=0.001).CONCLUSIONS: New tourniquet learners liked our VBF prototype and found it useful. Although confidence did not change over the course of the study for any group, application times improved. Future studies using outcomes of this study will allow us to continue VBF development as well as incorporate other quantitative measures of task performance to elucidate VBF's true benefit and help trainees achieve mastery in junctional tourniquet skills.

  View details for PubMedID 30502284

• What do you want to know? Operative experience predicts the type of questions practicing surgeons ask during a CME laparoscopic hernia repair course. American journal of surgery Godfrey, M., Rosser, A. A., Pugh, C. M., Sachdeva, A. K., Sullivan, S. 2018

  Abstract

  BACKGROUND: Given their variegated backgrounds, surgeons taking continuing medical education (CME) courses possess different learning needs. This study examines the relationship between surgeons' levels of experience and the questions they asked in a simulation-based CME course.METHODS: We analyzed transcribed audio-video data collected from surgeons participating in a simulated laparoscopic hernia repair CME course and identified four types of questions learners posed to their instructors. Linear regressions compared how often these questions were asked versus self-reported operative experience.RESULTS: Both Requesting Guidance and Requesting Confirmation were inversely proportional to experience, whereas Asking About a Specific Case was directly proportional to experience. Requesting Instructor Preference exhibited no significant correlation with experience.CONCLUSION: Practicing surgeons with relatively less experience tend to ask for confirmation and guidance, whereas those with greater experience tend to focus on specific hypothetical scenarios. This data can be used to tailor instruction based on learners' self-reported experience level.

  View details for PubMedID 30527925

• Surgical procedural map scoring for decision-making in laparoscopic cholecystectomy. American journal of surgery Hashimoto, D. A., Axelsson, C. G., Jones, C. B., Phitayakorn, R., Petrusa, E., McKinley, S. K., Gee, D., Pugh, C. 2018

  Abstract

  INTRODUCTION: The objective of this study was to determine whether decision-based procedural mapping demonstrates differences in attendings versus residents.METHODS: Attendings and residents were interviewed about operative decision-making in laparoscopic cholecystectomy (LC) using a cognitive task analysis framework. Interviews were converted into procedural maps. Operative steps, patient factors, and surgeon factors noted by attendings and residents were compared. Two scoring methods were used to compare map structures of attendings versus residents.RESULTS: Six attendings and six residents were interviewed. There were no significant differences in the number of patient or surgeon factors identified. Attendings had significantly more operative steps (29.67 ± 1.9 vs. 23.3 ± 1.9, p = 0.04) and crosslinks (3.2 ± 0.5 vs. 1 ± 0.4, p = 0.005) in their maps and a higher total score (90.2 ± 8.4 vs. 63.2 ± 3.8, p = 0.015) than residents.CONCLUSION: LC procedural map scoring for attendings and residents demonstrated significant differences in structural complexity and may provide a useful framework for assessing decision making.

  View details for PubMedID 30470551

• Shortcut assessment: Can residents' operative performance be determined in the first five minutes of an operative task? Surgery Mohamadipanah, H., Nathwani, J., Peterson, K., Forsyth, K., Maulson, L., DiMarco, S., Pugh, C. 2018; 163 (6): 1207–12

  Abstract

  BACKGROUND: The aim was to validate the potential use of a single, early procedure, operative task as a predictive metric for overall performance. The authors hypothesized that a shortcut psychomotor assessment would be as informative as a total procedural psychomotor assessment when evaluating laparoscopic ventral hernia repair performance on a simulator.METHODS: Using electromagnetic sensors, hand motion data were collected from 38 surgery residents during a simulated laparoscopic ventral hernia repair procedure. Three time-based phases of the procedure were defined: Early Phase (start time through completion of first anchoring suture), Mid Phase (start time through completion of second anchoring suture), and Total Operative Time. Correlations were calculated comparing time and motion metrics for each phase with the final laparoscopic ventral hernia repair score.RESULTS: Analyses revealed that execution time and motion, for the first anchoring suture, predicted procedural outcomes. Greater execution times and path lengths correlated to lesser laparoscopic ventral hernia repair scores (r = -0.56, P = .0008 and r = -0.51, P = .0025, respectively). Greater bimanual dexterity measures correlated to Greater LVH repair scores (r = + 0.47, P = .0058).CONCLUSIONS: This study provides validity evidence for use of a single, early operative task as a shortcut assessment to predict resident performance during a simulated laparoscopic ventral hernia repair procedure. With the continued development and decreasing costs of motion technology, faculty should be well-versed in the use of motion metrics for performance measurements. The results strongly support the use of dexterity and economy of motion (path length + execution time) metrics as early predictors of operative performance.

  View details for PubMedID 29728259

• Shortcut assessment: Can residents' operative performance be determined in the first five minutes of an operative task? SURGERY Mohamadipanah, H., Nathwani, J., Peterson, K., Forsyth, K., Maulson, L., DiMarco, S., Pugh, C. 2018; 163 (6): 1207-1212

  View details for DOI 10.1016/j.surg.2018.02.012

  View details for Web of Science ID 000435227300005

• Faculty perceptions of resident skills decay during dedicated research fellowships AMERICAN JOURNAL OF SURGERY D'Angelo, A. D., D'Angelo, J. D., Rogers, D. A., Pugh, C. M. 2018; 215 (2): 336–40

  Abstract

  Residents engaging in dedicated research experiences may return to clinical training with less surgical skill. The study aims were 1) to evaluate faculty perceptions of residents skills decay during dedicated research fellowships, and 2) to compare faculty and resident perceptions of residents skills decay.Faculty and residents were surveyed on resident research practices and perceptions of resident skills decay.Faculty thought residents returning from research demonstrate less technical skill (Median = 4; 5-point Likert scale, 1 = Strongly disagree, 5 = Strongly agree), demonstrate less confidence (Median = 4), and require more instruction (Median = 4). Both faculty and residents perceived the largest skill reduction in complex procedures, technical surgical skills, and knowledge of procedure steps (p < 0.05).While dedicated research experiences provide valuable academic experience, there is a cost to clinical skills retention and confidence specifically in the areas of complex operative procedures and technical surgical skills.

  View details for DOI 10.1016/j.amjsurg.2017.11.018

  View details for Web of Science ID 000425193700025

  View details for PubMedID 29169821

• A structured, extended training program to facilitate adoption of new techniques for practicing surgeons Greenberg, J. A., Jolles, S., Sullivan, S., Quamme, S., Funk, L. M., Lidor, A. O., Greenberg, C., Pugh, C. M. SPRINGER. 2018: 217–24

  Abstract

  Laparoscopic inguinal hernia repair has been shown to have significant benefits when compared to open inguinal hernia repair, yet remains underutilized in the United States. The traditional model of short, hands-on, cognitive courses to enhance the adoption of new techniques fails to lead to significant levels of practice implementation for most surgeons. We hypothesized that a comprehensive program would facilitate the adoption of laparoscopic inguinal hernia repair (TEP) for practicing surgeons.A team of experts in simulation, coaching, and hernia care created a comprehensive training program to facilitate the adoption of TEP. Three surgeons who routinely performed open inguinal hernia repair with greater than 50 cases annually were recruited to participate in the program. Coaches were selected based on their procedural expertise and underwent formal training in surgical coaching. Participants were required to evaluate all aspects of the educational program and were surveyed out to one year following completion of the program to assess for sustained adoption of TEP.All three participants successfully completed the first three steps of the seven-step program. Two participants completed the full course, while the third dropped out of the program due to time constraints and low case volume. Participant surgeons rated Orientation (4.7/5), GlovesOn training (5/5), and Preceptored Cases (5/5) as highly important training activities that contributed to advancing their knowledge and technical performance of the TEP procedure. At one year, both participants were performing TEPs for "most of their cases" and were confident in their ability to perform the procedure. The total cost of the program including all travel, personal coaching, and simulation was $8638.60 per participant.Our comprehensive educational program led to full and sustained adoption of TEP for those who completed the course. Time constraints, travel costs, and case volume are major considerations for successful completion; however, the program is feasible, acceptable, and affordable.

  View details for DOI 10.1007/s00464-017-5662-2

  View details for Web of Science ID 000422854700025

  View details for PubMedID 28643054

• Residents' response to bleeding during a simulated robotic surgery task JOURNAL OF SURGICAL RESEARCH Walker, J. L., Nathwani, J. N., Mohamadipanah, H., Laufer, S., Jocewicz, F. F., Gwillim, E., Pugh, C. M. 2017; 220: 385–90

  View details for DOI 10.1016/j.jss.2017.07.023

  View details for Web of Science ID 000417699100048

• A Holistic Model of Surgical Expertise and Competency ANNALS OF SURGERY Pugh, C. M. 2017; 265 (2): 268–69

  View details for DOI 10.1097/SLA.0000000000002066

  View details for Web of Science ID 000392295200009

  View details for PubMedID 27805965

• Sensor technology in assessments of clinical skill. The New England journal of medicine Laufer, S., Cohen, E. R., Kwan, C., D'Angelo, A. D., Yudkowsky, R., Boulet, J. R., McGaghie, W. C., Pugh, C. M. 2015; 372 (8): 784-6

  View details for DOI 10.1056/NEJMc1414210

  View details for PubMedID 25693026

  View details for PubMedCentralID PMC4425402

• Characterizing Touch Using Pressure Data and Auto Regressive Models Laufer, S., Pugh, C. M., Van Veen, B. D., IEEE IEEE. 2014: 1839–42

  Abstract

  Palpation plays a critical role in medical physical exams. Despite the wide range of exams, there are several reproducible and subconscious sets of maneuvers that are common to examination by palpation. Previous studies by our group demonstrated the use of manikins and pressure sensors for measuring and quantifying how physicians palpate during different physical exams. In this study we develop mathematical models that describe some of these common maneuvers. Dynamic pressure data was measured using a simplified testbed and different autoregressive models were used to describe the motion of interest. The frequency, direction and type of motion used were identified from the models. We believe these models can a provide better understanding of how humans explore objects in general and more specifically give insights to understand medical physical exams.

  View details for Web of Science ID 000350044701204

  View details for PubMedID 25570335

  View details for PubMedCentralID PMC4288476

• Perception of stiffness in laparoscopy - the fulcrum effect. Studies in health technology and informatics Nisky, I., Huang, F., Milstein, A., Pugh, C. M., Mussa-Ivaldi, F. A., Karniel, A. 2012; 173: 313-319

  Abstract

  We explored how the perception of stiffness can be distorted in Minimally Invasive Surgery. We combined a mechanical simulator with a haptic device, and implemented linear springs at the tip of the simulated laparoscopic device. To explore the influence of mechanical advantage on perception, we set different values of the ratio between internal and external length of the tool. We found that a nonsymmetrical ratio causes bias in the perceived stiffness when novice tangential probing is compared to radial probing. In contrast, haptic experts did not show similar perceptual bias.

  View details for PubMedID 22357009

• Intra-operative decision making: More than meets the eye JOURNAL OF BIOMEDICAL INFORMATICS Pugh, C. M., Santacaterina, S., DaRosa, D. A., Clark, R. E. 2011; 44 (3): 486–96

  Abstract

  Operating room teams consist of team members with diverse training backgrounds. In addition to differences in training, each team member has unique and complex decision making paths. As such, team members may function in the same environment largely unaware of their team members' perspectives. The goal of our work was to use a theory-based approach to better understand the complexity of knowledge-based intra-operative decision making. Cognitive task analysis methods were used to extract the knowledge, thought processes, goal structures and critical decisions that provide the foundation for surgical task performance. A triangulated and iterative approach is presented.

  View details for DOI 10.1016/j.jbi.2010.01.001

  View details for Web of Science ID 000291768200014

  View details for PubMedID 20096376

• Qualitative and quantitative analysis of pressure sensor data acquired by the E-Pelvis simulator during simulated pelvic examinations. Studies in health technology and informatics Pugh, C. M., Rosen, J. 2002; 85: 376-9

  View details for PubMedID 15458117

• Development and validation of assessment measures for a newly developed physical examination simulator JOURNAL OF THE AMERICAN MEDICAL INFORMATICS ASSOCIATION Pugh, C. M., Youngblood, P. 2002; 9 (5): 448-460

  Abstract

  Define, extract and evaluate potential performance indicators from computer-generated data collected during simulated clinical female pelvic examinations.Qualitative and quantitative study analyzing computer generated simulator data and written clinical assessments collected from medical students who performed physical examinations on three clinically different pelvic simulators.Introduction to patient care course at a major United States medical school.Seventy-three pre-clinical medical students performed 219 simulated pelvic examinations and generated 219 written clinical assessments.Cronbach's alpha for the newly defined performance indicators, Pearson's correlation of performance indicators with scored written clinical assessments of simulator findings.Four novel performance indicators were defined: time to perform a complete examination, number of critical areas touched during the exam, the maximum pressure used, and the frequency at which these areas were touched. The reliability coefficients (alpha) were time = 0.7240, critical areas = 0.6329, maximum pressure = 0.7701, and frequency = 0.5011. Of the four indicators, three correlated positively and significantly with the written clinical assessment scores: critical areas, p < 0.01; frequency, p < 0.05; and maximum pressure, p < 0.05.This study demonstrates a novel method of analyzing raw numerical data generated from a newly developed patient simulator; deriving performance indicators from computer generated simulator data; and assessing validity of those indicators by comparing them with written assessment scores. Results show the new assessment measures provide an objective, reliable, and valid method of assessing students' physical examination techniques on the pelvic exam simulator.

  View details for DOI 10.1197/jamia.M1107

  View details for Web of Science ID 000178205000003

  View details for PubMedID 12223497

  View details for PubMedCentralID PMC346632

• Visual representations of physical abilities: Reverse haptic technology? 10th Annual Medicine Meets Virtual Reality Conference Pugh, C. M., Srivastava, S., Heinrichs, M. L. I O S PRESS. 2002: 380–381

  View details for Web of Science ID 000176591900068

  View details for PubMedID 15458118