Translating motion tracking data into resident feedback: An opportunity for streamlined video coaching
EXCERPTA MEDICA INC-ELSEVIER SCIENCE INC. 2020: 552–56
We hypothesized that differences in motion data during a simulated laparoscopic ventral hernia repair (LVH) can be used to stratify top and lower tier performers and streamline video review.Surgical residents (N = 94) performed a simulated partial LVH repair while wearing motion tracking sensors. We identified the top ten and lower ten performers based on a final product quality score (FPQS) of the repair. Two blinded raters independently reviewed motion plots to identify patterns and stratify top and lower tier performers.Top performers had significantly higher FPQS (23.3 ± 1.2 vs 5.7 ± 1.6 p < 0.01). Raters identified patterns and stratified top performers from lower tier performers (Rater 1 χ2 = 3.2 p = 0.07 and Rater 2 χ2 = 2.0 p = 0.16). During video review, we correlated motion plots with the relevant portion of the procedure.Differences in motion data can identify learning needs and enable rapid review of surgical videos for coaching.
View details for DOI 10.1016/j.amjsurg.2020.01.032
View details for Web of Science ID 000525802700004
View details for PubMedID 32014295
Sensors and Psychomotor Metrics: A Unique Opportunity to Close the Gap on Surgical Processes and Outcomes.
ACS biomaterials science & engineering
2020; 6 (5): 2630–40
The surgical process remains elusive to many. This paper presents two independent empirical investigations where psychomotor skill metrics were used to quantify elements of the surgical process in a procedural context during surgical tasks in a simulated environment. The overarching goal of both investigations was to address the following hypothesis: Basic motion metrics can be used to quantify specific aspects of the surgical process including instrument autonomy, psychomotor efficiency, procedural readiness, and clinical errors. Electromagnetic motion tracking sensors were secured to surgical trainees' (N = 64) hands for both studies, and several motion metrics were investigated as a measure of surgical skill. The first study assessed performance during a bowel repair and laparoscopic ventral hernia (LVH) repair in comparison to a suturing board task. The second study assessed performance in a VR task in comparison to placement of a subclavian central line. The findings of the first study support our subhypothesis that motion metrics have a generalizable application to surgical skill by showing significant correlations in instrument autonomy and psychomotor efficiency during the suturing task and bowel repair (idle time: r = 0.46, p < 0.05; average velocity: r = 0.57, p < 0.05) and the suturing task and LVH repair (jerk magnitude: r = 0.36, p < 0.05; bimanual dexterity: r = 0.35, p < 0.05). In the second study, performance in VR (steering and jerkiness) correlated to clinical errors (r = 0.58, p < 0.05) and insertion time (r = 0.55, p < 0.05) in placement of a subclavian central line. Both gross (dexterity) and fine motor skills (steering) were found to be important as well as efficiency (i.e., idle time, duration, velocity) when seeking to understand the quality of surgical performance. Both studies support our hypotheses that basic motion metrics can be used to quantify specific aspects of the surgical process and that the use of different technologies and metrics are important for comprehensive investigations of surgical skill.
View details for DOI 10.1021/acsbiomaterials.9b01019
View details for PubMedID 33463275
Use of sensors to quantify procedural idle time: Validity evidence for a new mastery metric.
BACKGROUND: Quantification of mastery is the first step in using objective metrics for teaching. We hypothesized that during orotracheal intubation, top tier performers have less idle time compared to lower tier performers.METHODS: At the Anesthesiology 2018 Annual Meeting, 82 participants intubated a normal airway simulator and a burnt airway simulator. The movements of the participant's laryngoscope were quantified using electromagnetic motion sensors. Top tier performers were defined as participants who intubated both simulators successfully in less than the median time for each simulator. Idle time was defined as the duration of time when the laryngoscope was not moving.RESULTS: Top performers showed less Idle Time when intubating the normal airway compared to lower tier performers (14.5 ± 9.8 seconds vs 34.0 ± 52.0 seconds, respectively P < .01). Likewise, top performers showed less Idle Time when intubating the burnt airway compared to lower tier performers (18.6 ± 15.2 seconds vs 63.4 ± 59.11 seconds; P < .01). Comparing performance on the burnt airway to the normal airway, there was a difference for lower tier performers (63.4 ± 59.1 seconds vs 34.0 ± 52.0 seconds; P < .01) but not for top tier performers (18.6 ± 15.2 seconds vs 14.5 ± 9.8 seconds; P= .07).CONCLUSION: Similar to our previous findings with other procedures, Idle Time was shown to have known group validity evidence when comparing top performers with lower tier performers. Further, Idle Time was correlated with procedure difficulty in our prior work. We observed statistically significant differences in Idle Times for lower tier performers when comparing the normal airway to the burnt airway but not for top tier performers. Our findings support the continued exploration of Idle Time for development of objective assessment and curricula.
View details for DOI 10.1016/j.surg.2019.09.016
View details for PubMedID 31708084
A Simulation-based, cognitive assessment of resident decision making during complex urinary catheterization scenarios.
American journal of surgery
2017; 213 (4): 622–26
This study explores general surgery residents' decision making skills in uncommon, complex urinary catheter scenarios.40 residents were presented with two scenarios. Scenario A was a male with traumatic urethral injury and scenario B was a male with complete urinary blockage. Residents verbalized whether they would catheterize the patient and described the workup and management of suspected pathologies. Residents' decision paths were documented and analyzed.In scenario A, 45% of participants chose to immediately consult Urology. 47.5% named five diagnostic tests to decide if catheterization was safe. In scenario B, 27% chose to catheterize with a 16 French Coude. When faced with catheterization failure, participants randomly upsized or downsized catheters. Chi-square analysis revealed no measurable consensus amongst participants.Residents need more training in complex decision making for urinary catheterization. The decision trees generated in this study provide a useful blueprint of residents' learning needs.Exploration of general surgery residents' decision making skills in uncommon, complex urinary catheter scenarios revealed major deficiencies. The resulting decision trees reveal residents' learning needs.
View details for DOI 10.1016/j.amjsurg.2017.01.007
View details for PubMedID 28089342
View details for PubMedCentralID PMC5485835
Relationship Between Technical Errors and Decision-Making Skills in the Junior Resident.
Journal of surgical education
2016; 73 (6): e84–e90
The purpose of this study is to coevaluate resident technical errors and decision-making capabilities during placement of a subclavian central venous catheter (CVC). We hypothesize that there would be significant correlations between scenario-based decision-making skills and technical proficiency in central line insertion. We also predict residents would face problems in anticipating common difficulties and generating solutions associated with line placement.Participants were asked to insert a subclavian central line on a simulator. After completion, residents were presented with a real-life patient photograph depicting CVC placement and asked to anticipate difficulties and generate solutions. Error rates were analyzed using chi-square tests and a 5% expected error rate. Correlations were sought by comparing technical errors and scenario-based decision-making skills.This study was performed at 7 tertiary care centers.Study participants (N = 46) largely consisted of first-year research residents who could be followed longitudinally. Second-year research and clinical residents were not excluded.In total, 6 checklist errors were committed more often than anticipated. Residents committed an average of 1.9 errors, significantly more than the 1 error, at most, per person expected (t(44) = 3.82, p < 0.001). The most common error was performance of the procedure steps in the wrong order (28.5%, p < 0.001). Some of the residents (24%) had no errors, 30% committed 1 error, and 46 % committed more than 1 error. The number of technical errors committed negatively correlated with the total number of commonly identified difficulties and generated solutions (r (33) = -0.429, p = 0.021, r (33) = -0.383, p = 0.044, respectively).Almost half of the surgical residents committed multiple errors while performing subclavian CVC placement. The correlation between technical errors and decision-making skills suggests a critical need to train residents in both technique and error management.
View details for DOI 10.1016/j.jsurg.2016.08.004
View details for PubMedID 27671618
View details for PubMedCentralID PMC5485830