Advertisement

Simulation in Procedural Training

At the Tipping Point
  • Susan Murin
    Correspondence
    Correspondence to: Susan Murin, MD, FCCP, Division of Pulmonary, Critical Care, and Sleep Medicine, UC Davis, 4150 V St, Ste 3400, Sacramento, CA 95817
    Affiliations
    Division of Pulmonary and Critical Care Medicine, Department of Internal Medicine, UC Davis School of Medicine, Sacramento, CA

    VA Northern California Health Care System, Sacramento, CA
    Search for articles by this author
  • Nicholas S. Stollenwerk
    Affiliations
    Division of Pulmonary and Critical Care Medicine, Department of Internal Medicine, UC Davis School of Medicine, Sacramento, CA

    VA Northern California Health Care System, Sacramento, CA
    Search for articles by this author
      Clinical medicine has long relied on the apprenticeship model for educating physicians in training. This model, although well suited for complex and dynamic learning environments, also has limits and flaws.
      • Pratt DD
      For medical procedures, the most serious problem with the apprenticeship model is that it is high risk: Learners train in real-life environments, and the burden of procedure-related training is borne by individual patients exposed to the increased risk of procedural complications posed by novice operators. Another serious shortcoming of the apprenticeship model is its lack of emphasis on objective measurement of trainee competence. Evaluation is usually subjective and global, with training time and/or volume of experience used as proxies for competence.
      Many occupations, including other high-risk occupations such as the military and aviation, have successfully integrated simulation into training and evaluation. Anesthesia training has used mannequins for advanced cardiac life support and difficult airway training since the late 1980s.
      • Cooper JB
      • Taqueti VR
      A brief history of the development of mannequin simulators for clinical education and training.
      Simulation allows for a controlled environment, repetitive practice, and timely and specific feedback to the learner. It allows for training on a variety of clinical situations, including less common but still critical patient care scenarios, and enables the environment to be tailored to the individual learner.
      • Issenberg SB
      • McGaghie WC
      • Petrusa ER
      • Lee Gordon D
      • Scalese RJ
      Features and uses of high-fidelity medical simulations that lead to effective learning: a BEME systematic review.
      Despite these strengths, the adoption of simulation in medical education and procedure training has been relatively slow. Reluctance to broadly embrace simulation as a tool in medical education is due to a variety of factors. Chief among these is insufficient outcomes data, with the most important outcomes being patient outcomes indicative of the successful transfer of simulation-learned skills to the bedside. Other obstacles to simulation training include the time and cost involved and the reluctance to change.
      In this issue of CHEST, two articles provide important additional support for the use of simulation for procedure training and assessment of procedure skills. Wahidi and colleagues
      • Wahidi MM
      • Silvestri GA
      • Coakley RD
      • et al.
      A prospective multicenter study of competency metrics and educational interventions in the learning of bronchoscopy among new pulmonary fellows.
      (see page 1040) describe the results of a prospective study comparing usual bronchoscopy education to a structured educational approach that included simulation training. A validated bronchoscopy skills assessment tool, the Bronchoscopy Skills and Tasks Assessment Tool (BSTAT), was used to evaluate trainees at a variety of milestones. Dong and colleagues
      • Dong Y
      • Suri HS
      • Cook DA
      • et al.
      Simulation-based objective assessment discerns clinical proficiency in central line placement: a construct validation.
      (see page 1050) describe their development and validation of a performance assessment tool for use in a simulation-based central line workshop.
      The study by Wahidi et al
      • Wahidi MM
      • Silvestri GA
      • Coakley RD
      • et al.
      A prospective multicenter study of competency metrics and educational interventions in the learning of bronchoscopy among new pulmonary fellows.
      provides several important contributions in the area of bronchoscopy training and assessment. The first report of the efficacy of simulator use for bronchoscopy training was published in CHEST in 2001,
      • Colt HG
      • Crawford SW
      • Galbraith III, O
      Virtual reality bronchoscopy simulation: a revolution in procedural training.
      and since that time, several other studies have been published, as recently reviewed by Davoudi and Colt.
      • Davoudi M
      • Colt HG
      Bronchoscopy simulation: a brief review.
      Collectively, prior studies demonstrate that bronchoscopy simulation leads to decreased procedure time, decreased wall contact, improved procedure efficiency, increased thoroughness, and increased accuracy of the airway examination. However, with the exception of a single, small study, all of these improved outcomes were shown in a simulated environment—in other words, prior work showed that simulation improved performance on a simulator. The study by Wahidi et al
      • Wahidi MM
      • Silvestri GA
      • Coakley RD
      • et al.
      A prospective multicenter study of competency metrics and educational interventions in the learning of bronchoscopy among new pulmonary fellows.
      demonstrates that structured training including simulation results in measurable, statistically significant, sustained improvements in bronchoscopy technique for procedures subsequently performed on actual patients. Fellows who trained using simulation achieved higher BSTAT scores much earlier in training; scores at their 20th bronchoscopy were the same as scores for the 50th bronchoscopy in nonsimulation-trained fellows, evidence of a substantial shift in the learning curve by simulation. The simulation cohort also outperformed the nonsimulation cohort until the 100th bronchoscopy. The longitudinal, repeated evaluation of the learners as they moved to the real environment, the use of a structured, validated measurement tool, and the demonstration of sustained improvements in skills as a result of initial simulation training all make this study an important contribution to the field.
      The study's use of a validated assessment tool for bronchoscopy training warrants emphasis. As we have moved toward competency-based education in medicine, we have lagged in developing competency-based evaluation methods. This BSTAT, although imperfect, is a step in the right direction and provides a structured, quantifiable method of evaluating procedural competency that has face validity, expert buy in, and acceptable interrater reliability, and discriminates skill level between experienced and inexperienced operators. The tool would be improved if the elements it assesses included preprocedure preparation of the patient and safety-oriented measures, such as incorporation of a procedural pause, and was more broadly validated.
      Another important aspect of the study by Wahidi et al
      • Wahidi MM
      • Silvestri GA
      • Coakley RD
      • et al.
      A prospective multicenter study of competency metrics and educational interventions in the learning of bronchoscopy among new pulmonary fellows.
      is that it involved collaboration between multiple institutions, with trainees from multiple fellowships receiving simulation training at a single site. The sharing of manpower and technology is key to addressing the incorporation of an expensive new tool into training, particularly for procedures for which only a small number of trainees require training each year. If resources are shared across multiple programs, feasibility and cost-effectiveness are improved. Future studies to see if similar improvements in technique could be obtained using much less expensive, lower-fidelity forms of simulation, such as structured, mentored training on lung models, would be welcome given the current high costs of available high-fidelity simulators.
      An additional contribution of this study is the validation of minimum number of bronchoscopies required for training. Currently, the Accreditation Council for Graduate Medical Education requires that trainees perform a minimum of 50 bronchoscopies during pulmonary fellowship training, a number chosen by expert consensus without supporting research. By gathering longitudinal data on multiple trainees, this study shows that expert opinion was on target. The BSTAT score curves demonstrate that skills improve rapidly over the first 30 procedures, then continue to improve more slowly. The skill level of trainees not trained with simulation did not match that of simulation-trained operators until they reached the 75th procedure. These data support not only the specification of a minimum number of procedures in training but also the recent proposed increase in the minimum number of procedures required. Ultimately, the goal of procedure training is competence, and any minimum number of procedures must be coupled with objective measurement of competence as was done by Wahidi et al.
      • Wahidi MM
      • Silvestri GA
      • Coakley RD
      • et al.
      A prospective multicenter study of competency metrics and educational interventions in the learning of bronchoscopy among new pulmonary fellows.
      Central venous catheter (CVC) placement is a common procedure performed by a variety of operators, and safer central venous catheter insertion is a goal set by many groups in health care. The Agency for Healthcare Research and Quality and others have recommended ultrasound use for CVC placement to reduce complication risk. Procedure teams have been implemented at some institutions to provide focused expertise and training. These teams are a step in the right direction, but they still use a high-risk environment for learning.
      • Smith CC
      • Gordon CE
      • Feller-Kopman D
      • et al.
      Creation of an innovative inpatient medical procedure service and a method to evaluate house staff competency.
      Simulation, on the other hand, offers a zero-risk environment in which to learn procedures. Given the frequency of CVC insertion, the broad focus on performing it safely, the known increased risk associated with novice operators, and the availability of relatively inexpensive CVC task trainers, there has been relatively little research on the use of simulation to teach both CVC insertion and ultrasound guidance for CVC insertion or on methods to assess mastery of the tasks and techniques involved before procedures are done on patients. Several small studies have shown the benefits of simulation training, including reduced complications and fewer needle passes, and increased operator comfort/confidence with the procedure.
      • Britt RC
      • Novosel TJ
      • Britt LD
      • Sullivan M
      The impact of central line simulation before the ICU experience.
      • Barsuk JH
      • McGaghie WC
      • Cohen ER
      • Balachandran JS
      • Wayne DB
      Use of simulation-based mastery learning to improve the quality of central venous catheter placement in a medical intensive care unit.
      The contribution of the study by Dong et al
      • Dong Y
      • Suri HS
      • Cook DA
      • et al.
      Simulation-based objective assessment discerns clinical proficiency in central line placement: a construct validation.
      is the group's development and validation of a tool for assessing CVC insertion proficiency after simulator training. The tool incorporates a variety of measures across multiple domains, has face validity and acceptable interrater reliability, and discriminates based on operator experience. Another such tool was recently described by Huang and colleagues,
      • Huang GC
      • Newman LR
      • Schwartzstein RM
      • et al.
      Procedural competence in internal medicine residents: validity of a central venous catheter insertion assessment instrument.
      which is specific to the subclavian site, did not include ultrasound use, and is more detailed in its breakdown of specific tasks involved in the procedure. The Huang instrument was developed through rigorous methods, but has not yet had the validation of the tool described by Dong. For both tools, use and validation in the real-world environment should follow.
      It is time to put “see one, do one, teach one” behind us. Procedure training should involve a combination of didactics and simulation, with objective evidence of technical competency before exposing patients to the risk of procedures performed by novice operators. Yes, additional research is needed to determine how to optimize the use of simulation, but we should not wait to broadly adopt this tool in our teaching and in our assessment of competency. The case for the incorporation of clinical simulation in internal medicine has been eloquently made.
      • Ogden PE
      • Cobbs LS
      • Howell MR
      • Sibbitt SJ
      • DiPette DJ
      Clinical simulation: importance to the internal medicine educational mission.
      The articles in this issue add to the support for simulation, and we have reached the tipping point.

      References

        • Pratt DD
        Five Perspectives on Teaching in Adult and Higher Education. Krieger Publishing Co, Melbourne, FL1998: 83-103
        • Cooper JB
        • Taqueti VR
        A brief history of the development of mannequin simulators for clinical education and training.
        Postgrad Med J. 2008; 84: 563-570
        • Issenberg SB
        • McGaghie WC
        • Petrusa ER
        • Lee Gordon D
        • Scalese RJ
        Features and uses of high-fidelity medical simulations that lead to effective learning: a BEME systematic review.
        Med Teach. 2005; 27: 10-28
        • Wahidi MM
        • Silvestri GA
        • Coakley RD
        • et al.
        A prospective multicenter study of competency metrics and educational interventions in the learning of bronchoscopy among new pulmonary fellows.
        Chest. 2010; 137: 1040-1049
        • Dong Y
        • Suri HS
        • Cook DA
        • et al.
        Simulation-based objective assessment discerns clinical proficiency in central line placement: a construct validation.
        Chest. 2010; 137: 1050-1056
        • Colt HG
        • Crawford SW
        • Galbraith III, O
        Virtual reality bronchoscopy simulation: a revolution in procedural training.
        Chest. 2001; 120: 1333-1339
        • Davoudi M
        • Colt HG
        Bronchoscopy simulation: a brief review.
        Adv Health Sci Educ Theory Pract. 2009; 14: 287-296
        • Smith CC
        • Gordon CE
        • Feller-Kopman D
        • et al.
        Creation of an innovative inpatient medical procedure service and a method to evaluate house staff competency.
        J Gen Intern Med. 2004; 19: 510-513
        • Britt RC
        • Novosel TJ
        • Britt LD
        • Sullivan M
        The impact of central line simulation before the ICU experience.
        Am J Surg. 2009; 197: 533-536
        • Barsuk JH
        • McGaghie WC
        • Cohen ER
        • Balachandran JS
        • Wayne DB
        Use of simulation-based mastery learning to improve the quality of central venous catheter placement in a medical intensive care unit.
        J Hosp Med. 2009; 4: 397-403
        • Huang GC
        • Newman LR
        • Schwartzstein RM
        • et al.
        Procedural competence in internal medicine residents: validity of a central venous catheter insertion assessment instrument.
        Acad Med. 2009; 84: 1127-1134
        • Ogden PE
        • Cobbs LS
        • Howell MR
        • Sibbitt SJ
        • DiPette DJ
        Clinical simulation: importance to the internal medicine educational mission.
        Am J Med. 2007; 120: 820-824