How I Do It: Objective Structured Clinical Examination for Training in Regional Anesthesia
 |  | |
| Amanda Kumar, MD Duke University Medical Center (DUMC) Durham, North Carolina | Jeff Gadsden, MD Duke University Medical Center (DUMC) Durham, North Carolina | |
 |  |  |
| Joon-Hyung Kim, MD Westchester Medical Center/New York Medical College (WMC/NYMC) Valhalla, New York | Irim Salik, MD Westchester Medical Center/New York Medical College (WMC/NYMC) | Jeff L. Xu, MD Westchester Medical Center/New York Medical College (WMC/NYMC) |
 |  | |
Leila Zuo, MD | Dawn Dillman, MD Oregon Health and Science University (OHSU) Portland, Oregon | |
 |  | |
| Nadia Hernandez, MD University of Texas Health Science Center at Houston (UTH) Houston, Texas | Johanna Blair de Haan, MD University of Texas Health Science Center at Houston (UTH) |
Objective structured clinical examination (OSCE) is a common adjunct to modern medical education used to measure trainees’ competency in history taking or data gathering, physical examination, communication, clinical acumen, and professionalism. OSCE can be used as both formative and summative assessments while providing feedback to the trainee.
Despite the growing popularity of OSCE in medical training, its role in regional anesthesia training is less clear. Research has suggested that use of OSCE in medical training is reliable but only modestly valid to provide a comprehensive assessment of a trainee’s competence.[1] Instead, it may be considered as complementary to existing assessment modalities. We recently surveyed educators from four different United States anesthesia programs to understand how OSCE is used in regional anesthesia training.
How long have you been performing OSCE exams for regional anesthesia training, and why did you introduce it?
DUMC: We incorporated OSCE in our curriculum in 2017 as mock exams for learning purposes for our regional anesthesia and acute pain medicine fellows. Our goal was to broaden their learning experience and overall competency in regional anesthesia with a particular emphasis on uncommon or challenging clinical scenarios that they may otherwise not have a chance to experience in their daily practice throughout training.
WMC/NYMC: We started a regional anesthesia OSCE project in 2017 to supplement a structured curriculum in resident education during the regional anesthesia rotation. We felt the regional anesthesia OSCE could improve the objective assessment component of resident competency and expertise in the service to our patients and increase residents' confidence in their readiness for OSCEs of the American Board of Anesthesiology applied examination.
OHSU: We have administered OSCEs for regional anesthesia since 2017. The OSCEs were added to assess for competency in a more structured way and to build out our point-of-care ultrasound curriculum.
UTH: We integrated OSCEs into the curriculum for the month-long regional anesthesia rotation in 2018. We wanted to give residents early exposure to the OSCE process, as well as incentivize engagement in the medical decision making processes of regional anesthesia.
What challenges did you encounter setting up the OSCE?
DUMC: We are extremely fortunate to have a high-fidelity simulation center and the support of our department to allow nonclinical time for education. One of the more challenging aspects of creating the OSCEs was the development of assessment tools, validating the tools, and determining how, or if, the performance on the OSCEs should be incorporated into fellows’ global milestone evaluations.
WMC/NYMC: The OSCE simulation center is located off campus at our affiliated medical school, and it can be challenging to gather faculty and residents at an offsite location. Appropriate training is required for the faculty administering the OSCE. Although all residents have protected didactic time each week, it can be difficult for our anesthesia department to spare multiple faculty members with nonclinical time to facilitate the OSCE simulation. Our residents receive a percentage score on each encounter with a standardized patient based on our scoring rubric; however, translating the results into their milestone evaluations can be challenging.
OHSU: Barriers to the OSCE include training the faculty and fellows on administering the OSCE, as well as instilling a cultural understanding that it is important to complete the curriculum as outlined. We have had to reinforce the need to adhere to the curriculum on multiple occasions with the faculty. Also, using a faculty member as the simulated patient could bias the evaluation because the faculty may potentially play the role of the patient differently based on their previous interactions with the resident.
UTH: Our main challenge has been related to faculty motivation. The resident undergoes two OSCE assessments: one at the midpoint of the regional anesthesia rotation and the other on their final day of the rotation at the end of their month. At the beginning of the OSCE implementation, the faculty would be the attending for the regional anesthesia of the day. Because of the challenges in achieving full participation from all attendings, either the division chief or the regional anesthesia resident education director now administers the OSCE.
When do you perform the OSCE exams?
DUMC: We administer the regional anesthesia OSCE in January for our fellows, which is five months into their one-year clinical fellowship. We believe this is an ideal time to consolidate skills and knowledge learned throughout the first half of their fellowship. Furthermore, it also refreshes their crisis resource management skills in anticipation of their American Board of Anesthesiology applied oral board examinations and OSCEs.
WMC/NYMC: Residents’ level of training ranges from clinical anesthesia year one (CA-1) to CA-3. In the last week of the four-week regional anesthesia rotation, the resident completes two OSCE stations. Residents also participate in annual simulation training sessions at the New York Medical College clinical skills simulation center, which include regional anesthesia–related scenarios.
OHSU: Two OSCEs are incorporated into the four-week basic rotation that occurs during late CA-1 or CA-2 in our main operating rooms. Two additional OSCEs occur during the four-week advanced rotation in CA-3 at our ambulatory facility.
UTH: Our residents rotate on the regional anesthesia service for a full month. The OSCE is administered halfway through the month and repeated at the end of the month, with different question stems. Residents on our regional anesthesia rotation range from CA-1 in the second half of their year to CA-3 near graduation. We aim to achieve the same depth of knowledge with our first-year residents as we do with our graduating residents.
What scenario(s) do you include in the OSCE? What domains do you assess?
DUMC: Our OSCE has six scenarios, including high-fidelity simulations such as an intrathecal pump malfunction, pneumothorax after paravertebral block, and postpartum hemorrhage after a transverse abdominis plane block for Cesarean section. The simulations assess the learner’s monitoring and data interpretation as well as management of hemodynamic instability, local anesthetic systemic toxicity (LAST), opioid overdose, and advanced cardiac life support. We include skills stations such as block performance and neuraxial anatomy identification. Other stations require learners to demonstrate knowledge of local anesthetic toxic doses and LAST treatment algorithms, communication skills, and ability to discuss a treatment plan for a patient with a peripheral nerve injury after a nerve block. We continually evaluate our OSCE curriculum and receive input from faculty in the regional division as well as our fellows to update the curriculum with relevant stations each year.
WMC/NYMC: The rotating resident completes one OSCE with two cases (case 1: adductor canal block and case 2a: supraclavicular nerve block or 2b: infraclavicular nerve block). Prior to the nerve block procedures, the resident is asked to (1) describe how to position the transducer to approach the target, (2) identify relevant anatomical structures using an ultrasound probe, and (3) demonstrate simulated needle placement on how to insert the needle and where to deposit the local anesthetics. We have created additional case scenarios using various peripheral nerve blocks and interfascial plane blocks. During NYMC’s annual simulation training sessions, residents participate in a series of stations that focus on obtaining informed consent, formulating and discussing treatment options, and managing periprocedural complications related to regional anesthesia. Residents are assessed on their proficiency in interpersonal and communication skills, professionalism, and technical skills related to patient care, which are difficult to evaluate in written or oral exams.
OHSU: Residents undertake four OSCE stations on four separate occasions. The first basic rotation OSCE is peripheral nerve block basic preparation and setup, which is usually administered on the afternoon of the resident’s first day on the rotation. The OSCE involves the physical setup of the patient and equipment for an interscalene peripheral nerve catheter. This OSCE may be performed with a mannequin, with a standardized patient, or as an observed structured assessment of technical skills with a patient. The second week basic rotation OSCE is evaluation and consent for a regional anesthesia procedure, which involves an interview and obtaining consent from a standardized patient for a regional anesthetic. The topic is also included in the American Board of Anesthesiology OSCE content outline. The first advanced rotation OSCE measures response to LAST complications and is ideally done in the first week of the rotation with a CPR mannequin. This OSCE is particularly useful to ensure that the resident is aware of where rescue equipment is located in a facility where they practice less often. The final advanced rotation OSCE is nerve injury complications and is scheduled for the last week of the rotation. It is usually done with a faculty member playing the role of the patient.
UTH: We have a bank of six OSCE scenarios: (1) patient with lateral ankle pain following an open reduction internal fixation of the ankle, (2) patient with knee pain following anterior cruciate ligament surgery whose surgeon wants them to participate in physical therapy, (3) patient with traumatic amputation of his right leg presenting for right above-knee amputation, with surgeons requesting a nerve block for postoperative analgesia, (4) patient undergoing left inguinal hernia repair where the surgeon requests an opioid-sparing regimen with a peripheral nerve block, (5) surgical regional anesthesia for a patient presenting to the operating room for creation of a left upper-extremity arteriovenous fistula for dialysis access, and (6) requested postoperative analgesia with brachial plexus blockade following a rotator cuff repair. We select three scenarios for the midpoint OSCE and three for the final OSCE.
What tactic(s) do you use?
DUMC: We use SimMan for our high-fidelity scenarios. A faculty member serves as the patient for our communication and skills stations.
WMC/NYMC: We use OSCE-like simulation for teaching purposes in performing ultrasound-guided nerve blocks. OSCE sessions with standardized patients take place on an annual basis at the simulation center.
OHSU: We use Anesthesia Toolbox, a multi-institution collaboration containing a peer-reviewed curriculum, including educational online modules, podcasts, recommended readings, and OSCEs. It has curricula for regional resident basic and advanced rotations that include all modalities. Each OSCE in the Anesthesia Toolbox is accompanied by a script and a checklist used for evaluation purposes. Mannequins or standardized patients may be used, depending on the OSCE scenario.
UTH: In the course of providing routine postoperative regional anesthesia, OSCE scenarios are presented to learners. If no patients are available, residents are asked to identify sonographic anatomy on a volunteer faculty member or medical student who consents to scanning. Needling is not included in the OSCE exam; our OSCE focuses more on correct medical decision making. Residents are asked to determine an appropriate regional anesthesia technique for a given scenario, to correctly identify anatomy, and to understand the different types of local anesthetic used.
Who administers the OSCE?
DUMC: Our regional anesthesia faculty have a strong commitment to education, and several are involved as instructors for our OSCE sessions. They help facilitate sessions, serve as confederates during high-fidelity simulations, and lead debriefing sessions. Additionally, we have support from staff in our simulation center.
WMC/NYMC: OSCE is administered by either the regional anesthesia fellow or regional anesthesia attending. Anonymous survey responses from current residents-in-training indicate that residents slightly favor OSCE conducted by the regional anesthesia attending.
OHSU: The OSCE is administered by the regional anesthesia attending on service. Our workflow is typically lighter in the afternoon, and it is usually possible to work in a 15-minute OSCE. Multiple OSCEs spread over time (as opposed to blocked into one group) are easier to administer and limit the number of learning objectives addressed in a single day.
UTH: Our OSCE exams are administered by either the service division chief or director of resident education for the rotation. Initially this was because of a lack of faculty participation; however, we have also found that it is useful for consistency and evaluation for improvement in the resident from midpoint to final OSCE assessment.
How are trainees evaluated?
DUMC: Fellows are evaluated using an analytic grading rubric that includes 10–25 items per station. They also receive a score for any knowledge worksheets that are incorporated into the station and a global holistic score from 1–4 (unsatisfactory to very good) on their overall performance. These evaluations are sent to each individual learner. One of the OSCE’s most important aspects is that we facilitate debriefing sessions after each station to allow for immediate feedback, reflection, and education.
WMC/NYMC: During regional anesthesia OSCEs, each of the OSCE scenarios (case 1 and case 2a or 2b) have 10 questions. Each correctly managed question nets the examinee one point. A passing grade is 14 out of a total of 20 points. During the annual simulation sessions, residents are given direct feedback from simulated patient actors on interpersonal and communication skills and professionalism within a simulated environment as well as a web link that allows them to review their own performance.
OHSU: Each OSCE has a 20- to 25-point grading rubric. We currently use OSCEs as formative feedback only, with immediate debriefing between the faculty and resident.
UTH: Residents are evaluated on the following factors: (1) identification of the correct block for the question stem, (2) correct identification of ultrasound anatomy for the desired block, and (3) appropriate local anesthetic concentration and volume selected. The factors are graded yes or no for three question stems, and they receive one point for each “yes” and zero points for each “no.” The resident must achieve six of nine points to pass the OSCE. The grade on the midpoint OSCE is not recorded but is used as a motivating tool for the resident. The final OSCE score (pass or fail) is recorded on the resident’s rotation evaluation.
What general outcomes or observations have you seen with using OSCE? What are the measures of resident satisfaction or improvement?
DUMC: We have completed two years of our OSCE curriculum and have collected anonymous survey feedback. OSCE is used for teaching (debriefing sessions after each station) and evaluation (fellows receive their scored evaluations, which are also shared with the program director). Every fellow has rated the OSCE simulation as excellent, the highest possible rating. In particular, they noted that the OSCE allowed them to perform tasks they otherwise would not have a chance to practice and that the knowledge they gained during the session would change their future practice of anesthesia. Seventy-five percent of our fellows expressed that they prefer or strongly prefer the OSCE simulation format over traditional learning methods such as a lecture series.
WMC/NYMC: Residents who have completed the proctored sessions in regional anesthesia rate the OSCE sessions highly because they provide them with an organized way to plan the procedure and identify the salient steps examiners might want to test. Based on our experience, the addition of hands-on OSCE sessions to formal resident didactic curriculum results in significant improvements in resident satisfaction because it constitutes constructive feedback based on Accreditation Council for Graduate Medical Education core competency domains, such as patient care (technical skills in regional anesthesia), interpersonal and communications skills, and medical knowledge.
OHSU: Simulation and the OSCEs have the highest satisfaction scores of any part of our program on our annual program evaluation. Residents appreciate the opportunity to apply their knowledge and ensure that they are providing optimal care. The OSCE format frequently uncovers deficits that allow for immediate feedback and improvement of care.
UTH: Administration of the OSCE, especially the midpoint OSCE, has motivated our residents to be more academically engaged with the regional anesthesia service. The ability for support staff to direct residents in a structured format during the exam reduces anxiety.
References
- Hastie MJ, Spellman JL, Pagnano PP, Hastie J, Egan BJ. Designing and Implementing the Objective Structured Clinical Examination in anesthesiology. Anesthesiology. 2014:120(1);196–203. https://doi.org/10.1097/ALN.0000000000000068
This article was developed by the Education in Regional Anesthesia SIG.
Leave a commentOrder by
Newest on top Oldest on top