Multiple tools have been created for assessment of trainee procedural performance during regional anesthesia tasks, and published studies evaluating their use can be found in a number of journals. Assessment tool formats vary, as do the study designs in which they are used. Some studies seek to create a tool, others to validate one, and yet others to describe patient or training outcomes in the context of an assessment.

The following highlighted article summaries showcase a number of regional assessment tools reported in the anesthesia literature. See the Table for a list of selected tools. The tools have been grouped into neuraxial and peripheral block categories. The most frequently used tools are task-specific checklists, global rating scales, quality compromising behaviors, or, most commonly, a combination of two types of tools.

Task-specific checklists provide the steps of a procedure or task that an observer expects to see performed in the course of an encounter. For example, “asking for initial aspiration to rule out intravascular injection” is a step common to all peripheral nerve blocks and present in all related checklist tools. The rater expects the step to occur at the proper time and marks “yes” or “no” to whether it was performed. Some checklists allow a three-point rating scale with specification of whether a task was performed (1) well, (2) poorly, or (3) not at all. Checklists allow more granular assessment of whether procedural steps occurred but are, as a result, more procedure-specific and tend to be longer than global rating scales. Each procedure may therefore need its own checklist assessment.

Global rating scales (GRS) allow assessment of overall performance or performance on subtasks, typically on a 5- or 7-point scale. The scales can be given “anchors,” or descriptions of performance corresponding to different scores. For example, in the category “Instrument Handling,” a score of 1 may correspond to repeated awkward movements whereas 3 means occasional awkward movements, and a perfect 5 denotes fluid movements without awkwardness. GRS domains may be more generalizable between different procedures, with the same instrument potentially used for different types of assessment. Fewer categories are usually present than with checklist tools, but a GRS scale does not provide the definitive “yes” or “no” about a procedural step as a checklist.

Quality-compromising behaviors (QCB) are used in a minority of performance assessment tools. Instead of categorizing each step of a procedure, QCB highlight any errors that occur. Whereas checklists are written in the affirmative, asking a mark for each procedural step that happens, QCB are written negatively, with errors or “novice behaviors” flagged by the assessor.

A few of the cited studies do not contain specific assessment tools but include checklists or guides that could be helpful in assessing regional anesthesia performance or building your own assessments for clinical practice, simulation, or preparation for Objective Structured Clinical Exams (OSCE).

The purpose of this article is to highlight some available assessment tools that readers may incorporate into their practice. The intent is not to rank one tool over any other. Yet, as you review summaries and read the actual papers, it is worth asking several questions about each tool: What is the assessment tool designed to evaluate? Is the tool broadly applicable or limited to a single type of procedure? What validity evidence is presented to suggest that the assessment does what it is intended to do? Is it intended for summative use or formative use? How many items are included in each assessment? How feasible is incorporation into clinical practice? What are the relative merits of one assessment type over another?

Peripheral Nerve Block Assessment Tools

Sites B, Spence B, Gallagher J, et al. Characterizing novice behavior associated with learning ultrasound-guided peripheral regional anesthesia. Reg Anesth Pain Med. 2007;32(2):107–115.

Sites et al sought to characterize mistakes by trainees during ultrasound-guided single injection peripheral nerve blocks. Six residents were videotaped during performance of 520 blocks. Their mistakes—which are referred to as novice behaviors or QCB—were categorized by type and tracked over time. Seven error types were identified before the study, with five other errors identified during the study. The 12 types of quality-compromising behaviors were not designed as an assessment tool but have the potential to be used as one. The most common novice behaviors were needle advancement without visualization and unintentional probe movement.

Number of items: 12; Assessment type: QCB; Clinical use: ultrasound-guided regional anesthesia (UGRA) single-injection peripheral block

Cheung JJH, Chen EW, Darani R, et al. The creation of an objective assessment tool for ultrasound-guided regional anesthesia using the Delphi method. Reg Anesth Pain Med. 2012;37(3):329–333.

Cheung et al reported on the creation of a 22-item checklist and 9-item global rating scale tool for ultrasound-guided singleinjection peripheral nerve block, developed using a modified Delphi method. The modified Delphi method is well described in the study.

Number of items: 31; Assessment type: Checklist and GRS; Clinical use: UGRA single-injection peripheral block with or without nerve-stimulation

Burckett-St. Laurent DA, Niazi AU, Cunningham MS, et al. A valid and reliable assessment tool for remote simulation-based ultrasound-guided regional anesthesia. Reg Anesth Pain Med. 2014;39(6):496–501.

Burckett–St. Laurent et al tested the checklist developed by Cheung et al for UGRA single-injection peripheral nerve blocks. This study provided validity evidence for the tool's use based on good intraclass correlation, an ability to distinguish novices from experienced proceduralists, and similar scores in simulated and patient-care scenarios.

Number of items: 31; Assessment type: checklist and GRS; Clinical use: UGRA single-injection peripheral block

Wong DM, Watson MJ, Kluger R, et al. Evaluation of a task-specific checklist and global rating scale for ultrasound-guided regional anesthesia. Reg Anesth Pain Med. 2014;39(5):399–408.

Wong et al adapted the tool used by Cheung et al into a 31- item checklist and 10-item global rating scale for UGRA single-injection peripheral nerve blocks. In contrast to previous checklists that were written in the affirmative (“check if an action did occur”), Wong et al worded items as “QCB.” In this tool, points were given for the absence of an error or QCB. This use of QCB is reminiscent of the Sites et al study. This study evaluated 30 blocks using the tool and found an intraclass correlation coefficient of 0.44 for total score. The tool took about 4.5 min (median) to complete an assessment.

Number of items: 41; Assessment type: QCB and GRS; Clinical use: UGRA single-injection peripheral block

Watson MJ, Wong DM, Kluger R, et al. Psychometric evaluation of a direct observation of procedural skills assessment tool for ultrasound-guided regional anaesthesia. Anaesthesia. 2014;69(6):604–612.

Watson et al adapted the Direct Observation of Procedural Skills (DOPS) scoring system into a 11+2 item tool for UGRA singleinjection peripheral nerve blocks. Eleven items were graded on a nine-point scale with behavioral anchors to guide the rating. Two items were dichotomous. The assessments were done on trainees who were videotaped performing nerve blocks. An extensive instruction sheet was used to prompt questioning by supervisors preprocedure and intraprocedure and to score trainees. Appendix 1 could be very useful to anyone instructing trainees in regional anesthesia. Intraclass correlation coefficients were low at 0.1–0.49.

Number of items: 13; Assessment type: behaviorally anchored scale with dichotomous items; Clinical use: UGRA single-injection peripheral block

Naik V, Chandra D, Chung D, Chan V. An assessment tool for brachial plexus regional anesthesia performance: establishing construct validity and reliability. Reg Anesth Pain Med. 2007;32(1):41–45.

Naik et al sought to validate a 20-item checklist and 8-item global rating scale specific to nerve-stimulator–guided interscalene brachial plexus blockade. They used a pre-existing global rating scale with a checklist created by the modified Delphi method. Scores of senior trainees were compared to those of junior trainees, showing that the tool could discriminate between the groups.

Number of items: 28; Assessment type: checklist and GRS; Clinical use: stimulator-guided single-injection interscalene block

Sultan SF, Iohom G, Saunders J, Shorten G. A clinical assessment tool for ultrasound-guided axillary brachial plexus block. Acta Anaesthesiol Scand. 2012;56(5):616–623.

Sultan et al developed a 63-item checklist and 9-item global rating scale for use in UGRA axillary brachial plexus blocks. They used expert opinion to develop the checklist and then studied inter-rater reliability and the tool's ability to discriminate between providers of different skill levels.

Number of items: 72; Assessment type: checklist and GRS; Clinical use: UGRA single-injection axillary block

Ahmed OM, O'Donnell BD, Gallagher AG, Shorten GD. Development of performance and error metrics for ultrasound-guided axillary brachial plexus block. Adv Med Educ Pract. 2017;8:257–263.

Ahmed et al developed a 54-item checklist and 32-item QCB tool for UGRA single-injection axillary brachial plexus blocks. Using an intensive Delphi method, 54 requisite steps in axillary block were identified as well as 32 QCBs or errors. Errors were categorized as critical or noncritical for scoring. The study only described the tool's development and did not test its use.

Number of items: 86; Assessment type: checklist and QCB; Clinical use: UGRA single-injection axillary block

Ahmed OMA, O'Donnell BD, Gallagher AG, et al. Construct validity of a novel assessment tool for ultrasound-guided axillary brachial plexus block. Anaesthesia. 2016;71(11):1324–1331.

Ahmed et al took their UGRA single-injection axillary brachial plexus block tool and recorded a very small number of blocks by experts and novices. Experts were shown to commit fewer errors and slightly fewer critical errors (0.8 vs 1.3) compared to novices. The inter-rater reliability between two unidentified reviewers was very good (0.88). Times required to complete the assessment were not recorded.

Number of items: 86; Assessment type: checklist and QCB; Clinical use: UGRA single-injection axillary block

Neuraxial Anesthesia Assessment Tools

Friedman Z, Devito I, Siddiqui M, Chan V. Objective assessment of manual skills and proficiency in performing epidural anesthesia – video-assisted validation. Reg Anesth Pain Med. 2006;31(4):304– 310.

Friedman et al developed a 27-item checklist and 7-item global rating scale for use in labor epidural placement. A panel of obstetric (OB) anesthesiologists developed the assessment tool, although development details are not disclosed. Residents were videotaped while placing epidurals when they had performed 0–30, 31–90, and more than 90 epidural placements. The tool's validity was suggested by the fact that higher scores correlated to increased trainee experience. Good correlation in total scores between raters also supported validity and reliability of the tool.

Number of items: 34; Assessment type: checklist and GRS; Clinical use: lumbar epidural placement

Friedman Z, Siddiqui N, DeVito I. Experience is not enough: repeated breaches in epidural anesthesia aseptic technique by novice operators despite improved skill. Anesthesiology. 2008;108:914– 920.

Friedman et al developed a 15-item checklist for aseptic technique during lumbar epidural placement to complement their previously developed manual skills checklist and GRS. The sterile technique assessment tool was employed to assess residents with various levels of lumbar epidural experience. Progression of manual or technical skills was shown to be faster than progression in sterile technique.

Number of items: 15; Assessment type: checklist; Clinical use: sterile technique during epidural placement

Birnbach DJ, Santos AC, Bourlier RA, et al. The effectiveness of video technology as an adjunct to teach and evaluate epidural anesthesia performance skills. Anesthesiology. 2002;96(1):5–9.

Birnbach et al developed a 33-item checklist for labor epidural placement. Each item was assigned to one of 13 skills and judged on whether a major error, minor error, or no error occurred. Residents were videotaped performing blocks at the start, middle, and end of a month of OB anesthesia. Those who were allowed to review their performance tapes showed greater skill gains on the assessment tool than those who did not view their performances. The tool was developed by four OB anesthesiologists from a larger set of criteria, but the method of tool development was not further detailed.

Number of items: 33; Assessment type: checklist; Clinical use: lumbar epidural placement

Weed J, Finkel K, Beach ML, et al. Spinal anesthesia for orthopedic surgery: a detailed video assessment of quality. Reg Anesth Pain Med. 2011;36(1):51–55.

Weed et al recorded spinal anesthetics in 60 patients to categorize the novice behavior or QCB that occurs during spinal placement. Twelve different QCBs were identified. The study assessed placement times, provider training level, patient body mass index, number of needle passes, and QCB to make meaningful correlations. Although it was not designed as an assessment tool, the 12 QCB observed during spinal placement are useful to know and watch for.

Number of items: 12; Assessment type: QCB; Clinical use: spinal

Breen D, Bogar L, Heigl P, Rittberger J, Shorten GD. Validation of a clinical assessment tool for spinal anaesthesia. Acta Anaesthesiol Scand. 2011;55(6):653–657.

Breen et al developed a tool that included 11 errors, two time measurement intervals, and a six-item global rating scale for use in spinal placement. Items were developed by a consensus of a focus group that included anesthesiologists, trainees, and a psychologist. The errors differed from QCB in other studies (eg, Sites et al). Breen et al had errors that were a mixture of behaviors (doesn't palpate iliac crests) and outcomes (more than three spaces attempted, supervisor takes over procedure). Time measurements were T1 – positioning to + cerebrospinal fluid and T2 – palpation to skin local. The only validity evidence for the tool was an ability to distinguish novice, intermediate, and expert proceduralists from one another. No ratings of interrater reliability were performed. Time intervals did not differ significantly between groups.

Number of items: 19; Assessment type: QCB, GRS, and time measures; Clinical use: spinal

Other Useful Materials for Guiding Assessment and Training

Chuan A, Wan AS, Royse CF, Forrest K. Competency-based assessment tools for regional anaesthesia: a narrative review. Br J Anaesth. 2018;120(2):264–273.

Chuan et al performed an in-depth appraisal of assessment tools in this narrative review. Different modalities of assessment are described, as are the principles underlying them. The authors emphasized and provided thorough description of the properties of an optimal tool, including validity and reliability. Supplemental tables provide in-depth information about the highlighted tools.

Ben-Menachem E, Ezri T, Ziv A, et al. Objective structured clinical examination-based assessment of regional anesthesia skills: the Israeli National Board Examination in Anesthesiology experience. Anesth Analg. 2011;112(1):242–245.

Ben-Menachem et al described OSCE for the Israeli Anesthesiology boards. Assessments and tasks were developed via the Delphi method. The article presents the scoring checklist used in an axillary block OSCE scenario. The assessment tool was designed for a testing environment, not clinical practice, but it can provide insight into OSCE scoring.

Sites BD, Chan VW, Neal JM, et al. The American Society of Regional Anesthesia and Pain Medicine and the European Society of Regional Anaesthesia and Pain Therapy joint committee recommendations for education and training in ultrasound-guided regional anesthesia. Reg Anesth Pain Med. 2009;34(1):40–46.

Sites et al presented ASRA/European Society of Regional Anesthesia and Pain Therapy guidelines for UGRA training. Although the guideline did not create or report a specific assessment tool, it contains numerous helpful lists that can be adapted to support procedural assessments, including 10 key tasks for UGRA, skills associated with block proficiency, UGRA core competencies, ultrasound curriculum maps, and stepwise ultrasound scanning procedures.