How Well Do Global Rating Scales Assess Arthroscopy Skills?

With residents and fellows missing out on two months of training, it may be more important than ever to properly assess surgical competency. A team of researchers from the UK has waded into this issue with their recent work, “Global Rating Scales for the Assessment of Arthroscopic Surgical Skills: A Systematic Review,” which appears in the April 2020 edition of the Journal of Arthroscopy.

Co-author Kalpesh R.Vaghela, MBBS, a trauma and orthopedic resident at the Royal London Hospital in the UK, told OTW, “The traditional apprenticeship model of acquiring skills is dated and has been superseded by competency-based medical education. Although informal feedback about surgical performance has been the cornerstone of surgical education this is not a quantifiable or reliable method of assessing surgeons.”

“There has been a proliferation of educational tools to assess surgical competency, including procedural checklists, work-based assessments and simulator performance. One of these educational tools, Global Rating Scales (GRS), has been increasingly used in a number of different settings (operating theatre, virtual reality simulation, cadaveric labs) for providing quantitative feedback and have been used in high stakes, summative examinations and selection processes. We reviewed the role of GRS as an educational tool and assessed their validity and reliability.”

“We found 11 different Global Rating Scales related to the assessment of arthroscopic surgical skills. These appeared in 39 publications and have been used to assess 1,175 surgeons. Three GRS defined a minimum competency threshold. Six out of the 11 GRS demonstrated construct validity i.e., the ability to differentiate between surgeons of differing experience. Five out of the 11 GRS assessed inter-rater reliability.”

“The two leading GRS were the Arthroscopic Surgical Skills Evaluation Tool (ASSET) and the Basic Arthroscopic Knee Skill Scoring System (BAKSSS). ASSET was validated by 16 articles for a total of 537 surgeons for hip, knee, shoulder and ankle arthroscopy in both simulated and clinical environments but was found to be invalid in wrist arthroscopy. The BAKSSS was validated for 15 articles for a total of 497 surgeons for knee, hip and shoulder arthroscopy in both clinical and simulated environments.”

“GRS are a diverse, valid and reliable method of assessing arthroscopic surgical skills in multiple joints and in multiple environments. The body of literature reviewed demonstrates that GRS have contributed to training, feedback and formative assessment practices. Currently, there is sufficient evidence to support the use of GRS as a feedback tool. However, there is insufficient evidence for its use in high-stakes examinations or as a minimum competency assessment.”

“GRS are an important addition to the armamentarium of educational tools available for the assessment of arthroscopic surgical skills proficiency and should be used alongside other established methods within a competency-based education framework.”