Mayo Using Deep Learning to Create Radiography Registry

Researchers from Mayo Clinic in Rochester, Minnesota, are using a deep learning (DL), artificial intelligence (AI) type of advanced algorithm to “see” and analyze nearly 900,000 hip and pelvic radiography Digital Imaging and Communications in Medicine (DICOM) files from 20,378 total hip arthroplasty (THA) patients.

Their study, “Applying Deep Learning to Establish a Total Hip Arthroplasty Radiography Registry: A Stepwise Approach,” appears in the September 21, 2022, edition of The Journal of Bone and Joint Surgery.

“The past couple years have been an intensive effort to establish our Orthopedic Surgery Artificial Intelligence Laboratory,” explained co-author Cody C. Wyles, M.D. to OTW. “While undertaking a variety of projects involving deep learning evaluation of X-rays, we discovered one of the most challenging steps was curating a clean database for analysis for each respective project.”

“It was clear that we needed to pause and undertake an infrastructure effort to create a radiography registry of all our hip arthroplasty patients. In this way, we would have a well characterized and organized database of images that could then be linked to rich clinical databases.”

Dr. Wyles further explained that the patient metadata (name, sex, etc.), as well as the date, time, and magnification of imaging, is stored as “tags,” and that the AI deep learning algorithms can organize these tags. Of course, cleaning the data is a massive issue. Specifically, human error and missing information on, for example, the presence or absence of medical devices or pathology.

As a result, the researchers developed deep-learning algorithms to automatically construct a databank of hip and pelvic radiographs for an existing clinical registry of THA patients.

Their goals were twofold:

1. “to utilize these automated pipelines to identify all pelvic and hip radiographs with appropriate annotation of laterality and presence or absence of implants, and
2. “to automatically measure acetabular component inclination and version for THA images.”

“The study relied on a pipeline of deep-learning algorithms trained to classify and annotate important features from X-rays such as the radiographic view, the anatomic side, and whether an implant is present or not,” explained Dr. Wyles.

“It was also able to annotate acetabular component position for X-rays with those components.”

“All of the algorithms performed >99.5% on every metric. This enabled subsequent characterization of a 20-year database including nearly 850,000 DICOM files in one day.”

“Deep-learning algorithms enabled appropriate exclusion of 209,332 DICOM files (24.7%) as misclassified non-hip/pelvic radiographs or having corrupted pixel data,” stated the authors. “The final registry was automatically curated and annotated in <8 hours and included 168,551 anteroposterior pelvic, 176,890 anteroposterior hip, 174,637 lateral hip, and 117,578 oblique hip radiographs. The algorithms achieved 99.9% accuracy, 99.6% precision, 99.5% recall, and a 99.6% F1 score in determining the radiograph appearance.”

“We were pleasantly surprised with the accuracy and the speed of the algorithms,” Dr. Wyles told OTW. “Undertaking this effort manually would have required a small army of trained human annotators. This study draws a nice analogy to an infrastructure project. It is not flashy, but it is absolutely foundational to the success of nearly every downstream project. We hope the road map outlined in this paper to establish a radiography registry can be used for any anatomic area of interest by other institutions to facilitate their own research efforts.”