Reducing the Hassle of Patient-Reported Outcomes

Now more than ever, patients don’t want to spend time filling out lengthy, cumbersome questionnaires. New work from the Rothman Orthopaedic Institute in Philadelphia and Universal Research Solutions in Columbia, Missouri, addresses this issue in their new study, “Use of Computerized Adaptive Testing to Develop More Concise Patient-Reported Outcome Measures,” appears in the March 12, 2020 edition of The Journal of Bone and Joint Surgery.

Co-author Joseph Abboud, M.D., an orthopedic surgeon at The Sidney Kimmel Medical College at Thomas Jefferson University in Philadelphia explained the contradictory issues associated with patient-reported outcome logistics to OTW, “While the benefits of patient-reported outcome measures (PROMs) to the medical and scientific community are clear, there is demand from both the patient and the physician standpoint to develop more efficient PROMs that improve compliance while maintaining the integrity of the outcome score.”

“Advances in data science have shown that the score of an outcome measure can be accurately predicted from fewer questions if the correct questions are asked. Predictive models, developed through a process known as computerized adaptive testing, offer a potential solution. The goal of computerized adaptive testing is to identify the correct subset of questions selected from the full questionnaire to ask each patient on the basis of his/her previous responses.”

“Computerized adaptive testing is then ‘trained’ by way of so-called machine learning programs, also described as artificial intelligence, which analyze how response patterns affect overall outcome scores. The computerized adaptive testing model then uses its own recognition of these patterns to self-improve its efficiency and minimize question burden in an accurate manner.”

“Given its utility and widespread application, countless patients are asked to complete the Veterans RAND 12 Item Health Survey (VR-12) in its full form every day as a fixture of participation in the healthcare system. For this reason, the computerized adaptive testing model was developed in an effort to help reduce the question burden placed on patients, thereby improving the patient experience by shifting focus away from data collection instruments and toward patient-driven goals and patients’ relationship with their physician.”

So, these efforts involve taking the “parent” questionnaire and create a smaller, more concise one. “The computerized adaptive testing used by OBERD (Columbia, Missouri) in this study is distinct from alternative computerized adaptive testing systems, including those developed in PROMIS (Patient-Reported Outcomes Measurement Information System), which use methodology based on item response theory that requires a separate set of questions (i.e., ‘item bank’). The OBERD computerized adaptive testing, however, relies on machine learning methods rather than item response theory. It utilizes the questions of the historical forms rather than separate item banks to collect outcomes, making them interchangeable with the existing PROMs such as the VR12. In its practical application from a patient perspective, the administration of the computerized adaptive testing strongly resembles that of the full questionnaire.”

“While the full VR-12 form comprises 12 questions, the computerized adaptive testing model validated in this study required 8 questions to be answered in its application for all subjects, representing a 33% decrease in question burden. Taken together, these findings demonstrate support for the implementation of computerized adaptive testing in a live setting (while patients are responding to the survey) to elicit VR-12 outcome scores.”

Expanding on the artificial intelligence theme, Dr. Abboud noted, “The computerized adaptive testing system was designed to incorporate machine learning algorithms into PROM collection in order to improve PROM efficiency and improve patient experience. In this study, the application of the computerized adaptive testing model to the VR-12 survey demonstrated an ability to lessen the response burden for patients and had very little impact on score integrity. We have published papers on this topic using similar methodology to validate that other highly utilized specialty specific PROMs can use computerized adaptive testing systems to minimize question burden in an accurate manner.”

“Specifically, we have shown this with computerized adaptive testing Knee Osteoarthritis Outcome Scores (KOOS), computerized adaptive testing KOOS-JR, computerized adaptive testing Hip Disability and Osteoarthritis Outcome Score (HOOS), Hip Disability and Osteoarthritis Outcome Score Joint Replacement (HOOS-JR), and the CAT American Shoulder and Elbow Surgeons (ASES) questionnaire. Bottom line is that with the VR-12 and other PROMs we can achieve the same results yet decrease the burden on the patient and health system, allowing all involved to ‘win.’”