New Harvard Study Tests the Proposition: Garbage In, Garbage Out

How vital is a completed patient reported outcome form? This new Harvard Medical School study found evidence that it is much more important than previously thought and has a documented bearing on patient outcomes.

The new study, “Sociodemographic Factors Are Associated with Patient-Reported Outcome Measure Completion in Orthopaedic Surgery: An Analysis of Completion Rates and Determinants Among New Patients,” appears in the August 4, 2022, edition of The Journal of Bone and Joint Surgery: Open Access.

Co-author Daniel G. Tobert, M.D., an orthopedic spine surgeon and assistant professor at Harvard Medical School, told OTW, “We are interested in this topic because PROMIS (Patient-Reported Outcomes Measurement Information System) is a patient-reported outcome measure that is increasingly being utilized for clinical care, research, and administrative purposes. Therefore, much of the treatment plans, research results, and healthcare policies that utilize PROMIS will be based on the patients that actually complete the questionnaires. For any results to be applicable to our society, it must be reflective of our society as well.”

Flawed, incomplete or outright biased patient reported outcome forms at the front end could affect treatment plans and much more on the back end.

Garbage In, Garbage Out

To test this hypothesis, the Harvard team designed a study using data from every new orthopedic patient treated at one academic medical center from 2016 to 2020. Depending on subspecialty and presenting condition, patients were assigned various PROMIS forms and legacy PROMs. The team then also collected demographic and clinical information from each patient’s electronic medical record.

In total, the study was able to collect data from 40% (88,052 of 219.891 patients) of the PROMS forms.

“Non-English-speaking patients had a 90% increased likelihood of not completing assigned PROMs at presentation,” wrote the authors. “Older patients (≥65 years of age) and patients of Black race had a 23% and 24% increased likelihood of not completing assigned PROMs, respectively.”

Opting Out Means Being Left Out

“We analyzed the variables that were available with these data, which include age, sex, race, primary language, marital status, insurance status, and others,” Dr. Tobert told OTW. “Patients that are older, those that are less technologically engaged, non-English speakers, and certain race categories are less likely to fill out PROMIS questionnaires. This means their responses are underrepresented in orthopedic research and it risks excluding their input into future policy initiatives.”

“Research that utilizes PROMIS should include information about the cohort that filled out the questionnaires but also those that were eligible but did not complete the questionnaires,” added Dr. Tobert. “More importantly, efforts should be made to engage patients at risk for not completing the forms to create a more equal representation of the patient population.”