Is Orthopedic Research Falling Short?

To effect concrete changes in the clinical realm, surgeons increasingly require data from high level studies such as randomized clinical trials (RCTs). But, for a multitude of reasons, RCT studies are too few and too far between. With the storm of controversy erupting over past studies of such products as Infuse, is research into orthopedic products or procedures failing to meet the demands of a modern, multi-faceted practice?

The first in a series of three articles was recently published in the Journal of Bone and Joint Surgery (JBJS), and it attempted to answer an increasingly critical question—what is standing in the way of RCT research?

Dr. James Wright, an orthopedic surgeon with The Hospital for Sick Children in Toronto, was the article’s lead author. He explains, “The American Academy of Orthopaedic Surgeons (AAOS) and the Orthopaedic Research Society (ORS), with the support of JBJS and NIH [National Institutes of Health] held a symposium that tackled issues such as, ‘What is the appropriate research agenda for orthopedic surgery?’ and, ‘What are the key issues that are facilitating or impeding high level research?’ and ‘How can we improve the quality of orthopedic clinical trials?’ Although an increasing number of orthopedic surgeons are behind the idea that there is a substantial role for more and better RCTs, how we should proceed is not completely clear.”

Dr. Wright worked with Jeffrey Katz, M.D., Director of the Orthopaedics and Arthritis Center for Outcomes Research at Brigham and Women’s Hospital (BWH), and Elena Losina, Ph.D., Associate Professor of Orthopedic Surgery at BWH. Dr. Wright says, “In this particular article we highlight the cultural and practical barriers to research, and we address what key infrastructure is needed to support randomized trials in orthopedics.”

It may seem odd to discuss culture—a “soft” science—in relation to “hard” data, but, says Dr. Wright, culture is exactly what may be holding back the world of orthopedic research. “By ‘culture’ we mean how orthopedic surgeons address and resolve clinical questions, respond to clinical controversies, the degree to which they are enthused about participating in randomized trials, and their responses to the results of randomized clinical trials.”

Beware the Case Series

Donning a cultural anthropologist hat, Dr. Wright explains, “Cultural changes emanate from altering events such that people can have new experiences that let them think about things differently. Surgeons have largely functioned in isolation with their own strategies, something which has manifested itself in an excess of case series (which by lacking a control group limits the ability to answer the question, ‘What is the best treatment option?’). The case series served us well for years, but it is useful for grabbing the low hanging fruit in the field. When the benefit of a new treatment or technique is so overwhelmingly obvious we don’t need a RCT.”

If you ask any orthopedist if he or she is a lifelong learner, you are likely to receive a “yes” in response. But is that always the case? To learn something new, you must be able to let go of the old. Dr. Wright: “Those of us promoting evidence-based orthopedics often hear, ‘I don’t need to do a RCT. My case series approach works just fine.’ But what they are overlooking is that as differences between treatment outcomes become smaller, then there are other factors to explain why one treatment may be better than another. That is a high hanging fruit and so you need a better method. That better method is controlled studies such as RCTs.”

We all use the past to interpret the present and create the future. Surgeons, says Dr. Wright, are no different. “The personal experience of orthopedic surgeons has understandably been held in high regard—it has also resulted in the overabundance of case series. The problem is that this type of study only creates more uncertainty with regard to surgical success and other issues…not to mention that if you get 10 experts in a room you could well have 10 expert opinions.”

Medical education involves learning, alright, but not so much about learning to be comfortable with not knowing. Dr. Wright: “It is harder to embark on a major surgical intervention if you can’t say with certainty that you are proceeding in the right manner. But alas, the surgical personality is to go forward with complete assurance. While the healthy approach is one of equipoise, namely, comfort with recommending more than one treatment option, it is often regarded as a sign of weakness. My message to my colleagues who will listen is,

Evidence based medicine is not only the principle of using the best evidence available…it’s acknowledging when we don’t have the evidence.

But how to change human nature or, more specifically, surgeon nature? “One of the strategies was to bring opinion leaders to this symposium, have them begin the conversation, and then have them take these issues back to their institutions. Indeed, many participants made a personal commitment to continue the conversation at their institution, letting their colleagues know how important high level research is.”

Raise Journal Standards

A key issue says Dr. Wright, is the extent to which peer review journals will raise their standards. “We must get the journals on board. Fortunately, Dr. James Heckman, whose was editor of JBJS at the time, came to the symposium and shared his ideas on the importance of evidence-based medicine. Journals are in a position where they can only publish the best material they receive; on the other hand, they can signal the kind of material that they are looking for. My goal is to reach the point where journals say, ‘We will no longer accept case series’ or at least, ‘We will publish a case series only under XYZ conditions.’”

As things so often do, this—on some level—relates to the human ego. “No one likes to be wrong, ” states Dr. Wright, “and the fact is that when you present the results of trials they are controversial to a great extent because a wide swath of the audience is wrong. For example, several years ago the results of a high profile clinical trial were published in the New England Journal of Medicine showing that the use of arthroscopy was not useful for most patients with knee arthritis. That is obviously not going to be a popular outcome with a lot of orthopedists.”

Money, Money, Money

And other deterrents to high level research? First of all, you can’t do what you don’t understand. “Due to time and budgetary constraints, the majority of residency programs devote little time to training students in research methodology. And because of reimbursement issues, there is often a financial penalty for those who do become surgeon-researchers. Also, the fact is that being the principle investigator on a randomized trial is going to take between one to three days of your time per week. As it stands now, it’s a lot of time and effort for what may be seen as not a lot of reward.”

Especially, says Dr. Wright, if the institution views clinical trials as a headache. “Your hospital may not support your work by giving you time and/or research assistance, thus making a difficult process even more challenging. A wise hospital administrator will take the long view, and realize that clinical trials end up saving money because clinicians then do the right things the first time around.”

And if you’ve landed a meeting with the hospital CEO? Dr. Wright advises, “Try to help them see that conducting a high level clinical trial only enhances the reputation of the institution.”

Explain that the trials improve care, and with improved care comes more satisfied patients…patients who will return to your institution. Highlight the ways that a trial is in line with the hospital’s strategic directions. Furthermore, those who are motivated to do research are some of the most thoughtful individuals in the field. Let them know that their hospital could attract the cream of the crop if it were known for being supportive of these large research endeavors.

“Even if you have institutional support, running a trial is a huge undertaking, complete with patient enrollment problems and issues related to maintaining morale and organization at different sites. It is truly frustrating if the other investigators involved don’t obtain enough patients, don’t follow-up on all patients, or deviate from the protocol.”

The Intrepid Commitment to RCT Research

So as it is now, only the most intrepid surgeon-researchers will swim in these waters. Dr. Wright: “There are several strategies that can be used to lessen the burden on those who are willing to conduct clinical trials. First of all, the trials can be designed in a way to streamline logistics. For example, many aspects of routine clinical care are components of the required information and processes for an RCT…by harmonizing clinical care with an RCT both processes can be made more efficient. Secondly, institutions need to step up to the plate and provide a level of financial compensation for research that is commensurate with that of clinical activity. Ultimately, the solutions will come from the specialty societies whose members will eventually say, ‘enough is enough.’”

Dr. Wright: “If we do not find and implement concrete solutions to these issues, surgeons will continue to look at clinical trials and think, ‘This is unreasonably difficult. I’m going to stick to surgery.’”

And without high quality research, where we will be? Undoubtedly, going backward.