Using systematic reviews to reduce research waste—who really cares?

Written by Kamal Mahtani, NIHR Clinical Lecturer in General Practice, Nuffield Department of Primary Care Health Sciences, University of Oxford. First posted on the BMJ Opinion site.

The global spend on biomedical research and development is estimated to be about $250 billion (£203 bn; €233 bn) each year—a not insignificant figure. In fact, it roughly equates to the amount that the UK government spends each year on its combined education, defence, and welfare budget.

But suppose you heard that the UK government’s budget for education, defence, and welfare was being wasted and provided no public benefit at all—what would your reaction be? Surprise? Indignation? Anger?

Chalmers and Glasziou’s revelation that about 85% of biomedical research may be wasted should evoke at least as great an outraged response. This estimate of wastage is derived from a number of factors—including failure to conduct, describe, and publish research to standards that make the outputs useful. This is particularly concerning, given that much of this research is expected to provide some form of patient benefit.

Among the proposed steps to reduce this waste is a recommendation that new research should not be conducted until a systematic assessment of existing research has been performed. There are at least two reasons why this makes total sense. Why would you want to conduct a new study if the answer was already available? And, more importantly, if the answer was available already, would it even be ethical to enter participants into a new study when some might fare worse?

Some public funders of research, such as the UK’s National Institute for Health Research (NIHR), which manages an annual budget of about £1 billion for applied research, recognise the importance of systematic reviews in reducing research waste. As one example of this, new applicants seeking funding from the NIHR to support primary research must now ensure that their request is informed by knowledge of the existing evidence.

But what about other funders? The aim of a recent survey of the websites of 11 international research funding organisations was “to indicate the extent to which each organisation adopted waste reducing policies and processes.” The authors extracted information on whether the funder involved members of the public in setting research priorities; made full protocols for funded research publicly accessible; and had specific funding streams for methods research. The survey also identified information about how the funders supported the conduct of systematic reviews.

Although the authors reported that they used their judgment in the interpretation of their findings, they do also report that they checked the accuracy of their findings with each organisation. According to their survey results, only two of the funders, the NIHR and the Canadian Institutes of Health Research, had clearly dedicated streams to fund systematic reviews. The survey also concluded that only one funder, the NIHR, made it a prerequisite that all applications for support should be informed by a systematic review of the existing evidence.

Scientific, ethical, and economic reasons make it essential to conduct a systematic review of existing evidence before considering a new study. Failure to do this is simply poor practice. This recent survey shows that there is still work to be done to ensure that this principle is clearly and demonstrably supported. Given their large budgets and major influence on what does and does not get financial support, funding agencies have a particular responsibility to lead this initiative.

Kamal R Mahtani is an NHS GP and deputy director of the Centre for Evidence Based Medicine, Nuffield Department of Primary Care Health Sciences, University of Oxford. You can follow him on Twitter @krmahtani

Declared interests: I have received funding from the NHS National Institute for Health Research to conduct independent research, including systematic reviews. I am also the director of the evidence based healthcare MSc in systematic reviews and a founding member of the EBM manifesto team, which advocates better evidence for better healthcare.

Disclaimer: The views expressed in this commentary represent the views of the author and not necessarily those of his host institution, the NHS, the NIHR, or the Department of Health.

Acknowledgements: Jeffrey Aronson and Iain Chalmers for helpful discussions.

Cross posted from the BMJ Opinion site