Alternative medicine research has exploded in the 30 years I have studied it. In 1991, the Medline medical research database listed 90 articles in the category of “complementary and alternative” medicine; by 2021, the figure had risen to 1,664. Many of these articles also report positive results. Alternative medicine bears the superficial characteristics of a flourishing field.
However, once you have carefully considered the plausibility and quality of the research – which I regularly do on my blog – you are bound to disappointment. The vast majority of positive studies on alternative medicine lack of scientific validity.
This, of course, raises the question of who is responsible for this situation. In the first place, the fault clearly lies with the researchers and their supervisors. Then there is the funder of the project. Yet studies of alternative medicine are often conducted without external funding or with the support of private donors, bypassing the scrutiny of grant committees.
However, since 1991, all research involving human subjects must be subject to independent evaluation of its ethical acceptability. You could argue that this is an entirely separate question, but I disagree. Poor quality research is ethically unacceptable. It is a waste of resources. It abuses the cooperation of study participants. It encourages patients to make erroneous and potentially dangerous therapeutic choices. And, ultimately, it undermines trust in science.
Therefore, ethics committees that approve low-quality alternative medicine studies also bear some responsibility for their flaws. Unfortunately, I have the impression that studies on alternative medicine are considered less serious than other research and are therefore judged less harshly by research ethics boards (REBs), who consider their main task to be prevent harm to research subjects.
The possibilities of designing alternative medicine studies such that they generate a false positive result are almost endless. For example, a recent trial tested whether visceral osteopathy (VO) is effective for polycystic ovary syndrome (PCOS). Thirty women received either VO plus a low-calorie diet or the diet alone without further treatment. The authors found that the VO group had “greater improvement in menstrual pain, irregularities, and premenstrual symptoms.”
Another essay tested acupuncture as a treatment for obesity. Fifty women were treated with acupuncture, while the control group received no treatment. The authors found that acupuncture “promoted a reduction in abdominal waist circumference, suprailiac and abdominal skin folds, and body and abdominal fat percentage.”
A third study tested an Ayurvedic drug, AYUSH-64, on Covid infections. Eighty patients received either the remedy plus conventional care or conventional care alone. The authors found that “AYUSH-64 as an adjunct to standard care is safe and effective in accelerating clinical recovery in mild to moderate COVID-19 cases.”
It is easy to see how these conclusions could mislead us. All three studies are “randomized clinical trials” and therefore may appear reliable. Still, they have major design flaws. More importantly, none of them attempted to account for placebo effects. Arguably, with implausible treatments, it is more likely that the results have nothing to do with the therapies being tested but are entirely due to those effects.
In all of these cases, the flaws would have already been apparent when the protocols were subjected to ethical review. Yet the RECs obviously endorsed them. This means that the committees failed in their duty to ensure the ethical acceptability of these research projects.
The problem could be trivial if it only arose in isolated cases. However, the literature on alternative medicine is littered with fatally flawed studies. In fact, it is difficult to find an alternative medicine trial that is rigorous among the abundance of studies with serious design flaws leading to unwarranted positive conclusions.
If REBs ensured that studies with serious design flaws were modified or dropped, these problems could be avoided. Currently, REBs around the world rarely include experts in research methodology and seem too lenient with alternative medicine studies.
Therefore, I urge RECs to ensure that they are equipped to adequately check whether research projects are properly set up to find a conclusive answer to the research questions they purport to address. For studies testing alternative treatments, this scrutiny should be just as rigorous as for the rest of medical research.
Edzard Ernst is Emeritus Professor of Complementary Medicine at the University of Exeter.