It is with great excitement that I am able to present the following discussion as evidence that the current “discussions” of many topics online are not the only way to have a conversation on the web while disagreeing. I had the pleasure of a prolonged, intelligent discussion with Dr. Ben Howell. The initial discussion was presented as a comment on Quora, a website that allows easy access but poor visibility to search results. So that this discussion can receive the visibility it deserves, I have Dr. Howell’s permission to repost it here.
To summarize: Dr. Howell and I disagree on the merits of homeopathy, and expand that discussion to homeopathy from the original Quora Question. I will present the entire thread as it appears here, without interruptions.
Initial question (by anonymous):
How can the Sabadil be said to be working under some placebo function when the other medications didn’t work? If placebo was the trick here, wouldn’t the other three work as well?
Individuals who, having read a blog post or two, decide they have become experts in the field of homeopathy will make most of the claims. In my experience those who make the most amazing and derogatory assertions know the least about the subject.
Typically, those arguing for placebo-only will claim that homeopathy has been proven to have no effect. By doing so, they fall into the illogical null hypothesis fallacy. In science, a negative cannot be proven. The most definitive position they could claim is that homeopathy has never yet proven to be effective. They will also say that homeopathics are nothing more than water, which is false in terms of the homeopathic Boiron Sabadil.
A momentary look at the ingredients of Sabadil () will show that all of the ingredients are present in the final product, having only undergone a 5C dilution. This is well below the threshold that classical chemistry assigns to molecular functioning. So any claims about dilution negating any molecular function are simply generalizations that do not apply in this case.
As you note, your subjective results have followed the scientific method of experiment and results. You compared various treatments in your own body, and came to the conclusion that Sabadil was the most effective treatment for you. Your results, by the way, are shared by millions of other people. Sabadil has replaced allergy medication in many households, and many people who cannot take allergy medication have been helped.
In your particular case, those who would argue a placebo-effect have no leg to stand on. If they truly understood the placebo effect, they would remain silent and applaud you for having resolved your allergies with such a non-toxic solution. But they do not, and I do not. We have a new field of placebo effect study, which pushes the limits of what we understand about healing and the body. Based on the newest research, saying something is a placebo is equivalent to saying a substance has long-term, significant health benefits that rival and may exceed drug interventions with fewer side effects. In other words, they are saying that homeopathics are possibly better than the drugs, which is not what I think they intend to say. I go further into this fascinating topic here:
What you have asked is really: what is going in my body? How could you generate a placebo effect using a homeopathic and not have generated the placebo effect using one of the highly marketed allergy drugs? In the blog post above, the Wired article talks about how direct-consumer drug advertising is making it difficult for manufacturers to outperform growing drug placebo effects. So if anything was going to give you a placebo effect, it should have been the drugs. You are correct in your assessment, unless you have neglected to mention a blind faith in homeopathics or that you have been exposed to numerous homeopathic advertisements without equivalent exposure to drug ads. It is unlikely that Sabadil generated a placebo effect when the drugs did not.
While it is unlikely, your results are simply a trial of one. To prove that Sabadil is not simply placebo, we need evidence. A study or trial that shows Sabadil outperformed placebo, or simply that homeopathics outperform placebo pills in allergic patients. With a single good trial, all of the placebo-only arguments should fade away.
Unfortunately, what I have found is that in the blog-educated, no study is ever acceptable. The first line of argument is that there are no studies. The second is that you only have a few studies. The third is that the studies are all falsified. As I said, this is a belief system, not a rational line of inquiry. I have been fascinated by the labyrinthine lengths someone will go to not believe something. What I find most disturbing about this reluctance to engage in scientific discussion is that the nay-sayers’ claim of “absolutely no effect” is a dangerous and possibly lethal medical position. When Mr. Randi famously swallowed homeopathics in a TED talk, he was engaging in dangerous, unethical behavior that might harm a naive copycat. ()
But for you and I, evidence is evidence. If something has been tested in a randomized, placebo controlled trial, (RCT) and that trial has met the standards necessary to be published in a peer-reviewed medical journal, that is the medical standard. If it were a drug, it would be sold by pharmacies and marketed heavily.
Does homeopathy for allergies meet that criteria? Yes. Individually and in conjunction with conventional treatments.
A more dilute form of homeopathic was effective in a small RCT for cat allergies:
A larger analysis of multiple homeopathics showed benefit:
In a long-term, three year study of atopic eczema in children, homeopathy was equivalent to standard treatment:
In asthmatic children, homeopathic additional treatment improved outcomes:
An Austrian study showed improvement in patients given homeopathics in addition to conventional treatment:
A Belgian study showed benefits:
I hope this answers whether Sabadil might indeed have the effect you experienced and why many individuals will be unable to believe your results.
I read your answer with great interest as it is certainly my understanding that homeopathy has no evidence to support its claims.
I’m not interested in debating the philosophical arguments that you made – except maybe against your statement “those arguing for placebo-only will claim that homeopathy has been proven to have no effect”. This is not the claim that opponents make and nor is it the responsibility of people to prove homeopathy doesn’t work. If proponents of homeopathy want to claim it is an effective treatment for a particular disease then the onus is on them to supply evidence that this claim is true – not for opponents to prove that it doesn’t.
This leads me to the studies which you quoted. The placebo controlled study on cat saliva is extremely interesting. Although the sample size is very small, and they measured wheal size and made no comment about clinical symptoms, it would appear that this study does show an effect above placebo. Do you know whether the results of this study have been replicated and whether they have extended the study to look at symptom relief?
Finally, I hope I don’t cause offence, but I would politely suggest that you remove the remainder of those studies from your answer. They’re of very low quality / low level evidence and I think they detract from the potential shown by that placebo controlled study. Even the study on eczema compared to conventional treatment is too poorly done to be of any use – no blinding, the doctors making more profit from the homeopathy group than the conventional therapy group, no description of what “conventional therapy” means and permitting the use of “rescue” steroid creams in the homeopathy group.
Thanks for the comment. I just selected a few of the first studies I came across when I searched medline for “homeopathy allergic” (256 results) and aimed more for a diverse selection of journals rather than quality of studies. Since most of the RCT studies are published in the Homeopathy Journal, other commenters on other answer streams have stated that they don’t believe anything that journal publishes. So these are in answer to that concern. Saying a prospective study is worthless, particularly one run parallel to conventional therapy, belies another critique, which is that homeopathy may be effective but has nothing to offer when compared to conventional therapy. As you can see, I simply can’t please everyone, all the time.
If you peruse the answer streams of homeopathic questions, you will find answerers making the claim of “proven to have no effect” and “only water” constantly, often with colorful statements attached. I do not think it is possible to prove something doesn’t work, only that it has not been show to work yet. I certainly don’t give those who make the claim the burden of proving an impossibility. I merely point out that in discussing homeopathy and placebo, most people with very strong opinions aren’t open to evidence to the contrary.
As you may note from my answer, I am entirely open to the concept of homeopathy as a “doctor-patient placebo effect” but if you look at my blog post, the meta-analyses of placebo, and the Wired articles, the pharmaceutical community is coming to the rude awakening that many drugs (particularly those acting on the CNS) may not outperform the “doctor/patient placebo” over time. If that is the case, then the derogatory statement “just a placebo” does not mean what it did in the past.
Your request for more studies as a GP is easily answered, as a simple search for homeopathy on medline today turns up 4969 results. We also have warring meta-analyses of homeopathics declaring some positive results and no positive results. On this issue, I do not believe that Cochrane can be objective, as one of women who selects and validates studies at Cochrane is also a vehement and vocal blogger against homeopathy. I have spoken to her supervisor, but he believes she can separate her personal and work views on this issue.
Rather than make you slog through the other studies, let me pick two.and
Please have a look at them and get back to me with your thoughts.
Firstly, sincere thanks for selecting two studies for me. I’ve written elsewhere about this, but a common tactic that I’ve encountered when debating evidence about a topic is that of the “reference bomb”. That is, people trawl the internet for any study that remotely relates to their area of interest and then forward the entire list. I think the assumption is that the sheer volume of studies will overwhelm the recipient into accepting that at least one of them proves the point that they are trying to make.
Unfortunately, I believe in the common courtesy of actually looking at the individual studies and it doesn’t take long to discover that they do not support the claims being made. Therefore, I truly appreciate you selecting just a couple of good examples that are relevant to the topic we are discussing. I should also stress that quoting two relevant articles holds far more weight than pointing out there are 4969 results on medline.
I do think that one of your two studies is extremely interesting, however, before discussing this in detail I would like to explain the difficulties you have faced when debating evidence about homeopathy.
I should begin with a word of warning. I would caution that the following is extremely long. I apologise up front but I hope you take it as a compliment. You have struck me as an alternative medicine practitioner truly interested in the evidence behind your craft and I wouldn’t have bothered taking the time to pen this had I believed you were closed to the idea of evidence based practice. Also, I hope my conglomeration of various practitioners under the term ‘alternative medicine practitioner’ is OK, and doesn’t cause offence, because it is not intended.
Firstly, I believe it is a common misconception by alternative health practitioners that they face abnormal and unfair criticism of research done in their area of interest. I’m not sure you will believe me but I would like to reassure you that this is not the case. You are facing the same criticism and barriers that doctors level against research done on any therapeutic intervention.
Within medicine, discussion of any research is a bloodbath! Doctors attend meetings called “Journal Clubs” where they get together to look at recent studies, in their area of expertise, to determine whether there is any new evidence that should change their clinical practice. There is usually coffee and pastries at these meetings but that’s where the civility ends. There is a correct formula for conducting a good trial and, in these journal clubs, research is cruelly torn apart for any sign of weakness and, if found, the results of that study are ignored.
In my opinion a lot of alternative health practitioners breeze out of their yoga class and into this Colosseum and then take offence when the gladiators are swinging swords. If you are truly interested in competing in the arena of evidence-based practice you need to accept that the gladiators aren’t persecuting you. They are swinging swords at everybody.
Now it may not seem fair that, in these journal clubs, doctors quickly dismiss the results of a randomised controlled trial. After all, this is the highest level of evidence so surely the result must represent the truth? Unfortunately, this is not the case.
If we just focus our attention on randomised controlled trials that are perfect – proper blinding, actual randomisation, no confouding factors or bias, large numbers and small amounts of drop-out. Within these perfect studies 5% of statistically significant findings are wrong.
The usual definition of statistical significance (p value < 0.05) means that there is only a 5% chance that the result is a complete fluke. This means that for every 20 perfect studies that are run to see whether paracetamol cures cancer the chances are that one of them will find a statistically significant effect.
Now it is also important to know that there are around 500 new randomised controlled trials published around the world every day. 26,000 per year.
I’d also like to highlight that the above numbers are only forrandomised controlled trials. These numbers don’t include the far larger numbers of lesser quality evidence.
So, if we combine the above statistics – if there are 26,000 randomised control trials published each year, and ALL of them are perfect, then 1,300 of these perfect studies found a statistically significant result that was completely incorrect. That is 3.5 perfect studies published every day with ‘proof’ that is completely wrong.
And, this is just if all the studies are perfect. I hope it’s obvious that running a less than perfect study increases the chance that your statistically significant result is incorrect. Therefore, when doctors are evaluating the quality of evidence, if a randomised control study isn’t perfect there isn’t time to worry about its result. The result will be ignored, and judgement will be reserved, until the perfect study is run. It doesn’t matter whether the study is on chemotherapy, homeopathy or physiotherapy – the results of a poor quality study are irrelevant.
In fact, in evidence based medicine nobody even knows what a poor quality study actually found. People who are not used to the process of evaluating evidence often make the mistake of reading the results first. Anyone who does this needs to stop. It leads to conclusions such as “Even though the study had flaws it still showed a statistically significant result so it suggests that there is some evidence for my opinion”. In evidence based medicine you completely ignore the results until AFTER you’re satisfied the research is of sufficient quality. If the design and methodology of the research isn’t up to scratch then you cannot make any judgement about whether the results are meaningful in any way. It could be right, it could be wrong, but there’s not enough time to worry about it as I’ve got 499 other studies to get through today.
I also hope the above statistics make it obvious why quoting lower levels of evidence to support an argument is a waste of time. You could quote a million case studies and longitudinal studies to support your argument but they are trumped as soon as a few randomised control trials disagree with the results. I’m not saying that lower levels of evidence have no purpose – they are important stepping stones on the way to truth, but, NOBODY in the evidence-based Colosseum would make a conclusion based on these lower levels of evidence.
Unfortunately, a lot of alternative medicine research makes this mistake. It is unfortunately common to see nothing other than lower levels of evidence supporting an argument and to read recommendations for clinical practice based upon it.
For example, at the end of your second study the authors admit “larger sample sizes be used in future homeopathic research to ensure adequate statistical power”. The authors admit that the study is under-powered to draw any conclusions from it and yet their very next line is “Homeopathy should be considered for use as an adjunct to oral rehydration for this illness”. In the arena of evidence based medicine this conclusion would NEVER be drawn. A pharmaceutical company who tried to market it’s product on this level of evidence would face millions in fines.
In the evidence-based arena you simply cannot jump to conclusions about clinical practice until the effectiveness of that treatment is proven in high quality trials.
I haven’t located the original of your study to unleash the full fury of evidence evaluation onto it, but, let’s assume the methodology of that study is perfect. Reading this research (and this is being kind, in the comfort of my living room, rather than the arena of a journal club) I would interpret its findings as this …
“This is a very interesting and unexpected finding. Unfortunately, the study is under-powered to be able to draw any conclusions from its findings. However, it certainly warrants further investigation with larger sample sizes. I look forward to seeing whether the findings are true or not”.
Until the producers of this product supply a study that DOES have sufficent statistical power to prove its effectiveness they won’t be able to land a blow in the Colosseum. In short, no doctor who practices evidence-based medicine would prescribe this product until better studies are published, and, any phamaceutical company who tried to recommend it to doctors would be hammered with a massive fine.
So, if homeopathy wants to compete in the domain of evidence-based practice then it NEEDS to focus on some of these positive findings in lesser quality studies, like the two you have shared, and run high quality studies that prove the results are actually true. More and more lesser quality studies aren’t going to help your argument. Instead of bombarding people with lots of lower quality evidence the focus should be on conducting a few higher quality studies – which, as far as I’m aware, is lacking to date.
A nonsense argument I’ve heard from alternative medicine practioners is “There’s no money in it for Big Pharma so the studies will never be run”. As I’ve said previously, it’s not up to Big Pharma to run studies on competitors products. Big Pharma MUST produce high quality studies to support claims about its own products before it can sell them. It is up to the producers of homeopathy products to do the same. Alternative medicine is not a cottage industry – as I said in my answer, Boiron had first 6 monthsof 2014. There is no reason that they cannot use a fraction of this to run studies to prove the efficacy of their product. There are 26,000 other groups publishing randomised control trials every year – it can’t be too hard!in sales in the
Furthermore, if the homeopathy community is truly committed to evidence-based practice then they should be demanding this from the producers of their medicine. Doctors refuse to accept treatments until confronted by high quality evidence. Alternative medicine practitioners could demand the same from their own pharmacology industry and refuse to prescribe anything until the conclusive evidence is published. Only with this motivation will producers of alternative medicine products be forced to invest in proper research. Until then they will just continue making profits from products that have no good evidence as to whether they are effective or not.
Moving on … I would like to comment on the criticism you have faced about the journal Homeopathy. Again, the difficulties you have faced are not unique. A disregard for certain publications is extremely common in medicine. The fact is that some publications apply stringent evaluation of submitted research before publishing it – taking pride that the research in their publication is of the highest quality. Other journals don’t … and, as a consequence, are rarely read.
You may wonder why it matters where research is published? It’s because of those 1,300 perfect randomised controlled trials run every year that are wrong. If a publication selects only positive studies in its area of interest – and ignores studies that contradict those results – then it increases its chances that it is publishing these “wrong” studies. Now, I have deliberately not looked into the Homeopathy publication – because I don’t want to be accused of slander – and therefore cannot comment on its practices. However, if it just a collection of positive studies – and fails to investigate whether there are similar studies that failed to find a similar result – then I’m afraid it will never hold much weight amongst the gladiators.
And, again, this type of disregard is common. I probably shouldn’t share this in public but I have worked with a specialist who ignores any study from an entire country! In his area of expertise there is a group in this particular country who publishes an extraordinary amount of statistically significant randomised control trials. Unfortunately, no group outside of that country has ever been able to replicate their findings. So he refuses to read anything produced inside those borders.
You are probably not aware but the field of evidence-based medicine is not merely a single Colloseum but a multi-continental war. There are entire nations of doctors practising medcine based on the results of their own randomised control trials verus the rest of the world who rejects their findings because they have never been reproduced outside of that country. In the arena of evidence-based medicine there are bloody wars being raged between specialists inside their own field … and then people from alernative medicine wander into a corner of the arena armed with flowers and complain that it’s all a bit too rough.
And this is the final hurdle of evidence based medicine. Whenever you run a perfect randomised control trial it is expected that its results will be able to be reproduced 19 times out of 20. It is not enough to have a single perfect study. Studies are expected to give sufficent detail about their methodology so that any other group in the world can run the same trial and get the same results. Any study that is unable to be reproduced is quickly cut down in the arena of evidence.
To quote an infamous example – in 1998 The Lancet (a highly respected medical journal) published research from a medical doctor, Dr Andrew Wakefield, that linked the MMR vaccine with autism. The publication of this research had fatal ramifications, but, it turned out no other group was able to reproduce his results. On closer inspection, it was discovered that his research was fradulent. Unfortunately, the myth still persists – by people with no understanding of research and evidence – but it highlights that one good study is not enough. You need to be able to stand up against independant people checking your work.
And, for this reason, I would respectfully suggest you reconsider your arguments against the Cochrane Collaboration. What this organisation does is apply a very specific formula for performing meta-analysis on multiple randomised control trials to try and minimise that 5% error rate. I don’t profess to understand this process but there is a defined way to run a meta-analysis – just as there is a defined way to run a good quality randomised control trial. Arguing that the results of this calculation is affected by the attitudes of the people running it is akin to arguing that a mathematician decided that 2 + 2 = 4 because he doesn’t like odd numbers.
If, as you suggest, there are multiple meta-analysis that have been performed on the same data, and then found conflicting results, then I’m afraid that one of them has not adhered to the correct method of running a meta-analysis. In short, one of them is wrong. Maybe their maths is wrong? Or maybe they have included trials of poor quality. After all, the quality of a meta-analysis is only as good as the randomised control trials selected to go into it. You need to evaluate the quality of the randomised control trials – as discussed above – before you can decide whether they warrant inclusion into a meta-analysis. If a randomised control trial has massive methodological flaws then its raw data is unusable.
However, if you believe that the results of the Cochrane meta-analysis is wrong then there is a very simple way to argue against it – get the data that they collected and run the meta-analysis again. If you find that their maths is wrong then you have a valid criticism against them. Arguing that their maths is wrong because of their personal beliefs won’t hold much weight in the Colosseum.
Finally, I should close by commenting on the second paper you supplied. I hope I have invested sufficient time and energy into this comment that you’ll forgive me swinging a sword, but, it’s a study on animals – not even close to being able to draw conclusions about treatment in humans. I didn’t even make it past the title.
All the best,
After such an outpouring of effort on your part it is difficult to know how to respond. Part of me wants to honor your effort and match your eloquence. Another part wants to laugh or cry.
Distilling down your argument, to meet your standards an alternative medical study would need:
1) to be perfect, beyond reproach in any way. Large, with a robust significant effect, a human population so homogenous as to avoid confounding, and a set of researchers capable of remaining double-blind while watching for any possible misstep.
2) It’s not enough to be perfect. We need to repeat this perfection many times over, in multiple labs, in multiple locations in the globe. These results should be as robust for the New Guinean tribesmen as for the Manhattan globetrotters. Anyone, anywhere, should be able to obtain equal results. Ideally, to meet these standards, a perfect study would be repeatable nineteen times out of twenty.
3) Even the perfect study will fail one time in twenty, simply by statistical fluke. In its perfection, there will still be flaws enough that a negative result may occasionally occur. This negative result, which destroys the perfect score of the perfect study, is to be expected. Yet despite this built-in flaw, we somehow know that the study is perfect.
4) Until we know the study is perfect, and repeatable nineteen times out of twenty, no action should be taken on whatever is being studied. “In evidence based medicine you completely ignore the results until AFTER you’re satisfied the research is of sufficient quality. If the design and methodology of the research isn’t up to scratch then you cannot make any judgment about whether the results are meaningful in any way.”
I’m not sure how to even begin to answer this set of standards. Can I begin by applauding your Colosseum and your gladiators? Truly, if this is the edifice of evidence based medicine we have nothing to worry about in the field of medicine. No medication will be applied to a patient, no treatment used, and certainly no surgery done without an extraordinary level of work.
Could I please have a list of the medications and procedures you use? Truly, I need to have a list of pharmaceutical drugs that meet these stringent requirements. I cannot believe I haven’t found such a list before. Even on the Cochrane Database, the researchers mention the need for more studies in 96% of their meta-analyses. They have not achieved the level of stringent, reproducible data that you and your fellow gladiators have. I promise to reproduce and publicize your irreproachable list, to use it as a bible going forward.
How many years have you spent studying statistics? I simply was not aware of the level of statistical vigor being added to the medical curriculum. As a mere general practitioner, I would not dare to dissect studies done by men and women who have spent their lives in statistical learning. Truly, I am in awe of someone who can maintain a busy practice and also takes the time to read everything produced in his field. It is all I can do to simply become aware of some of the major advances in alternative medicine, and every month I feel I fall farther and farther behind in my reading.
I am puzzled that a man with your omnivorous breadth of learning would cite Dr. Wakefield’s MMR research as a fraudulent study. Surely you know, as I do in my own humble exploration, that Dr. Wakefield’s study was only repeated once, in another small study on measles in the gut tissue. While that second researcher did not find MMR measles, she did find one child with wild-strain. The other studies done to discredit Wakefield were prospective studies, of no merit whatsoever based on your guidelines. Then the BMJ hired an investigative reporter to assassinate Dr. Wakefield’s character in its pages. A quick review of the original study and subsequent studies should be no issue for someone like yourself who can go through five hundred studies in a day.
Yet this oversight on your part gives me pause. Perhaps this Colosseum, these gladiators, these voluptuous perfect studies are all Platonic ideals. I begin to think of what little I know of human studies, and of humanity at large. These men and women in studies may be asked to avoid this and take that, but how many truly follow those guidelines? It’s not like an animal study, where you can confine your experimental base to cages and watch them all day long. Some of these human subjects taking a pain medication will go out and overuse their already damaged knees, thus negating the effect of your expensive drug. The housewife, deprived of her favorite carbohydrates, will indulge and overturn your weight loss medication. Again and again I am faced with the ugly reality of the human condition, and how it will confound even the most omniscient and omnipotent researcher.
Let it not be so! Send me your list of perfect drugs posthaste, as I find myself backsliding, agnostic already of this evidence based medicine you speak of.
But let me muse a bit further. What if such a list does not exist? And even if it does exist, even the perfect drug, birthed of the perfect study, will fail me once in twenty patients.
Let us assume, for a moment, the failure of RCTs to prove or disprove anything. If I present you with twenty perfect studies of homeopathy, there is a statistically significant risk that my results will still be due to some unforeseen confounding factor. Although I agree absolutely with this logic, we are left with nothing on the table and nowhere to go from there. Reality itself may be simply Samsara, and you and I merely delusions. But surely we have truly left the edges of science and delved deeply into the nature of reality. Science must be based on these building blocks, and while these building blocks are imperfect, we cannot discard them simply because of their imperfections. I remember R. Bausell’s book, Snake Oil Science, in which he discards the .05 significant marker for alternative medical studies and dismisses them all. In passing he notes that much of pharmaceutical medicine will fall by the same sword, but I have yet to see his expose on how few drugs really have good data. If we are going to use the sword, it swings both ways, and in the end we are left with very little with which to practice medicine.
Huge prospective studies don’t follow the RCT experimental model, but they give us “on-the-ground” information with huge numbers that will find side effects that even a perfect smaller study will not. I remember the bloodbath around the WHI study, a huge study that has now borne out with lowered breast cancer incidence after we pulled millions of women off extra hormones. But I remember heated discussions with a female ob/gyn doc who claimed she not only didn’t think the study was valid, she was looking forward to taking hormones. WHI was enormous, and has borne out in many follow-up studies, but we still have significant resistance in the ob/gyn community and I still deal with 80-year-olds having their periods again. Sometimes we have to use the preponderance of the evidence, without waiting for a “prince charming perfect study” to come along and save us.
When you say: “The authors admit that the study is under-powered to draw any conclusions from it and yet their very next line is “Homeopathy should be considered for use as an adjunct to oral rehydration for this illness”. In the arena of evidence based medicine this conclusion would NEVER be drawn. A pharmaceutical company who tried to market its product on this level of evidence would face millions in fines,” there are two assumptions that I must address. The first is that off-label use is not rampant and out-of-control, despite the few fines leveled. Physicians use drugs for all sorts of purposes, and off-label uses are listed in reference manuals. To make the claim that this does not occur and that the fines do anything more than add slightly to the cost of business is to assume a level of integrity that does not exist, and possibly should not exist. Given the choice of doing no treatment and doing an off-label treatment, which of us would reach up and tell the doctor: “please, if there isn’t a set of twenty perfect RCTs for this, don’t give me that drug.”
Which brings me to the second point. Let us leave the evidence-based Colosseum, a place that in my doubting mind only exists in the clouds, and likely right next to the perfect RCT laboratory. Any examination of the Cochrane database will show that, far from conclusive, of all the reviews 96% recommended further research. Unless you have a list of studies and drugs you use that outperforms what the Database has compiled, I feel hopeless applying your standards to medicine. If we were going to wait for evidence-based medicine to give us conclusive answers, we would have nothing to offer the ill today.
Far from the sterile Colosseum, there are children dying. They lack basic nutrition and clean water. They lack a medical infrastructure and a governmental system that will ever support sufficient funding to put a western-style medical infrastructure based on the Colosseum in place. They are dying now. If I follow your argument to its conclusion, I should withhold the only option available, awaiting a perfect study to give me ethical permission while my patient expires? These are not children considering the option of IV fluids or supportive treatment in a climate-controlled children’s ward. My options are to do nothing or use an off-off-label treatment, a homeopathic remedy that may only be a placebo and has weak evidence. I should withhold care awaiting some perfect RCT and the twenty studies necessary for that to occur? Where does death and our oaths to relieve human suffering enter into this equation?
In the Colosseum, it may be possible to live with the decisions of the gladiators and feel the mighty weight of the perfect study. But gazing out on the misery of the world, we can see that the reach of the Colosseum only touches the wealthy few. Those lucky few have health insurance, a medical infrastructure, sufficient doctors, and 26,000 studies done for them every year. They also have disposable income and may opt for a roll of sweets or a homeopathic remedy with their spare change. But neither is essential for them, for they have many options. Gazing farther out, we see three quarters of the world. Three of every four children, three of every four aging older women in pain. The Colosseum does not reach them. They have no medical infrastructure, no access to the perfect drugs. For them, the options are to die or to try whatever traditional medicine they have on hand. In many cases, this is worse than nothing.
Walk with me through the slums, the ghettos, the dumpster cities, and speak to me of the need for more studies. Let us stop a sunken-eyed little boy, carrying a plastic water bottle home to his mother somewhere in this labyrinth of despair. From his soiled trousers it is clear that he is suffering from incontinence. You offer him a heavily discounted, government subsidized, bottle of pedialyte for a few pence. It is still more than he makes in a day, and will need to be repeated tomorrow. He must choose between your medication and eating. I offer him a few drops of homeopathic remedy gratis. Due to its dilution, it can be dispensed for nothing. He accepts my offer, because he can afford to. One of us has upheld his oath to relieve suffering this day, the other is waiting for more studies to act.
Again, I applaud your effort, as you have given me much to think about and I have spent a day musing on my reply. If you really have a list, I would love to see what you use daily that meets your standards. I would also like to wish you a very happy holiday season.
With great respect (truly),
Firstly, I’m afraid my analogy has caused some confusion – for which I truly apologise. I seemed to have sparked some unnecessary despair and I hope I can undo any confusion I have caused and be clearer about the points I was trying to make.
Firstly, however, I would like to clairfy my personal position. You are correct that I do not spend my days reading and evaluating original research. I don’t believe I said I did and I apologise if it was insinuated. The fact is that the vast majority of research that is published each day has no relevance to my day to day work. I am also in the fortunate position that research that is relevant, and important to my clinical practice, gets discussed in high quality medical publications that I do read, and, it then gets incorporated into clinical guidelines that I reference when treating patients. The fact is that there are people who do read and evaluate all of those 500 studies each day and they highlight the most relevant information for the rest of us.
That said, I would also like to defend myself against your slights and your doubt about my competence in this area as a “mere general practitioner”. I have 4 University degrees (Philosophy/ Politics; Psychology; Nursing and Medicine) and two of those (Psychology and Medicine) involved fairly detailed, and tedious, study into how to conduct and evaluate research. As an aside, I have also trained in alternative medicine – achieving a diploma at a Natural therapies college in my early twenties and being a member of the Australian Traditional Medicine Society for several years – but this is irrelevant as it involved zero training in research or the evaluation of evidence. As another aside, before medicine I worked for a decade in the Advertising industry where my job was to manipulate people’s fears and beliefs, using lies, to extract money from them. Again, this does not directly relate to evidence based medicine but I certainly know how to manipulate statistics to sell a worthless product.
Anyway, all told, I probably have about 8 years of training and practice, at a University level or beyond, in research and evidence-based practice. That said, I do not believe my background is relevant to our discussion, I’m not intending to boast and I truly believe that 8 years of University training is not required to apply these methods. I would not have mentioned my background but feel it is necessary to combat your suggestion that I have no business commenting on evidence-based practice. I only hope that the above detail is sufficient to prove that, when I choose to assess the strengths and weaknesses of a research paper, I know how to do it.
You mentioned “as a mere general practitioner, I would not dare to dissect studies done by men and women who have spent their lives in statistical learning“. My response to this is “WHY THE F$%K NOT!?!?“. If there is one thing I would like you to take away from our discussion it is that research and evidence is not the domain of Statistics Professors. It is within easy reach of any person who is interested. Furthermore, if you do not have the daring to evaluate and criticise the research of others then I’m afraid you can never sensibly participate in a debate about evidence for your profession. Without learning how to dissect studies you are unarmed when looking at research – there is no way to detect the difference between a quality study and the research equivalent of a shampoo commercial announcing 99% of its consumers had “fuller hair”. It is impossible to create, or even recognise, convincing evidence to support homeopathy if you don’t know what you are looking for. And, without the ability and bravery to dissect studies there is no ability to assess which randomised controlled trials should be included, and which should be omitted, from a meta-analysis. The Cochrane Group certainly have an intimate knowledge of how it is done and I fail to see how anyone can disagree with their findings if they cannot assess Cochrane’s methods?
There are well defined rules onand then widely available guidelines on . Everyone has access to these guides and anyone who interested in evidence needs to invest as much time as is necessary to understand the above processes. Without it there is simply no way to determine the quality of the studies you are reading and no way to understand any criticism that is levelled against them. Without an understanding of these principles I can understand how people are doomed to invent conspiracy theories about how their research is being unfairly discriminated against. To return to my Colloseum analogy – learning this stuff is akin to picking up a sword. It won’t guarantee success, but, without it you’re nothing but lion food.
Do you need to apply this method to all 500 studies published each day? Of course not. Do you need to understand these rules before taking to the internet to debate research? I believe so.
Now I’ve obviously led you astray by mentioning “the perfect study”. You are correct that a “perfect” study does not exist and nor is it required before deciding on clinical practice. By “perfection” I was referring to a study that closely follows the above rules and stands up against the process of critical evaluation. However, the only reason I used the term “perfect” was to point out that even in the best of possible circumstances there is always a possibility that the result of a clinical trial could be a fluke. You cannot run a study that guarantees truth – except a study run on every human on the planet.
I have also created some confusion when talking about the 19 out of 20 studies. I did not mean to imply that producers of homeopathy products need to produce 20 high quality trials before their research will be taken seriously. A single, high quality randomised control trial with reasonable numbers of participants will cause the evidence-based community to take notice. In fact, it will take so much notice that groups around the world will attempt to replicate its findings. My point was that if no other group is able to replicate the original study then it implies there was something wrong with the original and it’s results were – for some reason – false.
I cannot recall stating that we need guaranteed truth prior to action. Evidence based medicine does not mean that only perfect studies are sufficient, or that we have to sit on our hands until 20 perfect studies are run and the truth is undeniable. Evidence based practice is not refusing to act due to fear that a future study will disprove our current beliefs. Current practice does not have to be perfect – guaranteed to be right.
In fact, I’m arguing the opposite. Evidence based practice is a willingness to accept that we don’t know everything and that research may reveal our current beliefs to be false.BUT, when it does, evidence based practice is a commitment to accept new pieces of knowledge and adapt our practice accordingly. It is OK to be wrong about something – provided that you change your opinion and clinical practice when confronted with new and better studies.
As a disciple of evidence-based practice I will acknowledge the effectiveness of homeopathy, and refer patients your way, when evidence is published that adheres to the above rules of randomised controlled trials, stands up to critical appraisal and has not been contradicted by superior levels of evidence. Homeopathy has every opportunity to be accepted, and adopted, by every evidence-based medical practitioner but it is not to making progress along this path.
Unfortunately, I have not witnessed alternative medicine practitioners accepting the best evidence we have to date and altering their knowledge and treatments accordingly. When confronted by the evidence that a treatment is no more effective than placebo they do not admit to the natural human condition of being wrong and change their clinical practice in accordance with this new information. Instead they resort to conspiracy theories and cling to lesser quality evidence to support their faith. Rather than accept that we are learning new truths all the time, and that some things we used to believe have been proven to be false, they instead cling to outdated beliefs and refuse to accept change. In short, I will accept homeopathy as soon as it is proven by evidence, however, there is nothing that will convince an allied health practitioner that their beliefs are incorrect.
Examples are very easy to find. I would like you to consider the following two Cochrane reviews …
1)which found “that no strong evidence existed that usual forms of homeopathy for asthma are effective”. Now I’m not asking for your thoughts about this meta-analysis. I just want you think about how the homeopathy community has responded to this publication and to compare this with the next Cochrane review.
2)which shows arthroscopy does not improve pain or ability to function compared to placebo.
The response of the Orthopaedic Association of America to this evidence was to alter theirto state “We cannot recommend performing arthroscopy with lavage and/or debridement in patients with a primary diagnosis of symptomatic osteoarthritis of the knee”.
You can let me know whether any Homeopathic Association released a similar recommendation to its members about Homeopathy and asthma but I think the approach to these two meta-analysis is a good demonstration of the difference between a commitment to evidence-based practice versus attempts to explain away the evidence.
You have raised the issue of gastroenteritis in developing countries. I presume we are still talking about your second study –
Firstly, your comments suggest that you are working on the ground dispensing free treatment to children in third world countries affected by gastroenteritis. I sincerely applaud your sacrifice and efforts. To be honest, even if the homeopathy additive has no effect on these children the clean water that you are supplying will make a real difference. I do not mean my comments to detract from your self-sacrifice and attempts to make a real difference.
I have had a closer look at the study but I’m not going to delve into the flaws of its methodology – I don’t think it matters. All I ask is that during my following points you remember that your study had 242 children combined together in a meta-analysis and showed a reduction in “symptoms of gastroenteritis” by 0.66 of a day (16 hours). I can accept the results of this study as being true provided you acknowledge that there is no mention about the severity of disease, the number of hospitalisations or the number of deaths. Furthermore, all children were treated with oral rehydration solution – the recommended treatment for gastroenteritis. You have extrapolated from this study that this homeopathic treatment will save the lives of children in developing countries and is an alternative to “expensive oral rehydration solution” but, unfortunately, this study provides no evidence to support your claim. An effect on mortality was not tested or proven and there were no children given homeopathy instead of recommended medical treatment.
Fortunately, for me, gastroenteritis in developing countries serves as an excellent example of how to apply evidence based medicine to a health problem. After my discussion I hope you will agree that modern medicine is not sitting on its hands doing nothing – awaiting perfect evidence – but is heavily involved in interventions that are already proven to have incredible results.
As you are probably aware – working with children affected by gastroenteritis in developng countries – the most common cause of gastroenteritis in children is a virus called Rotavirus. It causes 600,000 deaths around the world each year – 95% of them in developing countries. This is 1/3 of all diarrhoeal deaths and 5% of all deaths in children under the age of 5. Because Rotavirus is a virus – not a bacteria or parasite – there are no antibiotics or other magic pills to eradicate the infection once established. Treatment of an established infection involves maintaining hydration whilst the body rids itself of the infection via vomiting and diarrhoea. As you aptly point out this treatment is a challenge in countries without clean drinking water and access to intravenous fluids.
So, is the answer to throw a placebo at these children in the absence of fluids? Evidence-based medicine doesn’t think so and is instead focused on an intervention that has actually been proven to make a difference – vaccination against Rotavirus.
PATH is distributing vaccinations to developing countries through its. They estimate that acceleation of rotavirus vaccine to children in the developing world would save more than 2.4 million child deaths by 2030. Here is a nice of the difference the program made to diarrhoeal disease and child deaths in Nicaragua – the same country where your homeopathy studies were done.
However, I don’t need to rely on a feel-good video to tell me the vaccine works. I’ve got good quality evidence that proves it.
showed a 98% reduction in severe rotatvirus gastroenteritis, a 94.5% reduction in hospitalisations for rotavirus gastroenteritis and an 86% reduction in clinic visits for rotavirus gastroenteritis following the administration of the vaccine. To prove the efficacy of their product they didn’t have 242 children in their study – they had 34,035 infants in the vaccine group and 34,003 in the placebo group.
And, if you don’t believe this study then I have another.showed similarly positive results for a second type of rotavirus vaccine. This study involved 63,225 children in its randomised controlled trials.
In fact, I don’t just have two studies to support my intervention. Similarly large rotavirus vaccination trials have been carried out on multiple continents to prove that the effects are reproducible on different populations around the world.
Therefore, the question when comparing my studies to yours is, in an evidence-based arena, should we be funding homeopathy treatments that are yet to be supported by significant evidence that proves their effectiveness, or, should we redirect that funding into interventions that have evidence they save millions of lives? By the way, PATH accepts donations on its.
Now … I’m worried that the size of above studies will again plunge you into hopelessness about the hurdle facing homeopathy in its mission to gather evidence to support its claims. I would like to point out that by involving the example of gastroenteritis in third world countries, and suggesting Western medicine was impotent in its efforts to combat it, you opened yourself to some of the largest randomised controlled trials every conducted. I would not have raised them except I feel you levelled an unfair criticism that failed to appreciate the earnest and effectivework of a lot of people to combat the tragedy that is gastroenteritis in the developing world.
To reassure you that these numbers of study participants are not necessary to mount a convincing evidence-based argument I would like to close by responding to your request to supply a list of evidence for the treatments that I prescribe. I’m hoping that focusing on evidence for the top 10 prescription medications will suffice, however, I am happy to keep going if you like? I can assure you that modern pharmaceutical treatments do not make it through the therapeutic goods administration without arming themselves with reasonable evidence for their effectiveness. I am happy to track down a randomised controlled trial to support the effectiveness of any medication that is approved for use by the TGA – I can certainly promise a much better study than is available for any homeopathy treatment.
I would like to point out that the following studies are not perfect and beyond reproach. A few of them are controversial and further research has called their results into question. I’ve selected this group for two reasons:
Firstly, I hope this evidence demonstrates that we’re not waiting for a conclusive answer for every medical treatment before we act. To be honest, I’m amazed that Cochrane has found that the answer to 4% of questions is closed and requires no further research. In evidence-based medicine research is always ongoing and medical treatment will continue to evolve with the findings of new research.
Secondly, I selected these studies because they all mounted a compelling argument at the time of their publication. They caused people to consider ideas that weren’t previously accepted – like ‘Homeopathy is more effective than placebo’. These studies achieved the benchmarks that homeopathy would need to achieve – both in terms of study participants and adherence to the standards expected of randomised controlled trials – to raise an eyebrow in the evidence based arena. I hope they can serve as examples as to what homeopathy should be aiming for – rather than more and more lower quality studies. I also hope it is obvious that homeopathy has not yet approached the same vicinity as these studies and therefore, at present, it deserves the doubt directed toward it by the evidence-based medicine community.
Anyway, here’s my list – ordered by number of prescriptions in Australia …
1) Atorvastatin –. Randomised controlled trial on 3086 adults following acute coronary syndrome.
2) Rosuvastatin –n. Randomised controlled trial involving 17,802 healthy participants.
3) Peridopril –Randomised controlled trial with 13,655 participants with coronary artery disease.
4) Paracetamol / acetaminphen –Metaanalysis of 10 RCTs including 1712 patients.
5) Irbesartan –. Randomised controlled trial involving 1715 hypertensive patients with nephropathy due to type 2 diabetes
6) Candesartan –. Randomised controlled trial with 3023 patients with congestive cardiac failure.
7) Amlodipine –. 15,245 patients, aged 50 years or older with treated or untreated hypertension and high risk of cardiac events.
8) Rampiril –Randomised controlled trial on 9297 high-risk patients (55 years of age or older) with evidence of vascualar disease or diabetes and one other cardiovascular risk factor but no heart failure.
9) Esomeprazole –. Randomised controlled trial on 2425 patients with erosive oesophagitis.
10) Simvastatin –. Randomised controlled trial in 4444 patients with coronary heart disease.
I promise you that if homeopathy was to match the methodology and sample size of any of these studies you would begin to mount an armament to carry into the arena. Would that be a guarantee that your study is true? No. But it would certainly cause people to take your therapy seriously and spark further research into the question to clarify the issue. This this is the level of evidence you require to initiate a discussion in evidence based medicine. More and more under-powered studies with less than 100 participants isn’t going to cut it.
Forgive me! It seems my last comment struck several chords. I want to reply in both a micro- and macro- sense, so that we both have a sense that there is continued purpose.
In the micro-sense, I don’t believe we have any conflict about your studies or the nature of modern medicine:
To summarize our last two interactions, I had responded to a request for evidence. Your response was a magnificent manifesto on the idealized platform for evidence-based medicine. It was as if you’d asked me to jump, and I said how high? Your response was “to the moon and the stars and back again, and do it twenty times.” A man can be forgiven if he says: “you’ve got me. Why don’t you show me how it’s done?”
In response, you have given me a number of large single studies on medications. I thank you for your effort, and would point out that I was asking for you to jump to the stars as well: a perfect study, and that repeated twenty times. If you had done so, I honestly would have fundamentally altered the way I saw medicine and the world. But I hold the medicine you do in no less esteem because you did not. It was your standard I was responding to, not mine.
In the macro-sense, I’m not sure that we are working toward the same goal:
Which brings us to a larger point, why are we here on the comment boards in the first place? For me, homeopathy is the one area of my practice that I have the most difficulty with. I am not a “true believer” and began very much as a skeptic of the practice. So I am here to work out how and why I use homeopathy in my practice, using the comment boards as an opportunity to defend one position. As I’ve hopefully said before, my fallback position is that homeopathy may simply be the placebo effect.
For you, this seems to be more of a defense of evidence-based medicine as a whole. While I applaud the undertaking, as a philosopher you understand that you are simply taking one philosophical view among many. We might view you as a Hegelian rationalist amidst a sea of Hume’s relativists. How frustrating it must be to try and pin down some new age belly-button reader on the evidence? They will just shrug and say “that’s your reality, man.” But if we could expand the philosophical more broadly, I wonder if you have looked at the larger aspect of what is going on? It is not what the evidence shows, it’s how we believe the evidence and how we act on it. I think Kierkegaard, in opposing Hegel, said: “the crucial thing is to find a truth which is truth for me, to find the idea for which I am willing to live and die”
In the micro-sense, I agree that homeopathy could use more rigorous studies. I think there is a fundamental difference between: “is there any evidence?” and is there really good evidence I can hang my hat on?” So the argument cannot be: in the absence of mega-studies, just say no. It has to be: in the absence of complete clarity, is this subject worth more exploration, and –more importantly- is there sufficient evidence that homeopathy should continue to be allowed to exist?
Within that process, let me say that homeopathy has one thing, hands-down, that modern medicine does not. A classical homeopath will spend three hours listening to a patient’s story, not just her symptoms, but everything about her life. He will listen and ask questions that allow the patient to expand on her world-view, her fears, her thoughts, her dreams. The patient is heard, met where she is, her story is honored and she is seen as a whole, a complete entity rather than a toe or an elbow.
A classical homeopath will not then describe a remedy. He will go and think about a remedy for a week or more. Out of hundreds of remedies, each of which has been documented by other homeopaths to have cured a thousand or more patients, the homeopath will select a dozen. At a second interview, again perhaps several hours, the homeopath will ask more questions, listening quietly. Only at the end of an enormous amount of labor will the homeopath prescribe a single remedy to be taken.
Add to this that most classical homeopaths are true believers. They profoundly believe that the right remedy will cure the patient. What we have here at its most fundamental is the best of all possible placebo effects, an effect that should not be dismissed or distained. It is what they call in medical circles an excellent doctor-patient relationship, and it can confound or even reverse an evidence-based drug effect. (see Kaptchuk’s study on IBS, summarized in Wired: )
Against that background, a drug researcher might well say “many of the studies of homeopathy do not show significant effect over placebo.” In frustration, a group of M.D.s might do a trial of a homeopathic to see if it “really works at all.” But by beginning this way, they are doing surgery without sterilizing the area or prescribing blood pressure medication without checking the patient’s resting rate. Homeopathics are not prescribed equivalent to drugs in that x does not do y. In homeopathics, x or y or z may do y, depending on the conditions of the patient. To take a relatively simple example, natrum muriaticum, homeopathic sea salt, is useful in about 30% of migraine sufferers. An M.D. doing a study on migraines might do a study that shows natrum mur only worked in 30% of migraine sufferers and declare it no better than placebo. A homeopath will just shake his head, because the homeopath knows there are dozens of remedies for headache within homeopathy, and you never would prescribe natrum muriaticum willy-nilly without taking the whole case. The only equivalent within modern medicine would be a doctor, blind-folded, prescribing whatever his hand touched in the pharmacy and declaring it placebo if it did not improve the patient’s condition.
In a macro-sense, I don’t think providing you will a large scale study will resolve the issues you have with homeopathy and alternative medicine.
A homeopath might ask each of us: why do you care so much what random people say on the internet? To which I would reply, I’m looking for a dialogue, but I think my friend Ben is looking for the respect he deserves from leaving a lucrative, souless ad job and working his butt off to become a doctor. Ben is evidently brilliant, caring, and has taken the time to investigate alternative medicine. He found, as I did, that it was heavy on hare-brained ideas and horribly light on evidence. So he discarded it and went forward to become a disciple in the evidence-based medicine school. But in the process, he finds himself as the scapegoat in a global shift away from respect and honor for medicine and a renaissance in the dubious ala the 1890’s. My own path was to take what was useful and discard much of the rest. In my attempts to discard homeopathy, I came to a sticking point where the darn stuff seemed to work when nothing else did. Unlike Ben’s experience, I find myself sought out at parties, and privy to some of the most amazing stories as patients tell me their strangest encounters with illness and the unknown. I also get to hear about all the shortfalls of modern medicine, and spend much of my time defending doctors while trying to convince my patients not to do things like drink miracle soap.
So in the macro-sense, Ben, would a large scale study of homeopathy with robust results really please you? It’s just one more reason to be angry at the changing world. Neither one of us is going to change the fact that we’re in a very woo-woo time right now. Truly, alternative practitioners are not your enemy in this. Most of us are not, I hope, as out there as our patients, who genuinely do very, very silly things to themselves. I would say that 80% of the time I’m trying to rein in my patients from doing something like “detoxing” using something on the internet that looks like dustbin leavings packaged in a capsule. You can be angry at people who’ve taken the time to educate themselves, but my world is full of ad salesmen who got themselves a correspondence degree and are making millions marketing dead bacteria as shelf-stable probiotics. My answer has to be that some education is hopefully better than none.
In a micro-sense, there are difficulties with the reviews you presented:
Now, let us look at the two meta-analyses you presented from the Cochrane Review. By simply placing “like cures like” in quotations, the authors of the asthma analysis demonstrate that they have no idea what homeopathy is and how it could possibly work. They then rightly say that given the heterogeneity of existing studies and study quality, they have no idea how to even judge the effect. But they conclude both that there is no strong evidence and that they have no way of judging the effect of the doctor-patient relationship within the Cochrane context. What we have here is not a mandate against homeopathy for asthma, it is a mandate for the education of reviewers.
The second Cochrane analysis is on debridement for osteoarthritis. By concluding that debridement is not effective, your assumption was that the debridement study changed the way medicine is practiced. We should see a drop-off since this publication in 2007 to almost no procedures done today. My response is no, that’s not how medicine works at all, and they just opened a new knee surgery specialty down the road specializing in debridements. What happens instead is that the doctors performing five hundred thousand surgeries a year do not put down their instruments and walk away. They do not “believe” the study. Other researchers do other studies, but until all the existing doctors retire, you won’t see a drop-off in knee surgeries until the existing crop of surgeons retire and new surgeons, with new surgeries to pay their bills, take their place.
In a macro-sense, I have no difficulty with your evidence-based medicine. But I need to point out the ideals of evidence-based medicine fall far short of the realities of medical practice. I would have to say that from where I’m sitting, modern medicine bears much more of a resemblance to the wandering of alternative medicine than it does to your Colosseum.
How can I be so cynical? Let’s look at the recent massive study on mammograms. No effect, and a 20% increase in false-positives. You’d think the result would be a drop-off in mammograms, but instead we see massive backlash and many smaller, shorter studies run out to try and offset the larger study. Just as I still see older patients getting PSA testing when it clearly does not benefit them.
I quote from the Dartmouth Atlas of Healthcare, which is a very cynical read for an evidence-based practitioner:
“For twenty years the Dartmouth Atlas has demonstrated wide regional variation in health services utilization that cannot be explained by patient needs or preferences alone. This variation largely reflects regional differences in the use of medical services of uncertain benefit, rather than differences in the use of effective care. As outlined in previous Atlas publications, local health care capacity and practice style appear to drive much of this observed variation.”
I think what the Dartmouth Atlas can make eminently clear is that evidence-based medicine in practice is not the rule. It may not even be that common.
In a micro-sense, I give you a perfect score on all your medicines. But in the macro-sense of clinical practice, it’s a different story.
Now, let’s examine your list of medications. I will give you a perfect score. All of your studies are perfect, and all of your medications are perfect. No complaints whatsoever.
Let us apply these medications to an individual patient. What is the effect of two of these medications to Joe, our theoretical sixty-year old couch potato? Do we have studies on the effect of both medications in his age group? No? No matter.
But Joe doesn’t take just two of these medications. He takes three, four, five, often in odd sequences as he remembers them. What are the effects now? Do the benefits of five medications for Joe outweigh the risks he may experience from interactions and side effects? We have no information, and at five medications we’ve reached polypharmacy, a place that Cochrane itself dares not tread. If Joe was the exception, it wouldn’t be a problem. But we’ve both seen people on far more medication. I think thirty-two is my patients’ record for medications taken at any one time, what’s your record?
If polypharmacy was limited to patients with worse illness, it could perhaps be forgivable, but it is a direct correlation to how much money a person has, not how ill they are. A richer patient with better insurance will get more drugs, more surgeries and more medical care. As we export this model of drug care overseas, you can see the effects of urban vs. rural care.
In a micro-sense, you are absolutely right about pursuing conventional medical options for the developing world. But in a macro-sense, the healthcare structure is completely unsustainable without outside grants and support. Unless you want to argue that we should support not only our own health care system but also the healthcare systems of the rest of the world, we have to find other options.
I do love the rotavirus studies as a trumpet call for modern medicine in developing nations. You picked the one area where massive support from developed nations is making an impact in countries that simply cannot afford the rest of our medical infrastructure. It’s the exception that proves the rule: in developed countries where the infrastructure exists, rotavirus is an annoyance, not a killer. But look at the funding sources for rotavirus development. The only way rotavirus can be delivered to those countries is by marketing it heavily to suburban housewives whose children are not at risk from dying of rotavirus and may experience serious side effects from intussusception as a result (1 in 20,000 now, the older version was 1 in 10,000) . Using this example is a demonstration that we simply cannot export medical infrastructure, ala the Colosseum, to places where people can’t feed themselves. Any cost is too much cost, and nothing in your armamentarium is going to change that.
So, that leaves me with a more appropriate task than the twenty perfect studies : one high-quality study of patients improved by homeopathy. But I think that’s a bit silly. At this point I’m not arguing for homeopathy to be used by your wealthy, happy patients. I’m arguing that people with no recourse could benefit from homeopathy. I don’t care if it’s simply effective, I care if it’s an idea worth fighting for, worth promoting in parts of the world where modern medicine cannot reach. You wouldn’t do it instead of vaccination, you would do it alongside, when the vaccine workers have moved on. As we both know, the causes of childhood diarrhea are myriad, and resolving rotavirus did no magically resolve deaths from childhood diarrhea in developing countries.
Let’s look at Bangladesh, where the poorest of the poor live in unbelievable circumstances and are forced to drink from wells that contain high levels of arsenic. Of course, we’d like them to have pure water. Of course, moving either of us to the squatter’s village with a full contingent of medicines would be better, but let us ask ourselves if homeopathics will make a difference to the desperately poor in Bangladesh.
First, you would do all in your power conventionally to alleviate the problem ( the “rotavirus approach”):
Failing to use conventional means, you would do a mouse study to see if there was any effect from homeopathics:
Then you would follow-up with a human study:
Now, you have the complete study, and you can tear it to shreds. But to do so, you have nothing to offer. Given the real, on the ground example, how can you claim that homeopathics serve no purpose? And realize that I’m talking about most of the world, not the coddled conclaves that you and I tend now.
I’ll comment on the meta-analysis but I’ll have to leave it there.
The problem as I see it is this is yet another analysis on the same data that has been looked at previously. The studies included in this meta-analysis are mostly 10 – 20 years old – with the most recent published in 2010.
These studies have been looked at previously. Before this meta-analysis even began we already KNEWthat it wouldn’t be able to tell us anything conclusively.
I think the meta-analysis spells out the problems pretty well …
“Twenty-nine of the 32 trials had unclear or high risk of bias in important domains of assessment … High and unclear risk of bias featured almost equally in our 22-trial analysis; thus, the overall quality of analysed evidence was low or unclear, necessitating caution in interpreting the findings“.
This is an admission of what we already know – unfortunately, the studies done on homeopathy to date haven’t been of high quality. We could run meta-analyses on the available data for eternity and it will never provide us with a conclusive answer.
This meta-analysis contains the truth in its conclusion – “New RCT research of high quality on individualised homeopathy is required to enhance the totality of reliable evidence and thus enable clearer interpretation and a more informed scientific debate“.
This fact is not a criticism of homeopathy or an attack against alternative medicine. The simple fact is that a high quality study has yet to be run on homeopathy – and there is no way to get to the truth by continuing to re-examine poor studies. Like Ted Kaptchuk, the alternative medicine community should seize upon this as constructive feedback and begin running quality trials that are not open to similar criticism and therefore CAN be used as evidence for efficacy above placebo.
While I do agree that more studies of high quality need to be run, we’re still at a place where you’re saying that no high quality studies have ever been run. Surely some set of researchers, somewhere, in the last fifty years, has done a high-quality study on homeopathy even if its results were an absolute “this is placebo.”
It’s one thing to say that better studies need to be done, it’s another to say that no studies of sufficient quality have ever been done. If that is the case, then no one, on either side of the discussion of homeopathy can conclude anything about it. I think is has to be untrue, based on Hahn’s discussion of the analyses: “There was a trend toward smaller benefit from studies of the highest quality, but the 10 trials with the highest Jadad score still showed homeopathy had a statistically significant effect.”Here’s the whole article: If you look at his discussion of Linde: “For all 6 Jadad score levels, homeopathy was still statistically superior compared to placebo.”
Hahn goes on to say that: “To conclude that homeopathy lacks clinical effect, more than 90% of the available clinical trials had to be disregarded.” To me that speaks of a determined effort to conclude the negative, not an objective assessment of admittedly flawed data.
In other words, I think a great many people in the research world are not objective about homeopathy. I wasn’t. It’s something we get to ridicule in chemistry class. It’s like the Yeti or the Loch Ness monster. Fun to think about, but nobody takes it seriously. If Bigfoot suddenly walks out of the woods one day, people aren’t going to say, “oh look, it’s Bigfoot. I knew there had to be something out there.” They’re going to say he’s a lost hippie or some escaped ape or anything but Bigfoot. But sometimes, just sometimes, the rumors are true and there really is something out there. Here’s a story about our local celebrity, the North Pond Hermit, a man who lived 27 years in Maine in a makeshift tent. The locals made up stories about him until one day they finally caught him.
I think homeopathy is the North Pond Hermit of medicine. It’s strange, it defines common sense from chemistry class, but when you get down to it, it’s real. I don’t think we’re at the stage where we say it doesn’t work at all, I think we’re at the stage where we need to start asking how it works when it does and what that means for our patients.
I don’t expect a reply, and I think you’re right about the overall tone of the quality, I just think there are a couple of reasonable RCTs in the bunch. Some of them are negative, and some of them are positive. But enough of them are positive we really need to move forward with more trials.