Your regular doc was ignoring your symptoms or told you to take a couple of aspirin and come back if the problem persists, so you headed to a practitioner who treats you holistically, who understands, who knows what you specifically need for your specific condition. You swallowed their talk of spectral quantum chi rebalancing with latent fascia therapy or whatever and you got better. Yay!
Spinning a yarn does not evidence make
Well, there are numerous reasons why you shouldn’t assume it was the spectral fascia therapy that did the trick.
The primary weakness of anecdotes as evidence is that they are not controlled. This means hidden variables might be at play, but you will never know for sure.
Science-Based Medicine offers some specific factors that make it impossible for practitioners and patients to know for sure that taking such and such a “remedy” has treated their “condition” real or imagined, whether or not it did nothing at all or whether the “placebo” effect is at play.
Regression to the mean: Many diseases/disorders fluctuate in symptoms, if you get a CAM remedy when you have severe symptoms and they wane is that just random fluctuation or the treatment, how would you know?
Many illnesses are self-limiting, which means intervening with CAM, or anything else, is generally pointless but will give you a false sense that the treatment helped as you get better.
Multiple treatments: Often people will try multiple treatments for a disease or ailment making it impossible to tell which treatment had a beneficial effect, if any.
Dead men tell no tales (the problem of reporting bias): Survivor groups for potentially lethal diseases do not have dead members. If you die of that disease, regardless of treatment, you won’t be around to tell people how it all went. But, those who do survive (regardless of lethal disease or not) are more likely to tell others about how a particular treatment helped them, those it fails will rarely brag about their experience.
Vague outcome measures: Good clinical trials use objective outcome measures — those that are binary (like death or survival), quantitative (like a blood level), or are based upon a specific physical finding. Anecdotes do not make good outcome measures because they require that judgements be made by patient or practitioner.
The Placebo Effect: The placebo effect is actually a host of many effects that give the appearance of a response to an inactive treatment. Without a control experiment (where a patient with the same symptoms are given a dummy treatment that looks real) to test for this effect, it is impossible to know whether the original treatment is snake oil or panacea. The above applies to all forms of treatment whether you think of them as alternative or conventional, but it’s only in conventional medicine where the principles are applied snakeoil sales reps tend to ignore evidence and rely on anecdote.
via The Role of Anecdotes in Science-Based Medicine « Science-Based Medicine.
There is an argument that “if it works, why does it matter?” Well, that’s fine, if you feel it’s worth paying for sugar pills and water and the placebo effect kicks in and you do feel better for it. But, practitioners of non-evidence-based medicine may convince you that their approach is best all the while the underlying cause of a condition may be worsening. There’s only so much the placebo can do. Many alt practitioners also tout protective treatments for malaria, HIV, cancer and other potentially lethal conditions. Again, it’s very unlikely a placebo will do anything more than lull someone into a false sense of security often leading to their early demise.
Now, I have an anecdote of my own.
A close relative with lower leg pain and numbness was told by her regular doctor to take some painkillers and see how things progressed. Opted to see a chiropractor who would do all kinds of manipulations over the course of many months at $100 per half hour session. Before first treatment, pain and numbness got so bad, saw doctor again, was rushed to hospital. Turned out it was the potentially fatal, often crippling nerve-destroying immunological disorder Guillain—Barré syndrome (GBS), which can only be treated with intravenous immunoglobulin to destroy the damaging white blood cells attacking the nerves. No amount of hand-waving woo, sugar pills, chi, or quantum therapy would have saved my relative from otherwise inevitable paralysis, respiratory collapse or worse…