Why your personal CAM story is not evidence

Your regular doc was ignoring your symptoms or told you to take a couple of aspirin and come back if the problem persists, so you headed to a practitioner who treats you holistically, who understands, who knows what you specifically need for your specific condition. You swallowed their talk of spectral quantum chi rebalancing with latent fascia therapy or whatever and you got better. Yay!

Spinning a Yarn - Photo by David Bradley
Spinning a yarn does not evidence make

Well, there are numerous reasons why you shouldn’t assume it was the spectral fascia therapy that did the trick.

The primary weakness of anecdotes as evidence is that they are not controlled. This means hidden variables might be at play, but you will never know for sure.

Science-Based Medicine offers some specific factors that make it impossible for practitioners and patients to know for sure that taking such and such a “remedy” has treated their “condition” real or imagined, whether or not it did nothing at all or whether the “placebo” effect is at play.

Regression to the mean: Many diseases/disorders fluctuate in symptoms, if you get a CAM remedy when you have severe symptoms and they wane is that just random fluctuation or the treatment, how would you know?

Many illnesses are self-limiting, which means intervening with CAM, or anything else, is generally pointless but will give you a false sense that the treatment helped as you get better.

Multiple treatments: Often people will try multiple treatments for a disease or ailment making it impossible to tell which treatment had a beneficial effect, if any.

Dead men tell no tales (the problem of reporting bias): Survivor groups for potentially lethal diseases do not have dead members. If you die of that disease, regardless of treatment, you won’t be around to tell people how it all went. But, those who do survive (regardless of lethal disease or not) are more likely to tell others about how a particular treatment helped them, those it fails will rarely brag about their experience.

Vague outcome measures: Good clinical trials use objective outcome measures – those that are binary (like death or survival), quantitative (like a blood level), or are based upon a specific physical finding. Anecdotes do not make good outcome measures because they require that judgements be made by patient or practitioner.

The Placebo Effect: The placebo effect is actually a host of many effects that give the appearance of a response to an inactive treatment. Without a control experiment (where a patient with the same symptoms are given a dummy treatment that looks real) to test for this effect, it is impossible to know whether the original treatment is snake oil or panacea. The above applies to all forms of treatment whether you think of them as alternative or conventional, but it’s only in conventional medicine where the principles are applied snakeoil sales reps tend to ignore evidence and rely on anecdote.

via The Role of Anecdotes in Science-Based Medicine « Science-Based Medicine.

There is an argument that “if it works, why does it matter?” Well, that’s fine, if you feel it’s worth paying for sugar pills and water and the placebo effect kicks in and you do feel better for it. But, practitioners of non-evidence-based medicine may convince you that their approach is best all the while the underlying cause of a condition may be worsening. There’s only so much the placebo can do. Many alt practitioners also tout protective treatments for malaria, HIV, cancer and other potentially lethal conditions. Again, it’s very unlikely a placebo will do anything more than lull someone into a false sense of security often leading to their early demise.

Now, I have an anecdote of my own.

A close relative with lower leg pain and numbness was told by her regular doctor to take some painkillers and see how things progressed. Opted to see a chiropractor who would do all kinds of manipulations over the course of many months at $100 per half hour session. Before first treatment, pain and numbness got so bad, saw doctor again, was rushed to hospital. Turned out it was the potentially fatal, often crippling nerve-destroying immunological disorder Guillain–Barré syndrome (GBS), which can only be treated with intravenous immunoglobulin to destroy the damaging white blood cells attacking the nerves. No amount of hand-waving woo, sugar pills, chi, or quantum therapy would have saved my relative from otherwise inevitable paralysis, respiratory collapse or worse…

5 thoughts on “Why your personal CAM story is not evidence”

  1. Mike, your comment got spam trapped, up-moderated now. Sorry ’bout that. As to “insurance” I’m a Brit, I don’t have private health insurance, we have a National Health Service, many lament its problems, but it beats the US system as far as I can see. Every. Single. Time. I think the US individualist, anti-government-interventionist, capitalist approach to healthcare probably doesn’t help the majority over there and is why so many people to turn to ineffective CAM they can essentially buy in Walmart rather than getting actual medical help…

  2. A related matter, not mentioned in this article, is the changing roll of the Primary Care Physician (PCP).

    We are often given Tylenol, antibiotics, or similar, in the odds that the cause is not more serious- and the patient’s inability to make time to willingly deal with a health issue in a work world world where many of us are a smart phone call away at any hour.

    If our chest hurts, does the PCP try to diagnose it, or does he suggest we see a cardiologist– and if so, is he/she willing to “walk us over” to a cardiologist he knows and trusts, assuming he can get us in for a new patient appointment?

    Has anyone recently attempted to find a new PCP that our insurance will accept? Has anyone received a good PCP referral from a specialist? And do we really need a gate-keeper to say “yes, you have chest pain….. it’s a cardiologist (as opposed to what, a podiatrist) that we need to see?

    When we see a specialist, sometimes they don’t want to hear about our treatment side-effects and symptoms as long as our numbers and pictures look ok. Then we are referred back to our PCP, who defers to the specialist on what to do next. Then on the other hand, I had my sinus infection prone friend who went to his PCP complaining of recent headaches along with flu-like symptoms. The doctor wanted to scan his brain— liability anyone?!

    The “system” is competitive, lacking communication, disorganized— doctors just trying to get through the day like the rest of us, and even trying to help us to do the same. Much work to be done.

    Was the patient in the article let down, or should his instincts have told him to be more persistant? In retrospect, who really knows. As for CAM, I’ve experienced situations where it has helped a great deal, and also harmed a great deal. The idea is we need to do our own research, discuss with our practitioners, and IMHO, use CAM, if at all, to complement rather than to replace what is scientifically proven. Even that is in the early stages.

  3. Actually, of course, she wasn’t ignored at all, the first doctor gave her a vague diagnosis and suggested painkillers. Bad wording on my part. Edited.

  4. Yes, but it was another doctor who got her into hospital to be fixed, the quack did nothing useful, just handed her a bill and told her to book a series of appointments…

  5. in your own anecdote surely the really significant words are “..was ignored by her regular doctor”

Comments are closed.

If you learned something from Sciencebase, enjoyed a song, snap, or the science, please consider leaving a tip to cover costs. The site no longer runs Google ads or similar systems, so your visit is untainted.