EBM versus ABM: Anecdotes Do in Fact “Add Up”

Jeff Clark, ND

The practice of medicine has long been described as an art informed by science. For millennia, we have cultivated and then relied upon the expertise of individuals who have specialized in health and healing.1 Doctors invest themselves in understanding human biology and medical treatments from knowledge passed down and viewed through multiple academic subjects. Doctors gain experience through personal observation of patients and in applying, adjusting, and modifying treatments according to individual response and need.

The Problem with Evidence-Based Medicine

The contemporary doctrine of evidence-based medicine (EBM) demands that information derived from specific types of population-based statistical evidence should always be used to guide medical treatments. According to this view, only “randomized, controlled, double-blind, statistically-significant evidence” (simplified to randomized controlled trials or RCTs) can lead to patient safety and efficacy in medical practice. Meta-analyses and systematic reviews of RCTs are also admissible within the EBM ideal. According to the EBM doctrine, all other evidence is deemed weak and overly influenced by observer bias and patient malleability.2,3

Anecdotal-based medicine (ABM) is sometimes held up as the antithesis of EBM. With just 1 patient (n = 1), all we can know is whether or not that person achieved an outcome while under treatment. What we cannot know is whether the observed outcome was due to the prescribed treatment, random chance, placebo effect, or some other unidentified factor. More importantly, with just this 1 experience, we have only the slimmest basis upon which to judge the treatment’s applicability to another patient in similar need.4

Modern pharmaceutical data presented to doctors and patients include treatment indications, recommended dosing schedules, and lists of possible side effects, which are categorized by severity, from major to minor. The fact that a drug has been FDA-approved misleads the naive observer to expect that the drug will always work and that rare side effects are the only confounding concerns. Experienced doctors know better.5,6

If it were not for the great variability between individuals, medicine might as well be a science, not an art.

(William Osler, 1892)

Every person who is given a treatment will respond individually, potentially landing anywhere on the distribution curve that describes the population level response. Ultimately, the only reliable means a doctor or patient has for learning a patient’s individual response is to try a treatment and observe what happens (Table 1).7,8

Table 1. Seven Possible Patient Treatment Outcomes

Patient receives intended benefit without any noticed side effects
Patient receives intended benefit along with minor side effects that are tolerable
Patient receives intended benefit along with minor side effects that are not tolerated
Patient receives intended benefit along with major side effects
Patient receives no intended benefit along with major side effects
Patient receives no intended benefit along with minor side effects
Patient receives no intended benefit without any side effects noted

In clinical application of a treatment, the observant doctor classifies patient responses falling into categories 3-7 as “treatment failed.” The category of “treatment succeeded” includes the responses that fall into categories 1 and 2. With effective treatments, the majority of patient results are “treatment succeeded.” For the attentive doctor in clinical practice, treatments that have been experienced repeatedly with multiple individual patients eventually fall into just 2 categories: a) “Works most of the time” and will be used again; or b) “Doesn’t seem to work” and unlikely to be used again.

Bernoulli Distributions

The clinical application and treatment success or failure in the doctor’s experience follows a statistical model known as a “Bernoulli distribution.” Daniel Bernoulli was an 18th century Swiss mathematician and physicist who first formally described this statistical technique and from whom it derives its name.9 Bernoulli distributions are created by accumulating the outcomes of independent trials of n=1. When formally tabulated, anecdotes of success and failure accumulate into a statistically relevant binomial distribution (Table 2).10

Table 2. Components of a Bernoulli Distribution

n = 1: independent trial; no past trial affects outcome of a future trial
N: number of trials conducted
p: accumulated successes / N
q: accumulated failures / N
1 = p+q: summation of probabilities
p = 1-q: success expressed in terms of failures
q = 1-p: failure expressed in terms of success

For attentive and pragmatic doctors beholden to patient well-being and outcomes, repeated use of a treatment with future patients requires an ongoing perception of p>q. The q value must also not contain examples of significant patient harm.

Bernoulli trials and resulting experience-based perceptions of success and failure describe mathematically the trial-and-error method used to develop treatments throughout the history of medicine – before our very recent era of the EBM doctrine.11 Economics, engineering, and manufacturing continue to embrace and consciously use this Bernoulli approach to acquire knowledge.12

EBM vs ABM

Are the probability data collected through Bernoulli trials inferior to the data collected by more sophisticated statistical methods used in RCTs? An either/or comparison doesn’t seem appropriate. High-quality and ethically-conducted RCTs characterize expected outcome probabilities for a sample population studied under controlled conditions. This is valuable information to possess. EBM based on an RCT is an appropriate requirement for pharmaceutical manufacturers to demonstrate to the FDA a potential population-level benefit of novel, patentable, and theoretically safe substances that they wish to market as specific medical treatments while demanding a high price.13 In contrast, Bernoulli experience resulting in ABM tells us how medical treatments translate into outcomes in an uncontrolled patient population when used by physicians providing individualized care.

Anecdotal Bernoulli-type data are formally gathered and tabulated for FDA-approved medications. Such activity is called “post-marketing surveillance.” The Bernoulli information gathered post-market focuses primarily on treatment failures resulting in significant patient harm. The FDA uses this accumulated anecdotal data to determine whether an EBM-based, FDA-approved medication should be allowed to stay on the market. At the end of the day, ABM trumps EBM within the FDA when it comes to patient safety.5,14

EBM, Individuality, and Economics

It is a crisis of faith for the practicing doctor when the population-level EBM found in peer-reviewed pharmaceutical literature suggests a treatment should work, but the ABM evidence from the doctor’s own clinical experience suggests the treatment is unsafe or does not work.5,6,15-18 It is a devotion of unyielding faith when doctors persist with treatments that in their own hands hurt patients and often do not work.19 And it is a challenge to doctrine when information other than EBM guides medical practice, enabling the doctor to utilize treatments involving legacy, off-label, and compounded pharmaceuticals, herbal medicines, and nutritional supplements.20-23

Many experienced physicians believe that constraining individualized medicine to only those treatments having population-level RCT evidence is unreasonable and impractical.24 In conventional medicine, not all treatments have supporting evidence meeting the doctrinal definition of EBM. Many legacy medications persist with Bernoulli experienced ABM as the only plausible basis for their continued use.25,26

EBM doctrine falls apart when treating the individual patient. For a given individual, all of the uncertainty confounders – including random, unidentifiable and uncontrollable variables, patient compliance, the combined effects of polypharmacy, epigenetic predispositions, age, patient health status, environmental factors, placebo effects, and more – return to affect the individual patient’s outcome.7,8,23,27

The ubiquitous treatment of hypertension is a poignant example of the “trial and error” nature of individualized medical practice despite substantial EBM and FDA approval supporting the use of various treatments. The published treatment algorithms boil down to this: try medication A; not working? Adjust dosing; still not working? Change to medication C or D, and then adjust dosing; still not working? Move to progressive polypharmacy with A + C or A + D, or A + C + D, or A + B + C + D, adjusting doses until the blood pressure targets are finally met… if possible.28

Insisting on RCTs resulting in EBM for all substances used in medicine goes beyond serving the FDA’s historic mission of protecting the public from dangerous, adulterated, and mislabeled food and drugs.29 The demand for compliance with EBM doctrine in all matters of clinical thinking creates an economic barrier that protects FDA approved drugs.30 Just how much fealty the FDA should have for defending the economics of the pharmaceutical industry is an important public policy question. There is a historical “revolving door” between FDA, pharmaceutical company experts, and policy makers.31,32 The FDA is currently being criticized for appearing to protect its pharmaceutical company clients from the consequences of research misconduct uncovered by the FDA’s own inspection processes.33

Allowing EBM doctrine and the FDA drug-approval process to narrowly define what may be viewed as legitimate medicine excludes most of the biologically relevant materials found on our planet. Substances occurring in nature and used traditionally for millenia as herbal and indigenous medicine do not generally possess the human-produced novelty that is essential for acquiring a patent.34 Patent protection is what creates the financial incentive to produce extremely expensive EBM to achieve FDA drug approval and to sell at a high price.35 Nutrients that occur in foods, isolated and concentrated to provide mass action to specific biochemical pathways, are the products of scientific discovery. Food supplements are significantly less expensive than FDA-approved patent drugs and are also not generally patentable.34 Supplemented nutrients do not generally present the same safety concerns as novel synthetic compounds.36-38

FDA-approved pharmaceuticals, legacy, off-label and compounded drugs, herbal medicines and concentrated nutrients all become one and the same thing at the door of medical practice. Each is a material that has 1 or more reasons to suggest to an aware physician that it might help a patient with a particular problem or condition. The use of any of these in a medical treatment becomes a Bernoulli trial of n=1. Each independent trial will result in an outcome of either success or failure. No amount of preceding population-level EBM can guarantee a safe and effective outcome for any individual patient, nor for every patient treated.

Summary

Where does this leave us? We return to what human societies have long known: the practice of medicine is an art informed by science. As a society, we correctly invest heavily in the training, skills, and experience of the physician to provide individualized care of the random, uncontrolled patient.39

We empower doctors to make treatment decisions by applying the wisdom that is created from their training, knowledge, skill, and first-hand experiences. At the same time, we demand that doctors not be reckless, that they “first do no harm.” Concomitantly, we should always expect doctors to take responsibility for the specific treatments they have prescribed to specific patients. This responsibility to the patient does not vary, whether that treatment has population-level EBM and FDA approval guiding its use or not.

The actions of doctors are governed by regulatory and public policy frameworks which require sustained licensure, and which are restrained by licensing boards, personal risk of financial loss through lawsuits, and even incarceration.40-43 The public interest is not served when we allow FDA approval and standards of care based on population-level statistics to absolve or indemnify prescriber responsibility in medical practice. Doctors should always be monitoring the individual patient they have under treatment, modifying those treatments that are not working and/or causing harm.

Naturopathic medical practice routinely utilizes FDA-approved, legacy, off-label, and compounded pharmaceuticals, as well as herbal medicines, concentrated nutrients, and dietary strategies. Indeed, the naturopathic materia medica potentially includes any biologically relevant material found on planet Earth that can be used successfully in the beneficial treatment of individual patients.

Evidence of exceptional rates of patient harm from treatments having only a legacy of Bernoulli-experienced ABM to guide the hands of licensed naturopathic physicians does not exist. To the contrary, NDs providing primary care pay one-half to one-third for the same malpractice insurance as MD internists and MD family practice physicians.44

In patient-centered medicine, the most important data point is the outcome of the uncontrolled patient receiving individualized care from an attentive physician.

 

jcbiopicsmallJeff Clark, ND, is in private practice and a co-owner of True Health Medicine, PC, in Tualatin, OR. He is a 2007 graduate of the NCNM in Portland, OR. Dr Clark had a previous career in Oregon high-tech engineering, and is a named inventor on 5 US patents. He is a 1983 graduate of Montana State University in Bozeman, MT, with a BS in Computer Science and a minor concentration in Industrial Engineering and Management. Dr Clark is currently a board director for the Oregon Association of Naturopathic Physicians, and chairs the OANP’s legislative committee. Dr Clark currently serves as the first ND on the Oregon Health Policy Board’s: Health Care Workforce Committee.

 

References:

  1. The History Learning Site. A History of Medicine. Available at: http://tinyurl.com/a6owo5e/. Accessed February 22, 2015.
  2. Kennedy HL. The importance of randomized clinical trials and evidence-based medicine: a clinician’s perspective. Clin Cardiol. 1999;22(1):6-12.
  3. American Family Physician. EBM Glossary. AAFP Web site. http://tinyurl.com/q36l9e7/. Accessed February 22, 2015.
  4. Novella S. The Role of Anecdotes in Science-Based Medicine. January 30, 2008. Science-Based Medicine Web site. http://tinyurl.com/q2ok8fn/. Accessed February 22, 2015.
  5. 35 FDA-approved Prescription Drugs Later Pulled From the Market. Last updated January 30, 2014. ProCon.org Web site. http://tinyurl.com/nwryde8/. Accessed February 22, 2015.
  6. Aleccia J. Drugs that don’t work: a tough pill to swallow. Updated April 4, 2008. NBC News Web site. http://tinyurl.com/nghc32d/. Accessed February 22, 2015.
  7. Varadhan R, Seeger JD. Estimation and Reporting of Heterogeneity of Treatment Effects. Agency for Healthcare Research and Quality Web site. http://tinyurl.com/pa8w3jl/. Accessed February 22, 2015.
  8. Spigel DR. The value of observational cohort studies for cancer drugs. Biotechnol Healthc. 2010;7(2):18-24.
  9. Wikipedia. Daniel Bernoulli. Available at: http://tinyurl.com/6hqeu/. Accessed February 23, 2015.
  10. Introduction (Bernoulli trials process): Basic Theory. The University of Alabama in Huntsville Web site. http://tinyurl.com/mopyook/. Accessed February 23, 2015.
  11. Claridge JA, Fabian TC. History and development of evidence-based medicine. World J Surg. 2005;29(5):547-553.
  12. Ayyub BM. Risk Analysis in Engineering and Economics. 2nd ed. Boca Raton, FL: Chapman and Hall/CRC; 2014. Page available at: http://tinyurl.com/lxqxpa4/. Accessed February 23, 2015.
  13. Chapter 3. RCTS and Health Policy. In: The Impact of Randomized Clinical Trials on Health Policy and Medical Practice: Background Paper. Washington, DC: U.S. Congress, Office of Technology Assessment, OTABP-H-22, August 1983. Princeton University Web site. http://tinyurl.com/lv253cc/. Accessed February 23, 2015.
  14. Vlahović-Palčevski V, Mentzer D. Postmarketing surveillance. Handb Exp Pharmacol. 2011;205:339-351.
  15. Hsu J. Dark Side of Medical Research: Widespread Bias and Omissions. June 24, 2010. LiveScience Web site. http://tinyurl.com/lmzsxwb/. Accessed February 22, 2015.
  16. Nassir Ghaemi S, Shirzadi AA, Filkowski M. Publication bias and the pharmaceutical industry: the case of lamotrigine in bipolar disorder. Medscape J Med. 2008;10(9):211.
  17. Russell D. The Astonishing Zyprexa Cover-Up. February 12, 2015. Mad in America Web site. http://tinyurl.com/ovrejhz/. Accessed February 22, 2015.
  18. Diamond DM, Ravnskov U. How statistical deception created the appearance that statins are safe and effective in primary and secondary prevention of cardiovascular disease. Expert Rev Clin Pharmacol. 2015;8(2):201-210.
  19. Brody JE. When Treating Cancer Is Not an Option. November 19, 2012. The New York Times Web site. http://tinyurl.com/ooyp4g6/. Accessed February 22, 2015.
  20. Wachtel-Galor S,  Benzie IF. Chapter 1: Herbal Medicine, An Introduction to Its History, Usage, Regulation, Current Trends, and Research Needs. In: Benzie IF, Wachtel-Galor S, eds. Herbal Medicine: Biomolecular and Clinical Aspects. 2nd ed. Boca Raton, FL: CRC Press; 2011. Available at: http://tinyurl.com/n4g5pna.
  21. American Botanical Council. The German Legal and Regulatory Environment and the History and Background of Commission E. Available at: http://tinyurl.com/jwhjfjt/. Accessed February 23, 2015.
  22. Mann JI. Evidence-based nutrition: Does it differ from evidence-based medicine? Ann Med. 2010;42(7):475-486.
  23. Walsh WJ. Biochemical Treatment, Medicines for the Next Century. Nutrition Digest. 1991;37(3). American Nutrition Association Web site. http://tinyurl.com/p5wrhue/. Accessed February 23, 2015.
  24. Goldman JJ, Shih TL. The Limitations of Evidence-Based Medicine—Applying Population-Based Recommendations to Individual Patients. AMA Journal of Ethics. 2011;13(1):26-30. Available at: http://tinyurl.com/jwnc77z/. Accessed February 23, 2015.
  25. Pre-1938 drugs not approved by FDA. CompoundingToday.com Web site. http://tinyurl.com/qaja8hs/. Accessed February 23, 2015.
  26. Derbis J, Evelyn B, McKeekin J. FDA aims to remove unapproved drugs from market Risk-based enforcement program focuses on removing potentially harmful products. August 2008. FDA Web site. http://tinyurl.com/m6ec76/. Accessed February 23, 2015.
  27. Bushardt RL, Massey EB, Simpson TW, et al. Polypharmacy: misleading, but manageable. Clin Interv Aging. 2008;3(2):383-389.
  28. Clinical Pharmacology Unit, Addenbrooke’s Hospital, University of Cambridge. Guidelines for the Management of Hypertension. October 2010. Available at: http://tinyurl.com/qzgkpkp/. Accessed February 23, 2015.
  29. U.S. Food and Drug Administration. Significant Dates in U.S. Food and Drug Law History. Updated December 19, 2014. FDA Web site. http://tinyurl.com/coh92fr/. Accessed February 23, 2015.
  30. Herper M. The Truly Staggering Cost Of Inventing New Drugs. February 10, 2012. Forbes Magazine Web site. http://tinyurl.com/8yqqd7r/. Accessed February 23, 2015.
  31. Silverman E. The FDA ‘Revolving Door’ Fosters Conflicts on Advisory Panels. Septemb er 15, 2014. Wall Street Journal Pharmalot Web site. http://tinyurl.com/knajhtx. Accessed March 3, 2015.
  32. Lewis H. How Big Pharma controls the FDA. October 19, 2013. The Refusers Web site. http://tinyurl.com/mdbc95e. Accessed March 3, 2015.
  33. Seife C. Research misconduct identified by the US Food and Drug Administration: out of sight, out of mind, out of the peer-reviewed literature. JAMA Intern Med. 2015;175(4):567-577.
  34. The New Patent Policy on Natural Products Is a Game Changer for Universities and Life Sciences Companies. September 16, 2014. Bradley Arant Boult Cummings LLP Web site. http://tinyurl.com/op3r87o/. Accessed February 13, 2015.
  35. Brekke KR, Straume OR. Pharmaceutical Patents: Incentives for Research and Development or Marketing? South Econ J. 2009;76(2):351-374. Available at: http://tinyurl.com/kx3g555/. Accessed February 13, 2015.
  36. Walsh WJ.. Biochemical Treatment: Medicines for the Next Century. Nutrition Digest. Summer 1991;37(2).2. American Nutrition Association Web site. http://tinyurl.com/p5wrhue/. Accessed February 23, 2015.
  37. Walsh WJ. Biochemical Individuality and Nutrition. Walsh Research Institute Web site. http://tinyurl.com/l7aljjl/. Accessed February 23, 2015.
  38. ANH Exclusive! Natural health products ultra-safe and drugs as dangerous as war. Alliance for Natural Health Web site. http://tinyurl.com/cbkm259. Accessed February 23, 2015.
  39. Panda SC. Medicine: Science or Art? Mens Sana Monogr. 2006;4(1):127-138. Available at: http://tinyurl.com/lwwn9bm/. Accessed February 23, 2015.
  40. Chaudhry HJ, Rhyne J, Cain FE, et al. Maintenance of Licensure: Protecting the Public, Promoting Quality Health Care. Journal of Medical Regulation. 2010;96(2). Federation of State Medical Boards Web site. http://mss.fsmb.org/FSMBJournal/V96/V96N2_Chaudhry.pdf.
  41. Carlson D, Thompson JN. The Role of State Medical Boards. AMA Journal of Ethics. 2005;7(4). Available at: http://tinyurl.com/kfvtrcg/. Accessed February 23, 2015.
  42. Roche B. Tort Law and Health Care. March 20, 20156. Brien Roche Law Web site. http://tinyurl.com/n8mfnk7/. Accessed February 23, 2015.
  43. Monico E, Kulkarni R, Calise A, Calabro J. The Criminal Prosecution of Medical Negligence. The Internet Journal of Law, Healthcare and Ethics. 2006;5(1). Internet Science Publications Web site. https://ispub.com/IJLHE/5/1/5237/. Accessed February 23, 2015.
  44. Author’s private conversation with malpractice insurance executive; December 2014.

Related Posts

Upcoming Events

Scroll to Top