The Ups and Downs of Evidence Based Medicine


In medicine the term evidence-based causes more arguments than you might expect and that’s quite apart from the recent political controversy over why certain words were avoided in CDC budget documents. The arguments don’t divide along predictable partisan lines either. Evidence-based medicine is the topic of this week’s Healthcare Triage. The mission of evidence-based medicine is surprisingly recent. Before its arrival much of medicine was based on clinical experience. Doctors tried to figure out what worked by trial and error and they passed their knowledge along to those who trained under them. Many were first introduced to evidence-based medicine through David Sackett’s handbook first published in 1997. The book taught me how to use test characteristics like sensitivity and specificity to interpret medical tests. It taught me how to understand absolute risk versus relative risk. It taught me the proper way to use statistics and diagnosis and treatment and in weighing benefits and harms. And hopefully I’ve used to teach all of you something about all of that over the years. It also firmly established in my mind the importance of randomized controlled trials and the great potential for meta-analysis which group individual trials for greater impact. This influence is apparent in what we do here at Healthcare Triage. But evidence-based medicine is often described quite differently. Everyone’s a bit right here, and everyone’s a bit wrong. This battle is not new; it’s been going on for some time. It’s the old guard versus the new. It’s the patient versus the system. It’s freedom versus rationing It’s even the individual physician versus the proclamations of the specialized elite. Further, too often we treat all evidence as if it’s equivalent. I’ve lost track of the number of times I’ve been told – even by many of you – that “research” proves I’m wrong. All research is not the same. A hierarchy of quality exists, and we have to be sure not to overreach. Finally, we have to recognize that … Years ago, Trisha Greenhalgh and colleagues wrote an article in the BMJ citing evidence-based medicine as “a movement in crisis” It argued that we’ve moved too much from focusing on disease to risk. This point, more than any other, highlights the problem evidence-based medicine seems to have in the public sphere. If evidence-based medicine is to live up to its potential, it seems the focus should be on THAT side of the equation as well instead of taking best guesses and calling them evidence-based. That, probably more than anything else, has made the term so widely mistrusted. We love to make videos about things that sometimes YouTube doesn’t like to sell to advertisers. And we’re fine with that because what we do is more important than the money we might make. That said, any support you can give us to help keep the show going helps to make it bigger and better. And one good way to do that is through a subscription service called patreon.com. You can go to patreon.com/healthcaretriage and give as little as $1 a month to help the show keep going. If you don’t that’s fine. We’re gonna keep on making it anyway. We’d also appreciate if you subscribe to the show maybe like the video if you liked it. That’d be great too. We’d especially like to thank our Surgeon Admiral Sam and our research associates Joe Sevits and Carlos Callirgos. We also have some great merch you might like. We have mugs and posters and lunch boxes and other stuff and you can check it all out at HCTMerch.com. I also have my book. It’s still available anywhere books are sold: “The Bad Food Bible.” I’d really appreciate if you pick up a copy.

52 thoughts on “The Ups and Downs of Evidence Based Medicine

  • How about the evidence that the FDA does far more harm than good?

    Smallpox vaccine, polio vaccine, antibiotics, elimination of malaria with DDT, and blood typing/transfusions. These 5 things alone saved billions of lives, and none of them were FDA-approved.

    The FDA delays much-needed medical research.

  • 20 years ago, my cholesterol levels were perfect. Now, the essentially same levels are "borderline high."
    should I not wonder if this is due to some Big Pharma company needing to sell more drugs? Has the data on "utcomes" changed that much?
    Please keep doing this and write more for the NYT.

  • So yeah, using a scientific approach seems like a reasonable idea when it comes to life and death decisions. Although to be fair, a lot of sciences have trouble with people not using effect sizes and only relying on statistical significance instead.

  • Whether in heath or other fields, including my own, (mental health) I often see EBP as an excuse to make radical shifts that are not always proven. Having discourse about research and effective practice is so challenging anytime humans get involved. We need more discussion and professional humility when attempting to care for others.

  • Medicine should not be evidence based. It should be science based. There is no reason to study acupuncture for blood pressure by doing double blind placebo controlled clinical trials. We don't throw out everything we already know about human body in order to perform double blind placebo controlled trials for something so implausible. We have to consider prior plausibility before considering doing clinical trials and that's what science based medicine does. Evidence based medicine would waste resources on doing clinical trials on implausible interventions. That is why science based is better than evidence based.

  • I wasn't always following with this one. Some examples would be helpful, if you don't want to give real examples, some hypothetical ones would be fine.

  • If one is concerned about clinical practices lacking robust evidence, surely the answer is to GATHER MORE ROBUST EVIDENCE–not to throw out the very concept of evidence entirely.

  • We need to focus on gene therapy, neurology, and immunology. These three areas are most negatively affected by evidence-based medicine. And also the most likely areas of medicine to revolutionize individual treatment options.

  • I'd love to know Dr. Carroll's thoughts on science-based medicine. It seems to me that tis concept/approach mediates a lot of the problems associated with evidence-based medicine.

  • I really dont think this video explained what youre even talking about when you say "evidence based" you were far to vague when describing the difference and didnt really use any examples of evidence and non-evidence based medicine. Im sitting ehre at the end of the video with basically no clue wtf you were walking about for 6 minutes

  • So wait evidence based medicine isn't based on math and science? More like anecdotal evidence? I'm confused anyway I live math and science and there isn't anything that we know today that proper application can't point you toward. So can't we have a systems of medical research based solely on the maths and sciences? Sure clinical trials are super important, but how do we get there, trying it on animals who have similarities to us like rats immune systems, but that is still way off it seems like math based trials would be a better start. However I had never heard of evidence based medicine so this is more just me asking why not.

  • Taking a complicated topic of medicine and attempting to apply solutions to a general population in a capitalist environment just adds chaos. I love capitalism AND science, but feel mixing the two creates markets out of hokum. It's possible to do the science and then use capitalism to promote the proven outcomes, but I don't know anywhere where that happens. How can one discern if a prescribed treatment is truly evidence based best practices or merely tradition? How can one understand "increased risk" studies where something doubles, yet the initial risk was so low to make that increased risk remain insignificant? Flat out confusing as a patient and I bet as a medical pro too.

  • It's mysterious that the title uses the (incorrect) phrase "Evidence Based Medicine" while the (correct) phrase used everywhere in the video is "evidence-based medicine".

  • Evidence-based medicine should include the evidence that shows that individualised care and beyond broad pathophysiology. Not to mention the importance of rapport, psychology and the end goal of medicine which is not just to simply extend life as much as possible.

    When I think of evidence-based medicine, I think of tailoring the best evidence to your patient.

  • I am a child care worker and I think that evidence-based parctice is super important, but also that there are limitations. Take ABA therapy for children with autism for example or psychopharmacotherapy for children with conduct disorder / disruptive behavioral disorders. Both are evidence-based (ABA more so than pharmacotherapy) but both should be discussed critically. Some outcomes cannot easily be measured, studies are often relatively short-term and conducted in a clinical setting and there are also ethical oncerns that need to be considered. We should always ask ourselves who is researching what and how? I will not implement ABA even if studies tell me that I should, because I think it's unethical and doesn't reflect the goals that I personally have when working with children.
    But maybe this is a specific problem for psychology and related fields?

  • Another way to phrase some of what Dr. Carroll is trying to say here is that even when there is good clinical and scientific method used to study a particular issue, the results may not be relevant to what they are being cited to support, because many medical care and public health issues are affected by a huge number of factors, and looking at only one contributor to them myopically… no matter how well researched and accurate the information on the one factor is… can give a highly skewed and often even entirely false view of the reality of the complex bigger issue at hand. Either due to deliberate efforts to deceive, or honestly ignorant use of the information. Dr. Carroll is especially on target here… and in others of his videos… with the support of irrational fads in the mainstream media after one narrow area of research findings are announced.

    This is especially important in the case of risks vs benefits, where there may be a given risk identified, but that risk may be massively outweighed by the benefits. The picture becomes even more complex when one is looking at benefits that are subjective, not objectively quantifiable and measurable or applicable to all, and result from taste and life style preference. A love of hang-gliding or mountaineer, for example. Or: A love of the taste of red meat to the point where one can rationally decide "I like this SO much I am quite willing to give up 5 years of my life, if it means I'll lose that from eating as much of this as I do, in order to increase the quality of the life I have, as I see and perceive quality of my own life to be."

    Where the near always in his videos here brilliant, articulate, and exceptionally accurate and to the point Dr. Carroll may be either missing or somewhat neglecting an important issue relevant to his subject in the subject of this video is the basic truth that trying to provide medical care on a for profit basis, in the interests of a few astronomically wealthy individuals behind the pharmaceutical and insurance companies, will always result in massive and obscene perversions of the provision of care to any population. To be sure, any real world society has finite resources, and will have to make decisions how to best spend them on medical care. And be helped by science and evidence based findings in doing so. But first you have to eliminate the situation of putting huge fractions of that money spent on "public health" into the pockets of a small group (some of us would use the word "class") of powerful and super-wealthy individuals.

  • Back in the years before the understanding of the relationship of HPV to cervical cancer, and the availability of a vaccine for HPV for children, it was determined by good evidence based studies that if a girl/ young woman started having sex early on in her life, and had sex with multiple partners, she would have a greater risk of later getting cervical cancer. Religious fundamentalists and prudes and puritans of all sorts cited this perfectly correct and properly done research as "evidence" that medical science objectively supports one being chaste and later monogamous. They conveniently neglected to mention that it was also found that girls / women who did not become sexually active until later in life and had one or few partners had a higher incidence of uterine cancer. Or that uterine cancer was harder to spot and harder to treat than cervical cancer. This is an example of selective neglect of data involving citing good research for false conclusions. And, of course, of confirmation bias.

  • I like to to that evidence does not exist, just interpretation. If one understands the concept of type I error one will realize that there is no such thing as absolute truth in science. It is only a reasonable guess towards something, knowing that if the same research was repeated 100 times it will be wrong less than 5% of the time. I read once that the inventor of the P value, which is standard statistics in journals these days, never intended it to be used to imply truth. But rather, depending on individual values each individual should decide on their own what the truth is. For one person maybe their critical alpha is .05 but to another it is .00000001. The trouble with guidelines is that it is misconstrued as truth. The public and the media live in this world of black and white, in good and bad. But as one YouTuber I follow says, truth has a habit of resisting simplicity.

    Have a good day.

  • https://www.theindychannel.com/news/local-news/crime/indianapolis-mother-puts-acid-into-drink-to-help-daughter-with-autism

    I bet they thought this was evidence based too…

  • Hey Aaron! I love your videos. I make sure to keep up with the healthcare sphere because I am pre-med and am applying to med schools this June. Your videos are an integral part of increasing my healthcare knowledge. I am curious if you have ever considered going on the Joe Rogan podcast, or any podcast for that matter. I think it would be a great opportunity to share your channel, your new book, and your knowledge about how to understand scientific research.

    Again, thank you for your wealth of videos!

  • Any chance you could do an episode on the "hierarchy of research" you mentioned in this video? I have a science background so I'm pretty familiar, but I'm sure it would be helpful to a lot of people.

  • Why would anyone be against evidenced-based anything? What else would you do, just guess? Even anecdotal evidence is still evidence ffs.

  • I agree with all points but the solution seems to be…more evidence based medicine in order to refine that path. Because evidence needs to be properly gained and findings transparently issued. Corporate funding distorting findings by 40% on average (Cochrane Collaboration) doesn't help.

  • At first, EBM sounds like a no-brainer.  But it becomes clear  that it is only as good as the evidence that is being relied upon.  When you look at all the industry funded studies, a red flag should appear.  Groups like the Cochrane Collaboration have found that many studies reach conclusions that are inconsistent withe the data being presented, or worse, blatant fraud.  Best practices have been highjacked by special interests who understand how decisions are made regarding care, and take great care to assure that their interests are being served. Through no fault of their own, docs and insurers (payers) assume that what they're being told is accurate.  The game has been rigged.

  • Your mention of AI intrigued me. We're all so aware of the dangers of it, yet we continue to voraciously approach that goal, and we have no plans on how to fix the potential problems that might arise. Humans are an interesting bunch.

  • One of the things that can be extremely frustrating with regards to evidence-based medicine is when it is used when comparing evidence gathered in one population to create guidance in another.

    A very significant example here is that of transgender women and hormone therapy: the vast majority of the research regarding the benefits and harms of feminizing hormones (typically some combination of testosterone blockers, estradiol, and/or progesterone) has been done in post-menopausal cis women, which is an incredibly different population.

    The evidence for hormone therapy in transgender women is extremely poorly-studied. We do have good evidence that overall it is of tremendous benefit psychologically for transgender women. We do know that the largest health risk appears to be blood clotting. But there are many dimensions along which there just isn't any good evidence, such as:
    1) What is the right hormone regimen to maximize breast growth? Transgender women tend to have less total breast growth on average than cis women, and tend to be rather unsatisfied with the amount of growth. Learning what regimen is most likely to be effective would be of enormous benefit.
    2) As most of the studies on harms for hormone therapy have been in post-menopausal cis women, do some of the harms translate at all to the usually much younger and healthier trans women? Many claim that progesterone is a bad idea because it has not been proven to improve the health of post-menopausal cis women, and may increase their cancer risk. There's no good reason to believe that either statement is accurate for trans women. The benefits for breast growth and the risk of breast cancer are both poorly-studied in trans women (progesterone may lead to rounder breasts: most trans women have rather pointy breasts, but it may also suppress the action of estradiol in promoting primary breast growth).
    3) Is the largest risk in trans women avoidable (deep venous thrombosis, i.e. blood clotting)? There is some supposition that it may be a result of swallowing estradiol pills. If the trans woman uses a patch, injection, or uses sublingual administration, is the risk of DVT still there?
    4) What are the surgery risks associated with various hormone regimens? Should trans women stop taking hormones prior to surgery? How long before? Note that some trans women experience severe increases in psychological illnesses as a result of stopping hormones.

    I really do want more evidence-based medicine, but it seems like all too often it's applied poorly to underrepresented populations. Right now, trans women's best resources for understanding the medical implications of their care are usually other trans women, which is very inconsistent and harmful. I want more research, but in the mean time I wish more doctors would avoid applying evidence that does not apply.

  • Its key to remember that every body is unique, and just because something worked for others, doesnt mean it will for that individual. I have many rare disorders and also tend to react strangely to things, and I have found that some dr's are so enamored with evidence based medicine that they arent willing to think "outside the box" with me. Which is sad because it denies me needed care….

  • Worthy subject but poor argument. I was going to point to Sacket’s work in the 70’s and 80’s and the first incarnation of EBM as “critical assessment.” In fact, the elements of scientific epistemology go back to Aristotle. That some of the western scientific tradition has succumbed to the “difficulty of understanding what their salary depends on” is likewise, not a new. Nothing in human nature is new. It can almost have been anticipated that monied interests would attempt to coopt or discredit modes of thought that hurt their growth plans. Much the same way, direct primary care is being coopted or, at least redefined, by elements that stand to lose if DPC gains much of a foothold. Sorry EBM is a useful framework for an external validation of evidence based on agreed-upon criteria. The easiest way to attack it, is undermine the agreement.
    Climate anyone?

  • I love your work. Specially about the generic and branded medicines topic. Showed it to my students and made my lecture so much easier to explain!

  • ME/CFS is a good example of this. "Evidence" said it would go away with exercise and happy thoughts. Turns out that evidence was based on some of the worst science ever done!

  • Does HTC have a video on R.I.C.E. (rest, ice, compression, elevation) and it's apparent lack of scientific evidence? If not, that's something that seems like it'd be pretty interesting.

  • at my hospital, they needed evidence in the form of survey results to realise that staff needed access to drinking water at the clinical workplace.

  • 3:45 slight but important thing to note is that having a large enough cohort will ensure that anything not clinically significant/generalisable will be rarer. Doing more studies on smaller cohorts will increase the chance of picking up error though

  • https://annals.org/aim/fullarticle/718215/systematic-review-relationship-between-clinical-experience-quality-health-care
    Systematic Review: The Relationship between Clinical Experience and Quality of Health Care

  • Society is based on progress such as: Evidence-based constructions buildings – machinesThis exists because every part of the construction is known down to the detail, composition, size and objective. This allows a scientific certificate of the degree of quality to be delivered.
    Evidence-based medicine of the human body with 100% living cells that can grow, multiply and die, but nobody knows exactly how that happens. Consequently, delivering a scientific certificate with the grade of quality does not exist, because:
    – one does not know the composition of 1 living cell, let alone that medicine will determine how to repair the cells.
    – the totality of the number of cells is unknown.
    – one does not know the functions of the cells that make the body function optimally.
    – one does not know the totality of functions in the body.- the name, proteins, vitamins, … and their composition are a vision, but nobody can determine the exact composition of the atoms and physical properties exactly. The same for neurons, genes … are a name for something that is not known exactly.
    Conclusion: therefore there is no evidence-based medicine if the basic elements cannot be precisely described. The result: the human body is not a well-known medium for internal processing without causing damage. But the problem is much worse:- the scientists, doctors and politicians have laws approved whereby dead cells = chemicals come into contact with a medium of 100% living cells: the human body.
    The result: dead cells (chemicals) do not have a medicinal effect, but poison and a number of side effects have arisen. The law encourages the use of poison, this is not possible because a law cannot be used to harm people. The same reasoning applies to animals and nature that also consist of 100% living cells.
    The scientists are responsible for their absurd conclusions and actions.

  • Evidence based medicine is to Prove Mr Pharma is Right and more suckers will no need drugs and "treatment". The Doctor has Spoken.

Leave a Reply

Your email address will not be published. Required fields are marked *