What Counts As Evidence?

We live in the age of the online Wild Wild West, where anyone can produce anything about any subject and share it in a way that can reach millions in a very short amount of time and have people assume a level of conviction about it that rivals the most sectarian and militant of believers.

Given that you’re reading this, there’s a very good chance you have come across posts, videos, and maybe even a podcast episode or two where someone is advocating for a particular opinion about health and medicine that goes against the current mainstream position. This is not a problem in and of itself. It’s often the case that science advances one funeral at a time. It’s very much a social institution where peer pressure and personal interests can make it less pristine than its PR would like to present it to be. If we go by history, it’s often that one individual who comes along to identify a problem in the great edifice everyone is working on that’s so fundamental to the functioning of normal science, and aggressively pokes at it until it eventually causes a major shift if not an outright collapse and a complete overhaul.

But there’s an essential aspect to what makes this one or perhaps two contrarian individuals historically successful in their challenge of the mainstream, and it has to do with the evidence they provided for their challenge. The eventual acceptance of their account was not based on activists going around the streets of Florence or London where they shared pamphlets containing their claims without any evidence cited for them other than:

Dr. So and So is an amazing expert and he says [insert claim]”; or

Professor So and So has been researching [insert issue] for the past 30 years and he is an incredible mind.”

What made their challenge take hold was the increasing number of relevant experts in the field taking their claims and testing them out. Antoine-Laurent Lavoisier did not come out of nowhere and simply wrote a book for a new way to think about chemistry because he felt unsettled about alchemy, Aristotle’s four elements, and phlogiston. Rather, he conducted laboratory experiments and gave analyses that could then be tested for their reproducibility AND generativity of testable hypotheses in a logical fashion that revolutionized the field. His challenge was based on practice, not appeals to emotions or subjective experiences and personal convictions.

Well, we don’t live in an age of pamphlets as such. We also don’t live in an age that can be characterized by the patience required for rigorous and quite often monotonous research. It’s the age of hacks and short cuts. We traffic in dabbling enough to have what we need to feel satisfied we have confirmed our prior opinions. Ours is the age of documentaries filled with emotive content to put the fear of God into you about public health measures that saved millions of lives and 3-hour-long interviews with individuals promoting the most fantastical, but more importantly unverifiable and untestable explanations about why certain things are happening and why certain protocols are in place. The Beloved ﷺ said in a Hadith related in the collection of Imam al-Bukhārī, “Certainly, in some eloquent speech there is magic.” Our age of social media memes and documentaries “exposing the truth” takes eloquent speech and pamphlets of the past and puts them on steroids.

So how do you separate the wheat from the chaff?

1SF38EqxRAKNSku3ZlluILQ.png

This figure is known as “The Evidence Pyramid”. In medicine and health research in general, it demonstrates how claims are evaluated for their veracity based on where the evidence supporting them lies. Whenever you see someone making a claim about anything related to what is beneficial or harmful, or to what is dangerous or not, bring this figure up and ask them:

What is this based on?

Where can I find the studies supporting this?

If all you’re getting are the names of people, that is a red flag. Even if the experts cited happen to be experts in the actual field they are being cited in, their expertise does not preclude them from human error, biases, and limits of knowledge. This is why expert opinion is the weakest form of evidence. This is especially so if the expert happened to have been discredited due to fraud and fabrications of studies in a failed attempt at making millions off of desperate parents.

You can test the credibility of a claim by asking for at least retrospective or prospective studies. Do not accept expert opinions or case reports. If the claim is going to be radical, the least you can expect is a middle of the road level of evidence. Ideally, you should be seeing systematic reviews and meta-analyses for grandiose claims.

If the claims are vague, push back for specificity. Simply claiming a public health policy is “harmful” is not enough. Harmful how? What are you worried about specifically? But most importantly, what would it take for you to change your mind? This last question will often reveal why in most cases what is being presented as a “call for conversation” was not really one at all, because the conclusion was already pre-formed and it would not have mattered. In such a case, it is better to just talk about the weather and your mutual appreciation for coffee.

It’s a sign of a healthy society to have robust debates about important issues that impact us all. But that debate needs to be based on rational terms of engagement and when it’s about empirical claims, evidence that each one of us can go and confirm without having to ultimately rely on trusting the personal opinions and convictions of a few individuals who happen to present their arguments in a commanding fashion. The evidence for our arguments cannot be documentaries produced by people with clever videographers and sound effects to generate their desired emotional states within us.

May Allah allow us to see matters as they actually are, not as they are made to appear to us, or as we wish them to be.