The limits of statistical thinking and the value of personal experience

This post is an answer to an article titled “The limits of our personal experience and the value of statistics” posted to the Our World In Data (OWiD) website the 27th of July. You can access and read freely the article at:

https://ourworldindata.org/limits-personal-experience

For context, I enjoy reading articles submitted to OWiD and even donated to them once.

However, that article is nonsense. Trying to display a purely philosophical and unfalsifiable argument as objective by wrapping it in pretend-math and using irrelevant data.

Hot take

First, it’s a reddit-level “hot take”. Saying loud what everyone thinks, preaching to the choir and pretending you have an original or unique opinion.

Data-driven decision making might be the most important aspect of modern society. This idea of “objective” decision making has been the definition of “modern state”, as Hegel puts it¹, since the 19th century. Engels and Marx’s philosophy is an extension of this idea. This is the founding philosophy of the soviet Union. Class struggle is only a means to this ideal “modern state”. OWiD coming out and saying we haven’t enough of it is really weird.

Most (if not all) policy making today is justified by some statistical datum (whether of quality or not). I suspect a very large majority of people would say “statistics” if they were asked “which of statistics or personal human experience should be used to make policy decisions?” It is absurd to say that usage of statistic for policy making is a minority view. Don’t limit yourself to the government when thinking “policy making”, also consider private companies. Both pretend to use statistics in their decisions.

Statistics are as sparse as personal experience

Second, the visualization is dumb and misleading. One thing it forgets is that no statistical data is exhaustive. So the last image with the filled rectangle is just wrong. Often time we don’t even have the privilege to sample even a hundredth of the whole population. In fact, “The perspective that global statistics offer” should be as sparse as “The fragmented perspective of the news media”. If not more.

I hear you say “but with statistical inference and the law of large numbers, we can expand the small sample to be representative of the whole”. Yes, maybe. But this is also true of human experience. We, as human have this capacity of taking several data points and construct a larger understanding based on them. The exact mechanisms used by the human brain are unknown, but might very well look similar to the maths used for statistical inference (Bayesian inference). So either way, regardless of how you look at it, if you use the same criteria to draw the two “perspective” panels, you end up with the same picture.

“Carefully constructed” statistics

Third, it puts a bit too much weight behind “carefully constructed” statistics.

By nature statistics are limiting. Each data point is extremely narrow, human condition has many, many, many different aspects. Do I have electricity? How much? Access to a fridge? Clean water? But to which quantity? And what do you mean by clean? With a statistical approach, you need to quantize² the information to the point it might become meaningless in non-obvious ways.

Even if you come at it in a naive way, open to all possible outcomes (which is rarely the case), you are still victim to hidden biases that will color statistical resutls in certain directions.

Talk with someone, and you immediately know that, yes they have access to “clean” water, but if you bring a lit match to it, it burns. With a statistical approach, the numbers invalidate human experience, while it should always be the opposite. Saying to someone that they are wrong about a fundamental part of their being “because the numbers say otherwise” is the worst insult you can make. Always put humans first, please.

So add a 3rd dimension to those sampling panels, representing the “aspect of human experience” that is being measured. There are millions of aspects to human experience. Each statistical data point will always be extremely narrowly contained withing a dozen aspects. Talking with someone, you have a much larger idea of what their life is, much more than a question sheet with multiple choice answers could. This isn’t even mentioning the aspects of human life that are simply immeasurable (faith, happiness). What do we do of them? We ignore them as “impractical”?

Sortition

Statistics do not look so good now, do they?

Well, I think the law of large numbers is still useful.

It is foolish to try to build up a representation of reality through statistics alone. Especially when making decisions impacting humans. It is also foolish to try to talk to every parties involved in full depth. No time for that! But we can still get humans to “create” statistics. This is what society is about!

Take a representative sample of humans, put them in a room and let them build consensus on what are the best policies.

This is called sortition, and it is what “democracy” first meant before it then meant “anarchy” in the 19th century, followed by “universally elected representative republic” in the 20th and 21st³.

The result will be a decision made through the synthesis of all aspects of human life, from all horizons. Not a fully lit cube, but probably the best lit cube. Debate and controversy is not trendy, I know, but it’s what drove us to where we are now. It’s also the only way to empower everyone, as opposed to imparting decisions from above⁴.

Remember! Statistics is about collecting a small sample and extrapolating it to the whole universe through very simple maths. Sortition does the same, but instead of using blind mathematical formulas that has no knowledge of the complexity of the world, we use humans to extrapolate to the universe.

This naive fetishism of data and science is dangerous⁵. We can’t ever know the world as a whole, we can only see through a narrow window into it⁶. The world is too complex to be understood fully. This is true of us humans, but also statistical samples or the (beautiful) scientific endeavor.

Science is not a representation of a superior unalterable platonic truth. It is not the oracle of the philosopher king. Science, in fact, starts with the specific premise that it indeed can never be such a truth. By extension, there can’t be a scientific philosopher king.

Science also has this property of ending all conversations. It is by definition an appeal to authority. You can’t argue with an appeal to authority, so what you end up with is two people talking past each other appealing to different authorities. It’s way worse than if you didn’t bring science into the debate in the first place!

Basically this article reads like a (half descent) 6th grade essay. It tries to drape itself into the cloth of science by showing they can do silly math on the width of hair, pretends to lobby for science, yet doesn’t even understand the first thing about science.

It is a plea to remove from policymaking the last remaining input taken from human experience. An Hegelian eulogy of humanless policy.

While I respect OWiD, and love the visualizations and insights their articles provide (I’ve donated to them in the past!) I think our society is already dehumanizing and cold. We should make it more human, listen to more people when making decisions.

More often than not, statistics is a way to launder personal opinions into “objective truth” in order to legitimize the unacceptable.

What we need is not statistics, or more science, or to bow ourselves to the altar of universal knowledge. What we need is modesty and an open ear.