Linear Correlations Anyone?

“correlation is charlatanism”
“Anything that relies on correlation is charlatanism” is a great article. But is correlation charlatanism? Yes it is. But it is not for the reasons explained in the article. Here is why.

Correlations are present everywhere. The concept of correlation is one of the key constructs of statistics, modelling, simulation. It is used to design portfolios, to estimate risks, in engineering design, in decision-making in biomedical research, in Big Data analysis, VaR, to impose lockdowns, etc., etc. Basically everywhere.

Over the years of work in the industry – not in front of computers – the lesson one learns is very simple: linear correlation may be used only when it may be used. If you don’t actually look at your data, if you don’t analyze it ‘visually’ first (the process is sometimes called ‘chi-by-eye’) you could be in serious trouble. Linear correlations can:

Destroy information
Distort information
Provide an overly optimistic picture (a sort of numerical placebo effect)
Miss crucial interdependencies

Just take a look at some scatter plots illustrated below and think about what a straight line passing through them could do to the information contained therein.