Complexity Economics Engineering Medicine Society

Linear Correlations Anyone?

“correlation is charlatanism”
“Anything that relies on correlation is charlatanism” is a great article. But is correlation charlatanism? Yes it is. But it is not for the reasons explained in the article. Here is why.

Correlations are present everywhere. The concept of correlation is one of the key constructs of statistics, modelling, simulation. It is used to design portfolios, to estimate risks, in engineering design, in decision-making in biomedical research, in Big Data analysis, VaR, to impose lockdowns, etc., etc. Basically everywhere.

Over the years of work in the industry – not in front of computers – the lesson one learns is very simple: linear correlation may be used only when it may be used. If you don’t actually look at your data, if you don’t analyze it ‘visually’ first (the process is sometimes called ‘chi-by-eye’) you could be in serious trouble. Linear correlations can:

  • Destroy information
  • Distort information
  • Provide an overly optimistic picture (a sort of numerical placebo effect)
  • Miss crucial interdependencies

Just take a look at some scatter plots illustrated below and think about what a straight line passing through them could do to the information contained therein.

Linear correlations anyone?

If you use linear correlations you will end up thinking linear. And good luck with that!

Established originally in 2005 in the USA, Ontonix is a technology company headquartered in Como, Italy. The unusual technology and solutions developed by Ontonix focus on countering what most threatens safety, advanced products, critical infrastructures, or IT network security - the rapid growth of complexity. In 2007 the company received recognition by being selected as Gartner's Cool Vendor. What makes Ontonix different from all those companies and research centers who claim to manage complexity is that we have a complexity metric. This means that we MEASURE complexity. We detect anomalies in complex defense systems without using Machine Learning for one very good reason: our clients don’t have the luxury of multiple examples of failures necessary to teach software to recognize them. We identify anomalies without having seen them before. Sometimes, you must get it right the first and only time!

0 comments on “Linear Correlations Anyone?

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: