On Data Quality and Data-driven Business Models.

Nowadays, many companies rely on a data-driven business model. This means they collect, store, process and analyse data in order to extract insights, knowledge and to make extrapolations, spot tendencies, make plans and build strategies. This not only helps understand their own products as well their customers, it is indispensable. The value of data cannot be overstated.

Ontonix analyses data from all its customers. Over the past two decades, thousands of data sets have been analysed with QCM technology. What we observe is surprising and can be summarized as follows:

With very few exceptions – the 80-20 rule is pretty much the case – most companies lack a proper data culture. Data is stored, for sure, but there appear to be very loose protocols in place. On occasion, there are no data collection protocols.

When data is extracted from devices by means of sensors, data is stored as is, without paying much attention to erroneous or off the scale readings.

When off the scale readings are encountered, the collecting device sometimes replaces them with some standard, typically very high integers. Unless spotted, these will disrupt and distort any analytics, providing false results.

Presence of temporal discontinuities in observation periods as well as overlap between periods is not uncommon. This can cause data to be ill-conditioned. Ill-conditioned data is often chaotic in nature and deprived of structure.

Because of memory limitations, especially on portable devices, instead of storing raw data, averages are stored. This not only destroys information, it also shows that no sound data collection policy is in place and that the entire process is designed merely to comply with memory limitations.

A final issue is that of number of samples. When temporal, spatial or frequency domain sampling are concerned, attempting to analyse data consisting of 3-4 samples is irrelevant, to say the least. An example is that of balance sheet data of corporations. Attempting to assess the dynamics and resilience of a business based on 4 quarterly samples is simply a waste of time.

Today,

Ontonix offers a data quality assessment service

This service is offered independently of performing consulting projects. Data quality is not only fundamental for all our customers whose business models are data driven, it is conditio-sine-quan-non for Ontonix to accept a project engagement.

Based on recent experience,

high data quality is indispensable in order to engage Ontonix

Measuring data quality has been addressed in a previous article. It describes how the QCM can offer a proxy for a condition number of a generic data set.