Big Data, Big Chasm

“On Exactitude in Science" or "Del rigor en la ciencia" is a one-paragraph short story written in 1946 by Jorge Luis Borges, about map–territory relation. It imagines an empire where the science of cartography becomes so exact that only a map on the same scale as the empire itself will suffice. But, then it’s not a map. It is the world just as it is!!

Same is true of data and brand building. Data should evidence patterns that give insight. Else, data is just everything about the world as it is and that is of little use!  

Data has been heralded as the ‘new oil’, but what good is oil be if nobody knows where to find it, extract it, refine it, store it and use it? In my opinion data is more like electricity. It is not finite, lying underground and waiting to be discovered. It has to be created, tapped, transmitted, utilised and handled carefully. Not knowing about your data or its location can cause compliance vulnerabilities and security risks. Since the GDPR legislation came into play, this has been a constant red flag. Its severity goes up for sensitive data such as financial or health records.

Data is undeniably the most valuable resource for an organisation today. As a key business decision driver across industries, understanding data, and data analytics, in particular, is often crucial to success. Modern businesses are adopting technology to organise and comprehend the huge amounts of information they collect. But when the data with which big decisions are being made is ‘bad’, or spurious what can that mean for data-driven businesses?

From evidence across industries, it is quite clear that customer experience ambitions are being challenged by this phenomenon. The three most common factors preventing businesses from using data to their advantage are data inaccuracy, lack of direct control needed to impact strategic objectives and information overload

A lack of trust in data derived insights isn’t necessarily a result of the data itself, but how it is being managed and collected. Historically, businesses have been slow to tackle data quality issues. Inertia makes them endure pains and fix issues sporadically and reactively. Data fragmentation is a common occurrence where legacy structures impose on data collection, storage, pooling, analysis and utilisation. 

Data is compartmentalised, scattered or located in pieces or multiple copies all over an organisation’s IT system, leading to an incomplete single view of the data, its components, and an inability to extract real value from it. The vast majority of data sets are typically located on secondary storage, used for backups, archives, object stores, file shares, test and development, and analytics

However, when fragmented – as is often the case – it can be extremely difficult to locate, manage or put to any use. I have led more than one digital transformation project and in each case the underlying data foundation was the most critical concern. The entire superstructure rests on that foundation. Any customer journey involves many moving parts that the company must connect and orchestrate, including the behind-the-scenes middle and back offices and various support functions that have no direct contact with customers, such as marketing, product, operations, HR, finance, legal, risk, and compliance.

For realising the power of data based transformation, customer journeys have to be treated collectively as one comprehensive project rather than as multiple projects. The only way that can happen is if the data flow and single view of customer allows for it. Data pooling, de-duplication and harmonisation is the biggest effort in such a program. 

As a first step, a company needs to develop a list of relevant journeys, evaluating the business benefits of each and tying the expected outcomes to the company’s overall strategy and purpose. These journeys then become the primary basis for organizing the required data and structuring the teams. For example, a bank should have a journey - “Helping a customer buy a home” rather than separate steps of filing a loan application, submission, documentation, validation, contract etc.

Many business leaders view their secondary data as expensive to store, of poor utility and a growing compliance risk. But a lack of control around data ownership will impact strategic ambitions, particularly around customer experience, agility, growth and competitiveness. It seems obvious that businesses that can’t get in front of mass data fragmentation, and tackle such data quality issues, face serious disadvantages that may jeopardise success for years to come.

The inability to manage and harness insights is a big competitive disadvantage when it comes to customer satisfaction and development of products and services. Secondly, the inability to know your multiple data sets and its location can cause compliance vulnerabilities and security risks. The problem of poor data and fragmentation is not only an IT concern: it’s a business one. If IT is expected to manage all the organisation’s secondary data and apps across all locations, but neither standard operating procedures nor technology is in place to accomplish that goal, IT leaders will understandably be worried about a wide array of major problems occurring in several different areas.

While technology plays a key role in data management and the improvement of data quality, changes in working processes, organization of cross functional teams, reward criteria and employee behaviour are critical too. There is clearly a need for a Chief Data Officer to reinforce both data compliance and security. The solution, of course, would have to encompass a better way of storing, managing, protecting and extracting value from the wide-ranging pools of secondary data. 

Breaking down business silos, preventing redundancies and using technology to help give real time access to data. These are cultural and business process issues. 

The world of business is on two sides of a chasm. On one side is a legacy world which, at best, allows incremental improvements to current processes. On the other side is a customer centric, data driven model of doing business with an alert eye to procuring, cleaning, storing and using data. Those individuals, teams and organizations who will leap over this chasm will flourish. The rest will become archival data.