Data are now part of every sector and function of the global economy and, like other essential factors of production such as hard assets and human capital, much of modern economic activity simply could not take place without them. The use of big data — large pools of data that can be brought together — will become the key basis of competition and growth for individual firms, enhancing productivity and creating significant value for the world economy.
Until now, the torrent of data flooding our world is a phenomenon that probably only excited a few data geeks. But we are now at an inflection point at which the sheer volume of data generated, stored, and mined for insights has become economically relevant to businesses, government, and consumers, finds new research from the McKinsey Global Institute (MGI) and McKinsey & Company’s Business Technology Office.
The history of IT investment and innovation and its impact on competitiveness and productivity strongly suggest that big data can have similar power to transform our lives. The same preconditions that enabled IT to power productivity are in place for big data and we expect suppliers of big data technology to have as much ongoing impact on productivity as suppliers of other kinds of IT.
All companies need to take big data and its potential to create value seriously if they want to compete. For example, a retailer embracing big data has the potential to increase its operating margin by 60 per cent.
The use of big data is becoming a key way for leading companies to outperform their peers. In most industries, established competitors and new entrants alike will leverage data-driven strategies to innovate, compete, and capture value. Indeed, we found early examples of such use of data in every sector we examined.
Big data will help to create new growth opportunities and entirely new categories of companies, such as those that aggregate and analyse industry data. Many of these will be companies that sit in the middle of large information flows where data about products and services, buyers and suppliers, consumer preferences and intent can be captured and analysed. Forward-thinking leaders across sectors should begin aggressively to build their organisations’ big data capabilities.
In addition to the sheer scale of big data, the real-time and high frequency nature of the data is also key. For example, ‘nowcasting’ is used extensively and adds considerable power to prediction. Similarly the high frequency of data allows users to test theories in near real-time and to a level never before possible.
We studied five domains in depth—healthcare and retail in the United States, the public sector in Europe, and manufacturing and personal location data globally—and some broadly applicable ways of leveraging big data emerged.
First, big data can unlock significant value by making information transparent. Second, as organisations create and store more transactional data in digital form, they can collect more accurate and detailed performance information on everything from product inventories to sick days and therefore expose variability and boost performance. In fact, some leading companies are using the ability to collect and analyse big data to conduct controlled experiments to make better management decisions.
Third, big data allows ever-narrower segmentation of their customers (or patients in the case of healthcare) and therefore much more precisely tailored products or services. Fourth, sophisticated analytics can substantially improve decision making, minimise risks, and unearth valuable insights that would otherwise remain hidden. Finally, and perhaps most intriguingly, big data can be used to improve the development of the next generation of products and services. For instance, manufacturers are using data obtained from sensors embedded in products to create innovative after-sales service offerings such as proactive maintenance (preventive measures that take place before a failure occurs or is even noticed).
So how significant could the value created by big data be? We find, for instance, that if the US healthcare system were to use big data creatively and effectively to drive efficiency and quality, the sector could create more than $300bn in value every year. Two-thirds of that would be in the form of reducing US healthcare expenditure by about 8 per cent. In the developed economies of Europe, government administrators could save more than €100bn ($149bn) in operational efficiency improvements alone by using big data – and that’s not including employing big data tools to reduce fraud and errors and boost the collection of tax revenues.
But it’s not just companies and organisations that stand to gain from the value that big data can create – consumers can reap highly significant benefits, too. For instance, users of services enabled by personal location data can capture $600bn in consumer surplus.
Take smart routing using real-time traffic information, which is one of the most heavily used applications of personal location data. As the penetration of smart phones increases, and free navigation applications are included in these devices, the use of smart routing is likely to grow. By 2020, more than 70 per cent of mobile phones are expected to have GPS capability, up from 20 per cent in 2010. All told, we estimate that the potential global value of smart routing in the form of time and fuel savings will be about $500bn by 2020. This is equivalent to saving drivers 20bn hours on the road, or 10 to 15 hours every year for each traveller, and about $150bn on fuel consumption.
Some of the most significant potential to generate value out of big data will come from combining different, separate pools of data. The US healthcare system, for instance, has four major pools of data – clinical; activity (claims) and cost; pharmaceutical and medical products R&D; and data about patient behaviour and sentiment – each of which is primarily held by a different constituency. MGI estimates that if US healthcare used all available big data tools to the full, the annual productivity of the sector could grow by an additional 0.7 per cent. But to achieve this welcome boost to productivity will require access to data -often on a scale beyond those that an individual organisation generates. Sets of data such as patient records and clinical claims may have to be combined.
Doing so would create benefits not just for the various industry players but for patients who would have broader, clearer access to a wider variety of healthcare information, making them more informed consumers of the medical system. Patients would be able to compare not only the prices of drugs, treatments, and physicians, but also their relative effectiveness, enabling them to choose more effective, better-targeted medicines, potentially even customised to their personal genetic and molecular make-up. To obtain those broad benefits, healthcare consumers may have to accept a slightly different trade-off between their privacy, a significant hurdle to the use of big data in many countries, and the benefits that wider pooling of data would bring.
Sensitivities around privacy – and indeed the security of data – are just one hurdle that companies and governments need to overcome if the economic benefits of big data are to be realised. One of the most pressing challenges is a significant shortage of people with the skills to analyse big data. By 2018, the United States alone could face a shortage of 140,000 to 190,000 people with deep analytical training (in statistics or machine learning) and another 1.5m people with the managerial and quantitative skills to be able to frame and interpret analyses effectively enough to base decisions on them.
There are also many technological issues that need to be resolved to make the most of big data. Legacy systems and incompatible standards and formats often prevent the integration of data and the more sophisticated analytics that create value and organisations will need to deploy new technologies and techniques as they develop to overcome this barrier. Ultimately, making use of large digital datasets will require the assembly of a technology stack from storage and computing through analytical and visualisation software applications.
Above all, access to data needs to broaden. Increasingly companies will need to access data from third parties and integrate them with their own but today there are few areas where there are efficient markets for the trading or sharing of data—for example, there is no market for the sharing of the aggregate movement patterns derived from mobile phones that retailers want to mine as they try to understand the behaviour of their customers. In the public sector, it looks unlikely that data sharing will occur even if this would be to the mutual benefit of the parties involved—and legislators may have to step in.
As long as companies and governments understand the power of big data to deliver higher productivity and consumer surplus and the next wave of growth in the global economy, there should be strong enough incentive to act robustly to overcome these barriers to its use. By doing so they will unleash avenues to new competitiveness among companies, higher efficiency in the public sector that will enable better services even in constrained fiscal times, and an injection of new productivity not only into firms but into entire economies. Big data is a big deal.
This article originally ran in FT.com The Connected Business.
