What’s now and next in analytics, AI, and automation

Rapid technological advances in digitization and data and analytics have been reshaping the business landscape, supercharging performance, and enabling the emergence of new business innovations and new forms of competition. At the same time, the technology itself continues to evolve, bringing new waves of advances in robotics, analytics, and artificial intelligence (AI), and especially machine learning. Together they amount to a step change in technical capabilities that could have profound implications for business, for the economy, and more broadly, for society. 

Table of contents

1. The opportunity available now
2. The next wave of opportunity
3. What about employment and work?
4. What should leaders do?

The opportunity available now

Some companies are gaining a competitive edge with their use of data and analytics, which can enable faster and larger-scale evidence-based decision making, insight generation, and process optimization. But there is room to catch up and to excel. Harnessing digitization’s potential is similarly uneven.

Data and analytics are transformational, yet many companies are capturing only a fraction of their value

Data and analytics have been changing the basis of competition in the years since our first report on big data in 2011. Leading companies are using their capabilities not only to improve their core operations but also to launch entirely new business models. The network effects of digital platforms are creating a winner-take-most dynamic in some markets. Yet while the volume of available data has grown exponentially in recent years, most companies are capturing only a fraction of the potential value in terms of revenue and profit gains.

Effective data and analytics transformations have several components:

• Asking fundamental questions to shape the strategic vision: What will data and analytics be used for? How will the insights drive value? Which data sets are most useful for the insights needed?
• Solving for the problems in the way data is generated, collected, and organized. Many incumbents struggle to switch from legacy data systems to a more nimble and flexible architecture that can get the most out of big data and analytics. They may also need to digitize their operations more fully in order to capture more data from their customer interactions, supply chains, equipment, and internal processes.
• Acquiring the skills needed to derive insights from data; organizations may choose to add in-house capabilities or outsource to specialists.
• Changing business processes to incorporate data insights into the actual workflow. This is a common stumbling block. It requires getting the right data insights into the hands of decision makers—and making sure that these executives and mid-level managers understand how to use data-driven insights.

Putting all these components in place is not easy. In a recent McKinsey survey of more than 500 executives representing companies across the spectrum of industries, regions, and sizes, more than 85% acknowledged that they were only somewhat effective at meeting goals they set for their data and analytics initiatives.

Data and analytics are disrupting business models and bringing performance benefits

Disruptive data-driven models and capabilities are reshaping some industries, and could transform many more. Certain characteristics of a given market open the door to disruption by those using new data-driven approaches, including:

• inefficient matching of supply and demand
• prevalence of underutilized assets
• dependence on large amounts of demographic data when behavioral data is now available
• human biases and errors in a data-rich environment 

In industries where most incumbents have become used to relying on a certain kind of standardized data to make decisions, bringing in fresh types of data sets (“orthogonal data”) to supplement those already in use can change the basis of competition. We see this playing out for example in property and casualty insurance, where new companies have entered the marketplace with telematics data that provides insight into driving behavior, beyond the demographic data that had previously been used for underwriting.

One of the most powerful uses is micro-segmentation based on behavioral characteristics of individuals. This is changing the fundamentals of competition in many sectors, including education, travel and leisure, media, retail, and advertising.

Digitization, more broadly, is also progressing unevenly among companies, sectors, and economies

The corporate world’s broader embrace of digitization is similarly uneven. Our use of the term digitization (and our measurement of it), encompasses:

• Assets, including infrastructure, connected machines, data, and data platforms, etc.,
• Operations, including processes, payments and business models, customer and supply chain interactions and
• The workforce, including worker use of digital tools, digitally-skilled workers, new digital jobs, and roles. In measuring each of these various aspects of digitization, we find relatively large disparities even among big companies (Exhibit 1).

1

Workforce transitions in a time of automation

Read more

Our research finds that companies with advanced digital capabilities across assets, operations, and workforces grow revenue and market shares faster than peers. They improve profit margins three times more rapidly than average and, more often than not, have been the fastest innovators and the disruptors in their sectors—and in some cases beyond them.

Many of these top performers were “born digital,” but perhaps more impressive are the smaller set of incumbent companies that have actively transformed themselves into digital leaders and benefit doubly from their traditional strengths and their new digital capabilities.

There are also disparities between sectors in terms of degree of digitization:

• In the United States, the information and communications technology (ICT) sector, media, financial services, and professional services are surging ahead, while utilities, mining, and manufacturing, among others, are in the early stages of digitizing. In labor-intensive industries such as retail and health care, substantial parts of their large workforces do not use technology extensively.
• This unevenness can also be observed across countries; all have significant room to increase their digitization:

• The US economy as a whole is reaching only 18% of its digital potential;
• France has achieved 12% of its digital potential, the European Union average, while Germany and Italy are at 10%;
• Emerging economies are even further behind, with countries in the Middle East and Brazil capturing less than 10% of their digital potential.

Digitization is transforming globalization, creating opportunities now for companies and economies

The world is more connected than ever, but the nature of its connections has changed in a fundamental way. The amount of cross-border data flows has grown 45 times larger since just 2005. It is projected to increase by an additional nine times over the next five years as flows of information, searches, communication, video, transactions, and intracompany traffic continue to surge.

In addition to transmitting valuable streams of information and ideas in their own right, data flows enable the movement of goods, services, finance, and people. Virtually every type of cross-border transaction now has a digital component.

Approximately 12% of the global goods trade is conducted via international e-commerce, with much of it driven by platforms such as Alibaba, Amazon, eBay, Flipkart, and Rakuten. Beyond e-commerce, digital platforms for both traditional employment and freelance assignments are beginning to create a more global labor market. Some 50% of the world’s traded services are already digitized. These transformations enable small and medium-sized enterprises around the world to compete head to head with larger industry incumbents.

The next wave of opportunity

Coming over the horizon is a new wave of opportunity related to the use of robotics, machine learning, and AI. Companies that deploy automation technologies can realize substantial performance gains and take the lead in their industries, even as their efforts contribute to economy-level increases in productivity. 

Advances in robotics, AI, and machine learning herald a new era of breakthrough innovation and opportunity

Recent advances in robotics, machine learning, and AI are pushing the frontier of what machines are capable of doing in all facets of business and the economy.

Physical robots have been around for a long time in manufacturing, but more capable, more flexible, safer, and less expensive robots are now engaging in ever expanding activities and combining both mechanization, cognitive and learning capabilities—and improving over time as they are trained by their human coworkers on the shop floor, or increasingly learn by themselves.

The idea of AI is not new, but the pace of recent breakthroughs is. Three factors are driving this acceleration:

• Machine-learning algorithms have progressed in recent years, especially through the development of deep learning and reinforcement-learning techniques based on neural networks.
• Computing capacity has become available to train larger and more complex models much faster. Graphics processing units (GPUs), originally designed to render the computer graphics in video games, have been repurposed to execute the data and algorithm crunching required for machine learning at speeds many times faster than traditional processor chips. More silicon-level advances beyond the current generation of GPUs are already emerging, such as Tensor Units.  This compute capacity has been aggregated in hyper-scalable data centers and is accessible to users through the cloud.
• Massive amounts of data that can be used to train machine learning models are being generated, for example through daily creation of billions of images, online click streams, voice and video, mobile locations, and sensors embedded in the Internet of Things. 

The combination of these breakthroughs has led to spectacular demonstrations like DeepMind’s AlphaGo, which defeated a human champion of the complex board game Go in March 2016. Google’s DeepMind and the University of Oxford applied deep learning to a huge data set of BBC programs in 2016 to create a lip-reading system that is more accurate than a professional lip reader.

Formidable technological challenges must still be overcome before machines can match human performance across the range of cognitive activities. One of the biggest technical challenges is for machines to acquire the capability to understand and generate natural language—capabilities that are indispensable for a multitude of work activities. Digital personal assistants such as Apple’s Siri, Amazon’s Alexa, and Google Assistant, are still in development—and often imperfect—even though their progress is palpable for millions of smartphone users.

Harnessing these evolving technologies will unlock multiple benefits for companies

For companies, successful adoption of these evolving technologies will significantly enhance performance. Some of the gains will come from labor substitution, but automation also has the potential to enhance productivity, raise throughput, improve predictions, outcomes, accuracy, and optimization, as well expand the discovery of new solutions in massively complex areas such as synthetic biology and material science.

Already today, a range of automation technologies is generating real value. For example:

• Rio Tinto has deployed automated haul trucks and drilling machines at its mines in Pilbara, Australia, and says it is seeing 10–20% increases in utilization there.
• Google has applied artificial intelligence from its DeepMind machine learning to its own data centers, cutting the amount of energy they use by 40%.
• In financial services, automation in the form of “straight-through processing,” where transaction workflows are digitized end-to-end, can increase the scalability of transaction throughput by 80%, while reducing errors by half.

Furthermore, a plethora of machine learning business use cases are emerging across sectors (Exhibit 2).

2

Scenarios we developed for several settings, including a hospital emergency department, aircraft maintenance, oil and gas operations, a grocery store, and mortgage brokering, show that the value of the potential benefits of automation—calculated as a percentage of operating costs—could range from between 10–15% for a hospital emergency department to 25% for aircraft maintenance, and to more than 90% for mortgage origination.

AI and Automation will provide a much-needed boost to global productivity and may help some ‘moonshot’ challenges

The application of AI and the automation of activities can enable productivity growth and other benefits not just for businesses, but also for entire economies. At a macroeconomic level, based on our scenario modeling, we estimate automation alone could raise productivity growth on a global basis by 0.8% to 1.4% annually.

AI and other technologies can also be broadly beneficial for society by helping tackle some “moonshot” challenges, including climate change or curing disease. AI is already being deployed in synthetic biology, cancer research, climate science, and material science. For example, researchers at McMaster and Vanderbilt University have used computers to exceed the human standard in predicting the most effective treatment for major depressive disorders and eventual outcomes of breast cancer patients.

What about employment and work?

The advent of a new automation age is raising public concerns about the effect on employment and the future of work. For most occupations, partial automation is more likely than full automation in the medium term, and the technologies will provide new opportunities for job creation.

About half the activities carried out by workers today have the potential to be automated

To assess the employment implications of automation, we focused on work activities rather than whole occupations as a starting point. We consider work activities to be a useful measure since occupations are aggregations of different activities, where each discrete activity has a different potential for automation. For example, a retail salesperson will spend some time interacting with customers, stocking shelves, or ringing up sales.

Estimating effect of automation on productivity growth globally

Activities that are more easily automatable include physical activities in highly predictable and structured environments, as well as data collection and data processing (Exhibit 3). These activities exist across the entire spectrum of sectors, as this data visualization of the automation potential of individual sectors shows.

3

Our analysis of the automation potential extends to 46 countries representing about 80% of the global workforce. Overall, we estimate that about half of the activities that people are paid almost $15 trillion to do in the global economy have the potential to be automated by adapting currently demonstrated technology. This data visualization of global automation potential shows sizable differences between countries, based mainly on the structure of their economies, the relative level of wages, and the size and dynamics of the workforce.

All occupations will be affected. Only a small proportion of all occupations, about 5%, consist of 100% of activities that are fully automatable using currently demonstrated technologies. However, we find that about 30% of the activities in 60% of all occupations could be automated (Exhibit 4). This means that many workers will work alongside rapidly evolving machines, which will require worker skills also evolve. This rapid evolution in the nature of work will affect everyone from welders to landscape gardeners, mortgage brokers—and CEOs; we estimate about 25% of CEOs’ time is currently spent on activities that machines could do, such as analyzing reports and data to inform decisions.

Our interactive data visualization of global automation potential shows sizable differences between countries.

Learn more on Tableau public

Several key factors will influence the pace and extent of automation. These include:

• Technical feasibility of automation, a critical first step that will depend on sustained breakthrough innovation, but alone is not sufficient;
• Cost of developing and deploying solutions;
• Labor market dynamics, including supply and demand, and costs of human labor as an alternative to automation;
• Business and economic benefits, not merely labor substitution benefits but also benefits from new capabilities that go beyond human capabilities;
• Regulatory, user and social acceptance, which can affect the rate of adoption even when deployment makes business and economic sense.

A useful analogy to consider is that electric vehicles were demonstrated to be technically feasible several decades ago, but it was not until some of these other factors became realistic that they showed up on the road.

4

Technology will also help create new jobs and new opportunities for generating income, and will help labor markets function better

The scale of shifts in the labor force over many decades that automation technologies will likely unleash is of a similar order of magnitude to the long-term technology-enabled shifts in the developed countries’ workforces as they moved most workers from farms to factories and service jobs. Those shifts did not result in long-term mass unemployment because they were accompanied by the creation of new types of work not foreseen at the time. We cannot definitively say whether historical precedent will be repeated this time. But our analysis shows that humans will still be needed in the workforce.

So even while technologies replace some jobs, they are creating new work in industries that most of us cannot even imagine, as well as new ways to generate income and match talent to jobs.

One third of new jobs created in the United States in the past 25 years were types that did not previously exist, or barely existed, in areas including IT development, hardware manufacturing, app creation, and IT systems management. The growing role of big data in the economy and business will create a significant need for statisticians and data analysts, for example; we estimate a shortfall of up to 250,000 data scientists in the US in a decade.

Technology helps work in other ways. Digital talent platforms such as LinkedIn have already begun to improve matching of workers with jobs, creating transparency and efficiency in labor markets, and thereby raising GDP. While it is early days, there is already evidence that such platforms can raise labor participation and working hours.

While independent work is nothing new (and self-employment is still the predominant form of work in emerging economies), the digital enablement of it is. Our research finds that 20% to 30% of the working age population in the US and the European Union is engaged in independent work. Just over half of these workers supplement their income and have traditional jobs, or are students, retirees, or caregivers. While 70% choose this type of work, 30% turn to it out of necessity because they cannot find a traditional job at all, or one that meets their income and flexibility needs. The proportion of independent work that is conducted on digital platforms, while only about 15% of independent work overall, is growing rapidly, driven by the scale, efficiency, and ease of use for workers and customers that these platforms enable.

Those who pursue independent work (digitally enabled or not) out of preference are generally satisfied, although those who pursue it out of necessity are unsatisfied with the income variability and the lack of benefits typically associated with traditional work.

What should leaders do?

Business leaders and policy makers have an imperative to find ways to harness the potential of these technologies, even as they will have to address the significant challenges.

Business leaders

For businesses, the opportunities are clear. Leaders should embrace the transformation and performance opportunities already available to them (and their competitors) from data, analytics, and digitization, as well as the rapidly evolving opportunities in AI, robotics, and automation. To harness these benefits, business leaders will not only have to invest in technology, but also in transforming their organizations. Specific approaches will vary business by business, however several new mindsets will be critical:

• Testing, experimenting, learning, and scaling fast: Beyond book knowledge, business leaders will need to amass practical knowledge from devoting resources to experiments applying technologies to real problems, and then scaling those that show promise.
• Reimagining business models and business processes: To make full use of the power of analytics, AI, and other digital technologies will require a thorough reimagining of processes, with priorities for which processes to transform. Similarly, leaders will need to reimagine how current business models could be transformed and how new business models could be created based on these capabilities.
• Digital assets and capabilities as the “new balance sheet”: These assets and capabilities, both hard and soft, are increasingly becoming a competitive differentiator and platforms for innovation and disruption. Each business regardless of industry and sector will likely need to assess how distinctive its digital assets and capabilities are vs. those of competitors.
• Staying calibrated and investing accordingly: When it comes to digital capabilities and progress on digitization initiatives, all too often business leaders are satisfied with progress vs. their own past. The most relevant calibration will be relative to the scale of the opportunity and vs. competitors and potential disruptors both from within their sectors and from outside them.
• A new focus on human capital, including integrating workers and machines: Companies are likely to face gaps in skills they need in a more technology-enabled workplace, and would benefit from playing a more active role in education and training. Humans and machines will need to work together much more closely. That will require retraining and often redeploying workers.

Policy makers and business leaders concerned with wider economic and societal implications

Policy makers also have a powerful incentive to embrace the productivity growth opportunity for their economies that these technologies offer. This will help ensure future prosperity, and create the surpluses that can be used to assist workers and society adapt to these rapid changes. At the same time, policy makers must evolve and innovate policies that help workers and institutions adapt to the impact on employment:

• Adopting policies to encourage investment: Through tax benefits and other incentives, policy makers can encourage companies to invest in human capital. Policy makers could accelerate the creation of jobs in general through stimulating investment, and accelerate creation of digital jobs in particular.
• Encouraging new forms of entrepreneurship and more rapid new business formation: Digitally enabled opportunities for individuals to earn incomes. In addition, accelerating the rate of new business formation will be critical. This will likely require simplifying regulations, creating tax and other incentives.
• Public–private partnerships to stimulate infrastructure investment: The lack of enabling digital infrastructure is holding back the digital benefits for some emerging economies—and even underserved regions in developed countries. Public–private partnerships could help address market failures.
• Rethinking education, training, and learning: Policy makers working with education providers could do more to improve basic science, technology, engineering, and math (STEM) skills through the school systems, and put a new emphasis on creativity as well as critical and systems thinking.
• Rethinking income support and safety nets: If automation (full or partial) does result in a significant reduction in employment and/or greater pressure on wages, some ideas such as universal basic income, conditional transfers, and adapted social safety nets may need to be considered and tested.
• Incent investment in human capital: A broad range of incentives exist for businesses to make capital and R&D incentives. Something similar needed to encourage investment in human capital.

A full version of this briefing note is available as a PDF download.