The Uneven Rise of Data Driven Decision Making

In most organizations, if its 'my way or the information superhighway,' the boss' way still stands a pretty good chance of prevailing. JL

Kristina McElheran and Erik Brynjolfsson report in Harvard Business Review:

As Jim Barksdale, the former CEO of Netscape quipped, “If we have data, let’s look at data. If all we have are opinions, let’s go with mine.”
Growing opportunities to collect and leverage digital information have led many managers to change how they make decisions – relying less on intuition and more on data. As Jim Barksdale, the former CEO of Netscape quipped, “If we have data, let’s look at data. If all we have are opinions, let’s go with mine.” Following pathbreakers such as Caesar’s CEO Gary Loveman – who attributes his firm’s success to the use of databases and cutting-edge analytical tools – managers at many levels are now consuming data and analytical output in unprecedented ways.
This should come as no surprise. At their most fundamental level, all organizations can be thought of as “information processors” that rely on the technologies of hierarchy, specialization, and human perception to collect, disseminate, and act on insights. Therefore, it’s only natural that technologies delivering faster, cheaper, more accurate information create opportunities to re-invent the managerial machinery.
At the same time, large corporations are not always nimble creatures. How quickly are managers actually making the investments and process changes required to embrace decision-making practices rooted in objective data? And should all firms jump on this latest managerial bandwagon?
We recently worked with a team at the U.S. Census Bureau and our colleagues Nick Bloom of Stanford and John van Reenen of the London School of Economics to design and field a large-scale survey to pursue these questions in the U.S. manufacturing sector. The survey targeted a representative group of roughly 50,000 American manufacturing establishments.
Our initial line of inquiry delves into the spread of data-driven decision making, or “DDD” for short. We find that the use of DDD in U.S. manufacturing nearly tripled between 2005 and 2010, from 11% to 30% of plants. However, adoption has been uneven. DDD is primarily concentrated in plants with four key advantages: 1) high levels of information technology, 2) educated workers, 3) greater size, and 4) better awareness.

Four factors are driving data-driven decision-making:
IT: DDD is more extensive in firms that have already made significant IT investments. Quite intuitively, firms make better use of DDD when they have more sophisticated IT to track, process, and communicate data. Likewise, they enjoy higher returns from IT when it guides decision-making and action at the firm.
College degrees: Having a larger share of workers (including both managers and non-managers) with Bachelor’s degrees also predicts the use of DDD. This may reflect the way formal education can make people more comfortable with quantitative and data-centric ways of understanding the world.
Size: Both single-plant firms, and those with multiple plants are increasing their reliance on DDD at roughly the same rate. However, single-plant establishments are still at less than half the adoption level of their bigger brethren (see Figure 1). That’s no surprise — plants that belong to larger, multi-unit firms have the advantage of being able to learn from each other and share infrastructure.
Awareness: Last but not least, even DDD-ready firms may lag behind due to a simple lack of awareness about its benefits. In order to adopt DDD, firms first have to learn about emerging practices and how they might work (or not) in their particular organization. Plants that report a larger number of opportunities to learn about new management practices – like hearing about it from other units of the same firm, from outside consultants or new employees, or from trade associations or supply chain partners – are far more likely to report being at the frontier of data-driven decision making. If you share this article with your co-workers, you might see your own firm’s use of DDD jump up a notch.

For all its benefits, DDD may not be the path to salvation for every firm. Even managers who have received the DDD gospel may oversee environments that do not permit reliable data collection. For many types of decisions, especially those for which little quantitative data exist, the broader knowledge and experience of leaders still outperforms purely data-driven approaches. Furthermore, the costs of moving to the DDD frontier are not trivial, and may outweigh the benefits – particularly if the scale of operations is just too small.
That said, the tripling of DDD rates in just five years suggests that firms are overcoming any implementation barriers quite rapidly. Our analysis sheds considerable light on what makes DDD a good fit for a wide range of firms. Yet even among plants that are, on paper, likely adopters, only a minority had adopted DDD by the end of our sample period in 2010. We expect adoption to continue to trend upward, as technology costs fall, management practices evolve, and awareness spreads.
Our follow-on research is focused on pinning down how much firms may expect to benefit from DDD, and on discovering the ingredients for success in different settings. No doubt the hype surrounding big data and analytics is great. However, our results offer objective empirical evidence that there is something beyond the hype: firms are rapidly adopting DDD and fundamentally changing how they approach management in the digital age.