A Blog by Jonathan Low

 

May 13, 2015

What Companies Get Wrong About Innovation Metrics

The contemporary enterprise has been engineered to become a ruthlessly efficient logistical machine. Risk is averted, uncertainty minimized and failure prevented.

All of which reduces economic friction and improves the odds of delivering consistently exceptional returns to investors.

Securing such unwaveringly happy outcomes is due, in part, to the application of ever-increasing amounts of data to the identification, dissection and solution of potential problems or the risk-adjusted embrace of opportunities.

If this sounds like a smoothly functioning model train set that goes round and round till someone flips off the switch, well, it should. That's how it's supposed to work. Investors get their accruals, managers get their bonuses and society gets an outcome whose benefits have been projected, predicted and programmed.

The only problem with all this certainty is that it has little to do with innovation. Which, statistically, tends to be messy, serendipitous, fraught with failure and  nearly impossible to measure in any meaningful way. This raises the question of what, exactly, enterprises are trying to measure and, perhaps more to the point, why.

Research conducted by my colleagues and me demonstrated that innovation rarely produces any sort of significant differentiation for which clients will pay a premium. Customers believe that innovating is the price of your staying in business and they'll be damned if they will be charged for it. As the following article explains, the time and effort spent attempting to assess the innovation process might better be focused on evaluating the world around you and figuring out what, if anything, you are doing to improve your position in it. JL

Scott Kirsner reports in Harvard Business Review:

The fear of getting Netflix-ed or Uber-ized is spurring companies to dial up their investment in innovation. The corporate impulse for dashboards and scorecards can’t be eliminated (but) there may be an inverse correlation between the intensity of a company’s obsession with measuring innovation and the breakthroughs it generates.
The fear of getting Netflix-ed or Uber-ized is spurring big companies to dial up their investment in innovation. Companies as diverse as AIG, Disney, and Intuit have been building innovation teams, launching “accelerator” programs to attract promising startups, and giving employees seed funding to test out new ideas with real customers.
But as investment increases, many companies are struggling with a challenging question: how do you know whether your chosen innovation strategy is actually bearing fruit?
Beneath that question is a very real worry. If this new crop of Chief Innovation Officers, company-bred venture capitalists, and creative catalysts can’t prove that they’re moving the needle on things that actually matter to their employer, their jobs will almost certainly evaporate.
When my publication, Innovation Leader, surveyed 198 senior innovation executives late last year, in partnership with the consulting firm Innosight, we found them using two different kinds of measures.
One type is what we call “activity” metrics, which show that you’re busy stoking the boilers of innovation. Some examples: how many employees have been trained in “lean startup” methodology, or how many new product ideas are currently being researched. Two-thirds of our respondents, for instance, said that they were tracking the number of projects in their development pipeline.
The other type of metric, an “impact” metric, shows that the ship is actually going somewhere. Market share, cost reductions, and profit margins of new businesses are examples of impact metrics. The most widely used metric among our respondents, for instance, was revenue being generated by new products or services over the course of their first few years in the market. Sixty-nine percent of respondents said they’re gathering that data.
The five most commonly used metrics were:
  1. Revenue generated by new products
  2. Number of projects in the innovation pipeline
  3. Stage-gate specific metrics, i.e. projects moving from one
    stage to the next
  4. P&L impact or other financial impact
  5. Number of ideas generated
The perfect innovation metric is elusive, and interviews with executives after the survey highlighted five ways that most measurement efforts go wrong.
Alignment can take a while. The first was the difficulty in creating a set of metrics that are aligned with what senior leadership actually cares about. It may be entry into a new geographic market, or the creation of a service revenue stream in a predominantly product-driven company. But innovation executives who had been in the role for two or more years almost universally said that they have moved away from more generic activity measures — like how many people had participated in a company crowdsourcing initiative — and toward more specific impact measures that matter to the CEO or COO.
Patience is a rarity. In businesses with billion-dollar bottom lines, it’s nearly impossible to create significant ripples in a year or two. Many of the startups now disrupting major industries like hospitality or media had four or five years to figure out what opportunity was actually worth pursuing, and build a working mechanism for attracting customers. After all, their survival hinged on figuring out how to make things click. (Netflix, for instance, got started in 1997, and introduced streaming a decade later.)
Large public companies can be more capricious. “The challenge everyone will face is patience,” says Adam Yaeger, Innovation Director at $9 billion insurance firm Assurant. “It takes a lot longer to hit that hockey stick [of growth] when you are building something from scratch,” rather than trying to layer incremental revenue onto an existing business. No matter what innovation measures a company decides to track, two or three years of data collection may not be enough to determine the real impact. Failure isnt fun to measure. Investors in new ventures acknowledge that most will fail, and corporate innovation leaders realize that most of their explorations and tests will lead to dead ends. But most established business units in a company have evolved to avoid failure, not “celebrate it,” as many Chief Innovation Officers like to exhort. Tracking dollars put into start-up partnerships that go kaput, or a new Apple Watch app that gets downloaded by two dozen people, doesn’t make you a superstar in most organizations. Core to the work of innovation, however, are edgy explorations and high-risk tests.
Hershey executive Deborah Arcoleo says she is interested in developing metrics related to what her team at the Advanced Innovation Center of Excellence has learned from its collaborations and experiments — one of which focuses on 3D printing with chocolate. “As you conduct learning experiments, the amount that you know goes up so the amount that you don’t know goes down,” Arcoleo says. “Then you feel more comfortable starting to invest a little bit more.” Putting a number on that understanding of new markets and technologies — whether it makes sense for one’s employer to enter them, and how — is difficult.
The vision thing. It can be easy to count the number of innovation workshops held at company outposts around the world, or tally revenues from add-on services developed by an innovation group and then handed over to the sales force. But white-sheet-of-paper innovation efforts are fundamentally different from growing and optimizing existing lines of business. Apple founder Steve Jobs famously didn’t believe in focus grouping new products, and when Starbucks founder Howard Schultz wanted to create a totally novel coffee-buying and drinking experience, he didn’t order up market research. The result? Products like the iPad and Starbucks’ new Reserve Roastery and Tasting Room in Seattle, a prototype for future Starbucks flagship stores.
Companies that aspire to be true innovators need to remember the role of vision and gut instinct, says Jason Berns, senior director of innovation at the athletic apparel company Under Armour. “Is there good revenue potential for what you’re working on, will it affect the business? That’s a group assessment,” Berns says. “How big could this be? Somebody needs to have a vision.” As an example, Berns brings up the company’s SpeedForm running shoes, which have interiors made using some of the same technology that produces bras, eliminating the interior seams that cause discomfort.
Measuring too much. There’s a danger that measurement sucks up resources better devoted to cultivating and testing new ideas. “A lot of large companies are highly analytical,” says Mona Vernon, vice president of the data innovation lab at Thomson Reuters, the financial media company. “They find real comfort with reports and governance. Which means that you can spend all your energy measuring stuff if you’re not careful.”
The writer E.B. White once compared attempts to analyze humor to dissecting a frog in a biology class: “Few people are interested, and the frog dies of it.” While the corporate impulse for dashboards and scorecards can’t be eliminated entirely — nor should it — there may be an inverse correlation between the intensity of a company’s obsession with measuring innovation and the breakthroughs it generates.
If they hope for their roles to endure and have an impact on their employer, corporate innovation executives need to develop a set of metrics that demonstrate a return-on-investment to senior leadership and business unit leaders. But they also need to develop the relationships — and at times fight — to have the resources and white space that allow them to incubate high-risk, potentially high-impact projects.

0 comments:

Post a Comment