CSuite Leaders Are Starting To Demand A Return On AI Investment

Hey, it's been fun! Getting in on the ground floor of the new, new thing, while impressing neighbors, relatives and the guy sitting next to you on that flight to Chicago. 

But as 2024 heads for its finale, there is a growing sense that the party is over. An increasing number of Csuite executives are demanding that the FOMO-driven experimentation window close - tightly - and that AI investments start to reveal not just operational benefits but, gasp, a path to profitability. The problem is that 70% of AI projects are in the pilot phase (eg, this might or might not work out). For AI to get greater traction, it is going to have to demonstrate that it can deliver meaningful results. Really soon. JL

Belle Lin reports in the Wall Street Journal:

While technology leaders previously had the blessing of corporate leadership to experiment with AI, “the window for experimenting is mostly behind us now. This is a year where you have to be expecting business results, a time when you should be getting benefits." The problem is, 70% of business customers’ gen AI projects are still  in pilot or testing phase. “When gen AI (first) came along, there was a certain amount of discretionary funding to test out the technology. (But) to scale beyond those experiments, we’re seeing the need to make a better business case.” 90% of gen AI experiments aren’t making it beyond the lab. “Accuracy and reliability is a big problem.”

Business technology leaders are winding down two years of fast-paced artificial intelligence experiments inside their companies, and putting their AI dollars toward proven projects focused on return on investment.

“When generative AI came along, there was a certain amount of discretionary funding that we could look at to go experiment and test out some of the technology,” said Jonny LeRoy, chief technology officer of industrial supplier W.W. Grainger. “But really to scale beyond some of those experiments, we’re seeing the need to actually make a better business case.”

While technology leaders previously had the blessing of corporate leadership to freely experiment with AI, “the window for experimenting is mostly behind us now,” Erik Brynjolfsson, co-founder of research and software company Workhelix, said Monday at The Wall Street Journal’s CIO Network Summit in New York.

 

“This is a year where you have to be expecting business results,” Brynjolfsson said, adding that the technology is mature enough to deliver them. “This is a time when you should be getting benefits, and hope that your competitors are just playing around and experimenting.” 

The problem is, roughly 70% of business customers’ generative AI projects are still stuck in pilot or testing phase, Philip Rathle, CTO of graph database software company Neo4j, said at the event. Generative AI models are good at summarizing text, for instance, but less capable of more sophisticated tasks, he added.

Naveen Rao, vice president of generative AI at cloud data firm Databricks, said that some 90% of generative AI experiments aren’t making it beyond the lab. “Accuracy and reliability is a big problem,” he said.

To get over the experimentation hump, businesses need to ensure there is widespread access to corporate data so that technology builders can use it to implement AI, said Jim Siders, CIO of data analytics company Palantir. “That’s how you get to the workable prototype that you know is fit for purpose, that you know will move the needle on the business,” he said.

 

One way to tell if the needle has moved: AI-based tools must be able to prove their worth in less than 12 months, said Nicholas Parrotta, chief digital and information officer and president of digital transformation solutions at Harman. That can be measured by either higher employee productivity or revenue generation, he said.

Another option: breaking down AI initiatives into smaller chunks that are more easily proven out, instead of approaching them as a “big, monolithic” beast, Rao said. That means getting to the point where AI projects can be treated as computer code that simply needs maintenance, he added.

 

For other CIOs, the technology’s limitations, and pressure from the business, are forcing them to take a pause and figure out where generative AI can actually produce better employee productivity or sales.

“We’re taking a little bit of time to work out what we think the savings or the benefits are, what are the next best places to go,” Grainger’s LeRoy said.

But that doesn’t mean his enthusiasm about AI has dampened. “I’m quite happy that it’s not a magic wand,” LeRoy said. “We understand that it really is a useful, powerful tool, but it fits into the broader ecosystem.”