Machine Learning: Sensor-Rich, Data-Driven...And Still Vulnerable?

One thing's for sure: we're not going to be lacking for data anytime soon.

If there's anything we have now learned how to do, it's connecting machines to people and other machines in order to generate numbers. Some of them are signal and some of them are noise.

Differentiating between the two may depend less on the sources of the data or its nature or its structure, but on the users' ability to make sense of it in ways that are meaningful. Which is probably going to call for a lot more human input than those who are creating and selling it might like. JL

Quentin Hardy reports in the New York Times:

The answer to the future will be in knowing how to ask the right kinds of questions.

The technology world is looking at ways to make everything more efficient.

The question is, will this be good for the many, or the mighty few?

Earlier this month, General Electric announced it was selling GE Capital, its financial arm. With less fanfare, G.E. also unveiled plans for computer-connected L.E.D. streetlights, so cities can collect and analyze performance data, for lower costs and better safety.

GE Capital was a huge profit center after the financial deregulation of the 1980s, but that was then. Sensor-rich lights, to be found eventually in offices and homes, are for a company that will sell knowledge of behavior as much as physical objects.

“The next generation of bulbs have a life cycle of 20 years; we can’t think of that as a transactional business anymore,” said Bill Ruh, the head of G.E.’s software center. “We can put cameras and more sensors on these, and measure motion, heat, air quality.” Retailers might want such lights to steer shoppers, he said, while consumers could better learn about their electricity consumption.

This sensor explosion is only starting: Huawei, a Chinese maker of computing and communications equipment with $47 billion in revenue, estimates that by 2025 over 100 billion things, including smartphones, vehicles, appliances and industrial equipment, will be connected to cloud computing systems.

The Internet will be almost fused with the physical world. The way Google now looks at online clicks to figure out what ad to next put in front of you will become the way companies gain once-hidden insights into the patterns of nature and society.

G.E., Google and others expect that knowing and manipulating these patterns is the heart of a new era of global efficiency, centered on machines that learn and predict what is likely to happen next.

“The core thing Google is doing is machine learning,” Eric Schmidt, Google’s executive chairman, said at an industry event on Wednesday. Sensor-rich self-driving cars, connected thermostats or wearable computers, he said, are part of Google’s plan “to do things that are likely to be big in five to 10 years. It just seems like automation and artificial intelligence makes people more productive, and smarter.”

Not only that, but big tech companies have also decided that supplying the means for others to analyze data is going to be a big business, too. Amazon, which last week stunned Wall Street with word that its online sales of computing were a $5 billion annual business vastly more profitable than the rest of the company, has started selling data analysis tools as part of its service.

Microsoft, which over the years invested billions in machine learning, last summer offered a service others can rent on the Microsoft cloud. IBM, rushing to catch up, has put Watson, its “Jeopardy!”-winning computer, in the cloud for others to use.

What we may be seeing here is a repeat of what happened in online search, the original big digital pattern-finding business. Google won that business partly by investing heavily in almost every aspect of computer science, until only Microsoft could afford to keep up.

The great data science companies of our sensor-packed world will have experts in arcane reaches of statistics, computer science, networking, visualization and database systems, among other fields. Graduates in those areas are already in high demand.

Nor is data analysis just a question of computing skills; data access is also critically important. As a general rule, the larger and richer a data set a company has, the better its predictions become.

If G.E. gains a lot of data on how people use lighting, it builds better lighting systems next time. Google’s cars could download new software that will upgrade their performance, based on what they learn. Amazon can sell statistical tools, but it could also possess an unassailable knowledge of the world’s shopping behavior.

“Many of the techniques are based on having lots of computers and lots of data,” said Michael Chui, a partner at McKinsey Global Institute. “If you are going to work at scale, you will need lots of assets.”

Does this mean that the computing tools that were supposed to empower us all will simply make the wealthiest companies even wealthier?
Mr. Chui thinks that much analysis, like aerospace or agriculture, will also require specialist knowledge of what the critical data means, and how new patterns can be exploited. “The tools will get better, and that will raise the bar, but you will still need domain knowledge,” he said.

Jeremy Howard, a data scientist who has founded a medical data science company called Enlitic, said that an emerging area of computer analysis known as “deep learning” will blow away older fields.

While both Facebook and Google have snapped up deep-learning specialists, Mr. Howard said, “they have far too much invested in traditional computing paradigms. They are the equivalent of Kodak in photography.” Echoing Mr. Chui’s point about specialization, he said he thought the new methods demanded understanding of specific fields to work well.

It is of course possible that both things are true: Big companies like Google and Amazon will have lots of commodity data analysis, and specialists will find niches. That means for most of us, the answer to the future will be in knowing how to ask the right kinds of questions.