Open Data: The Commercialization of Public Information

Census data provided the first mass application of government data to commercial uses, primarily for locating shopping malls, fast food restaurants and other real estate decisions.


Health data in cruder forms had preceded it from the 19th century on as had information used in mass conscription for militaries, but for purely private business development, the census ruled. Forty years later, there are thousands of government data bases, whose uses are beginning to be digitized and converted into truly Big Data. The issue, again, is whether the governments who paid for, collected and now provide that valuable information are being properly compensated for it. JL

Steve Lohr reports in the New York Times:

Gathering and preparing thousands of government data sets to be searched, sifted and deployed in software applications (for) analyzing business risks and fine-tuning supply chains.

The deaths of five people, including three children, in a raging fire that engulfed a home in New Orleans last November was “a terrible tragedy,” the city’s first deputy mayor, Andy Kopplin, said.

It was also preventable, he said. The house in the city’s Broadmoor neighborhood, like nearly all the homes with fire-related deaths in the city in recent years, had no smoke alarm.

Officials in New Orleans were well aware of the risks posed in homes without smoke detectors, and had a program to give them free to anyone who asked. But that clearly was not working. So after the Broadmoor fire, city officials decided to try to a new approach — targeted outreach to install smoke detectors in the homes most at risk.

To help pick the homes for the installation, they turned to a New York start-up, Enigma.io, a specialist in the field of open data, which involves collecting, curating and mining public government information for insights.

The New Orleans job was a pro bono project for Enigma, but one that demonstrated the sorts of insights that can be pulled out of open data. The young company and a handful of others like it are betting the same will increasingly prove to be true in the corporate world.

Open data, as a philosophy and a practice, has been animated by a sense of civic activism — that transparency will yield social benefits. The Obama administration endorsed the concept in 2009 when it introduced data.gov, a website providing access to federal government data sets. Many state and city governments took similar steps, as did governments around the world.

This new breed of open data companies represents the next step, pushing the applications into the commercial mainstream. Already, Enigma is working on projects with a handful of large corporations for analyzing business risks and fine-tuning supply chains — business that Enigma says generates millions of dollars in revenue.

The four-year-old company has built up gradually, gathering and preparing thousands of government data sets to be searched, sifted and deployed in software applications. But Enigma is embarking on a sizable expansion, planning to nearly double its staff to 60 people by the end of the year. The growth will be fueled by a $28.2 million round of venture funding, led by New Enterprise Associates, that will be announced on Tuesday. (The New York Times Company is among the investors.)

The expansion will be mainly to pursue corporate business. Drew Conway, co-founder of DataKind, an organization that puts together volunteer teams of data scientists for humanitarian purposes, called Enigma “a first version of the potential commercialization of public data.”

Other companies, too, are working to build businesses with the help of open data. The data start-up Reonomy, for example, provides research for the commercial real estate market. Another, Dataminr, is best known for analyzing Twitter feeds, but it also pulls in a lot of open data. Socrata, a Seattle-based start-up, supplies software for data-driven applications in government. Google’s new venture focused on technology for cities, Sidewalk Labs, announced this month, will be mining lots of open data.

Large software and services companies, like IBM and the Silicon Valley start-up Palantir, routinely cull public data in assignments for government and corporate clients. So do suppliers of analytics software, like SAS Institute and Microsoft.

But Enigma, analysts say, appears to occupy a distinctive niche, combining an unusually comprehensive data service of thousands of public data sets and software to help companies manipulate and analyze those data sets.The goal, said Marc DaCosta

Enigma’s chairman and co-founder, is “to bring open data in as tool of discovery and decision-making, integrated into the day-to-day operations of companies.”

For about a year, Enigma has been working with the pharmaceutical maker Merck to help fine-tune its supply chain and manufacturing operations. Enigma’s technology has been used to collect contract and tender information, especially from single-payer national health systems. Websites and online documents are scraped to give Merck a sharper global view of demand.

Michele D’Alessandro, vice president of information technology for Merck, said that “quality, public data is an important asset for us” and that the company had used Enigma’s data and software with “much success,” without detailing the gains.

Enigma was born of the challenges of working with open data experienced by its founders, Mr. DaCosta and Hicham Oudghiri, Enigma’s chief executive, former roommates at Columbia University. Years later, they both found themselves grappling with data-driven projects — climate mapping for Mr. DaCosta and sustainable finance for Mr. Oudghiri.

The open data movement was getting underway, creating lots of raw data in need of refinement. “We didn’t know where the value would be, but we were convinced public data would be valuable,” Mr. DaCosta recalled.

Enigma scored a recruiting coup last September, when Michael Flowers joined the start-up as its chief analytics officer. Mr. Flowers, 46, a former government prosecutor and Senate investigator, is also a star in the field of open data.

His reputation rests on his years in New York City government, under former Mayor Michael R. Bloomberg, as the city’s chief analytics officer. He oversaw a group of young quantitative analysts who used data to find stores selling bootleg cigarettes and buildings illegally converted to overcrowded firetraps, among other things.

Mr. Flowers said he was attracted to Enigma’s “nimble strike-force mentality” and its commitment to public service, like the New Orleans project, as well as to creating a successful business.

Enigma’s web home page is its “public data explorer,” a free search service of open data.

“We’re a company driven by a double bottom line,” Mr. DaCosta said. “The people who come here are people who believe in the civic, open data mission.”