Big Data and Analytical Anarchy

Data springs from a base of prior knowledge. The key is recognizing the context - and the connection. JL

Rebecca Blumenstein interviews Hilary Mason and Andreas Weigend in the Wall Street Journal:

One of the common fallacies is that data is opposed to intuition.Data is a tool for enhancing intuition.
Big data has launched a boom industry in data analytics and science. To find out where this revolution is headed and how companies can get a competitive advantage, The Wall Street Journal’s Rebecca Blumenstein spoke with Hilary Mason, chief executive and founder of Fast Forward Labs and former chief scientist at Bitly, and Andreas Weigend, director of the Social Data Lab and former chief scientist at Amazon.com Inc. Here are edited excerpts of the discussion.

The big missteps

MS. BLUMENSTEIN: What are some of the biggest misunderstandings about data?
MS. MASON: Often people think that individual data is the most valuable thing they can collect. But it’s not useful to know what I am doing or where I am, unless you’re particularly interested in me, which is weird. But it is very useful to know what a population of people are doing.
On the implementation side, one of the common mistakes is to think of their data as a liability, as something that can only go wrong. It leads to a defensive attitude.MS. BLUMENSTEIN: There’s a lot of talk of transparency, that the CIO role has shifted from protecting data to sharing it. Andreas, you had a good example, a top executive.

Hilary Mason describes how data helped a business owner discover customers he had previously overlooked.

Andreas Weigend discusses whether there are political influences on credit scores available online in China.

MR. WEIGEND: He opened up all of his inbox and outbox for anybody within the [company] domain.
Two things happened. From one day to the next, all the bickering stopped. And those people he was thinking, “How do I get rid of them?” They left.
And people were more interested in his outbox than his inbox.
MS. BLUMENSTEIN: What companies are making data transparent internally?
MS. MASON: The big tech companies do this well. They have the infrastructure in place to collect and count whatever they like in that data, and that’s generally available widely. But I’m more interested in the companies you would not think about.
We work with an insurance company that has put a ton of time and energy into taking information about customers. It used to be that if you wanted to do something with the data and you were not on the data team, you had to fill out a paper form and send it to somebody. A week later, they might send you a report that would also be on paper.
Now you have a tool that anyone can use where they can see these metrics and these dashboards, and they can configure them a little bit themselves. If they need a more complex analysis, then it goes to that team.
This democratization of data access has allowed that organization to become much more data oriented in decision making.

 MS. BLUMENSTEIN: Andreas, you were saying that companies need to radically change their mind-set about the customer relationship.
MR. WEIGEND: The other day I called up [an airline]. It was Ms. Mary I talked to. She told me if I buy a full-price economy ticket, which was quite expensive, I get upgraded to business class. So I bought that ticket, and I show up at the check-in counter, and they seat me in economy.
I said, “They said I would get upgraded.” And they said, “Who?” “Ms. Mary.” “Well, do you want to be on this flight or not?” Ms. Mary, with my frequent-flier number, she has visibility in who I am. Me, all I know is I talked to Ms. Mary. That is a lack of symmetry. I want the history of my conversations available as part of the ticket and not selectively when the airline feels it’s in their advantage to use it.

The role of intuition

MS. BLUMENSTEIN: What are some of the mistakes that you see companies making?
MS. MASON: One of the common fallacies is that data is opposed to intuition.
Data is a tool for enhancing intuition. When I worked at the social-media company, one day the CMO of a frozen-breakfast-sausage brand came into our office. The guy said, “I want to know what may customers do on the social web.” And I said, “Great. So first we’ll figure out who your customers are.”
He said, “I know who my customers are. They’re moms in the Midwest.” I said, “How do you know?” He looked at me like I was crazy. He said, “They’re my customers. I’ve been doing this for years.”
He was not wrong. He had many customers. We didn’t know if they were moms, but they were looking at mom-type things. They were in the Midwest. But he missed a cluster of customers in Texas, and they were into motorcycles and man things. He missed a cluster in the Northwest who were anti food additives [who liked his product because it] did not have these additives.
We were able to show that his intuition was in no way wrong. But he was missing things that were too small to come up on his human radar.

ENLARGE