Peter Fader and Eric Bradlow are professors of marketing at the University of Pennsylvania’s Wharton School. They are also co-directors of the Wharton Customer Analytics Initiative, an academic research centre that focuses on the development and application of customer analytic methods and data-driven business decision-making. And they are both critical of the approach businesses are currently taking to big data. The Economist Intelligence Unit conducted a joint interview with these thought leaders on the meaning of big data, and what needs to change.
Q: Is big data a boon to business?
Peter Fader: Not at the moment. In some ways we are going in the wrong direction. Back in the old days companies like Nielsen would put together these big syndicated reports. They would look at market share, wallet share and all that good stuff. But there used to be time to digest the information between data dumps. Companies would spend time thinking about the numbers, looking at benchmarks and making thoughtful decisions. But that idea of forecasting and diagnosing is getting lost today, because the data are coming so rapidly. In some ways we are processing the data less thoughtfully.
Eric Bradlow: There does seem to be a greater separation between the IT folks that can handle these big, real-time data sets, and the managers that want to use them. There is this massive fear of throwing away even the tiniest bits of information. You see companies saving records from 500m transactions so they can analyse what will happen if they drop their price. But they don’t need to do that. All they need is a sample set. But this is part of a natural evolution. The IT data capture always comes first. Then people will figure out how to deal with these massive data sets.
Q: So what is the next step for these “data hoarders”?
EB: I think that pretty soon the costs will be prohibitive and companies will begin to change their behaviour. Even though data warehousing is getting less expensive, they will realise that they are spending huge amounts on measurement and storage engines and the return is not what they had hoped for. I also think they need to start focusing first on what decisions they need to make, thinking about what they need to know, as opposed to what it is possible to know. If you work closely with the line of business guys, they’ll tell you what they need to make good decisions.
PF: They need to make the tradeoff between volume and quality. Then they can hone in on the 3 to 12 measures they really care about and focus on collecting and analysing the patterns that emerge.
Q: What is possible today in the era of big data that was not possible before?
PF: It is the speed and granularity of the data that set this time apart. As long as you know which measures to send to which people at which time, you can actually achieve real-time interactions. And that can lead to ever-more granular data.
EB: There is a balance, however. I mean, real time is great conceptually, and hyper-targeting is great theoretically. But you cannot make an infinite variety of products. You cannot offer 10bn different services to 10bn different people. So there is a difference between what a company can know, and what it can actually do about it.