Mike Olson, Cloudera CSO and co-founder
So why specifically Hadoop, when so little was known about it at the time?
Look, Hadoop back then was a technology that very few people knew much about, outside of Google. Google invented this Hadoop technology back in the early 2000s, and really it was to sell more advertisements. That’s the thinking behind it.
It turned out to be transportable, to more traditional industries, to be as flexible as we need to capture all of the data that is being generated. Fast forward to today, and we’re talking to giant telecommunications companies, retailers and financial services players both here and overseas. The use cases are just amazing. It's hard to believe that data management is now sexy.
Read more: Pivotal seeks to master Big Data with upgrades to analytics suite
The explosion in data has led to a dramatic expansion in what you can do with it.
It now drives business value, as well as some pretty compelling social returns. Hospitals are now delivering better care for patients, getting better results for chronic disease at lower cost. Energy is being produced and distributed more cheaply, effectively and people are using less of it.
All of this because we can analyse data on consumption, production, we’re able to explore how disease onsets, and how it progresses, and able to reason and drive value from that data.
What do you mean by ‘social returns'?
For example, patients at paediatric clinics have only one way to express distress – crying. Nobody was capturing the sound. Simply by capturing that aural environment, including the kids cry, they can correlate that with all the rest of the data they have got. They can measure the kids' progress over time as they get treated, and they can then reason about the quality of outcome based upon the baby’s care.
The nurses in this hospital went nuts when they realised what data was available to them right now. They now have data about baby’s stress, and can work on solutions.
Yes, of course Big Data is going to help with fraud and financial services, and we’re going to optimise our supply chains, and the rest. But I don’t often get to work on problems like this – I mean, who’s not rooting for the babies in this story?
We’re going to drive social change, not just business value. And we’re not even getting started yet.
You’re referring to the Internet of Things here, right?
Yes, hospital sensors. Building sensors, their temperature gauges, power meters, their elevators - all sensors that will be connected in next five years. Traffic lights, speed cameras. That’s when Big Data is really going to happen.
So this is what you all wanted to chase at Cloudera?
The key driver for all of us [the founders] was the conviction that what had happened with Google and Facebook, namely this explosion of information driven by machines, was going to happen to banks, insurance companies, hospitals, and other traditional non-IT enterprises.
We have a very strong presence in a number of very high value verticals. Financial services, telecommunications, retail, healthcare, energy. Those are spaces where we have seen customers adopt the platform and do really innovative things.
In the old days, humans made data. For example, you buy something and a transaction record happens. Nowadays, you and I don’t have to be involved. Machines talk to machines, sensors in buildings report to machines.
Think about how much of modern stockmarket trading is done these days. The vast majority of the volume is program trading. Machines trading with machines. Capturing and analysing that volume of data from those machines…
