It takes a certain amount of chutzpah — OK, mountains of it — to invent a new kind of database and assume it will take over the world. Or maybe not assume, exactly, but, as MongoDB co-founder Eliot Horowitz put it in an interview, “If anyone was going to do it, we had just about the best chance of anyone out there.”
Not Oracle, with its decades of dominance in relational databases (RDBMS). Not IBM, with a waning database business but hordes of talented engineers. Not Microsoft, which had breathed new life into the RDBMS world with SQL Server. Not even open source upstarts MySQL and the increasingly popular PostgreSQL.
No, it was Horowitz and Dwight Merriman, two New Yorkers who wanted to put a new spin on platform-as-a-service (PaaS) but somehow, instead, built a database. “The database world is forever changed because of what we did,” said Horowitz, which might sound arrogant except for the fact that it’s true. Why it’s true, however, is worth diving deep to understand.
Horowitz recently retired from MongoDB after 13 years with the company and product, providing an opportune time to evaluate the work he did.
‘We pushed the industry’
But let’s back up a moment first. It’s easy to look at things like the DB-Engines database popularity rankings and come to the wrong conclusion. “MongoDB is the fifth most popular database, and still one-third as widely used as Oracle and MySQL!” Given how reluctant enterprises are to switch out battle-tested databases, even that level of adoption is impressive. Databases are the “stickiest” product within an organization, the least likely to change. So for MongoDB to move past databases that had been broadly adopted (DB2, Ingres, etc.) over decades, and to continue to grow in popularity relative to RDBMS stalwarts like Oracle…? That’s a big deal.
However, an even more potent indicator of MongoDB’s influence is how much these incumbents have aped the upstart.
“Every other traditional product, Postgres, MySQL, even Oracle and SQL Server, have taken a lot of the MongoDB ideas and are trying to bastardize them in their own way,” Horowitz said. “Even developers who say, ‘I would never use MongoDB! I’m just going to use Postgres because it has JSONB and all these other things.’” As Horowitz highlighted, such things as JSONB exist precisely because MongoDB pushed the industry toward adopting them. For those who “hate MongoDB yet gush about JSONB” Horowitz simply says, “You’re welcome.”
But, again, the hubris. Or chutzpah. Or whatever it was that prompted Horowitz and Merriman to push forward despite everything — everything — in the industry set up to ensure their failure. Where did that come from?
‘Databases sucked and someone needed to fix it’
Horowitz and Merriman had worked together in a few companies, DoubleClick and Shopwiki included, and, as Horowitz explained, the database kept getting in the way. Or, to be more blunt, “Using databases sucked, and someone needed to fix it and if no one else was going to do it, it might as well be me and Dwight. We knew we had a good shot. It was far from a slam dunk, but... if anyone was going to do it, we had just about the best chance of anyone out there.”
The obvious thing to do at that point would be to build a better RDBMS; to fill in the gaps left by MySQL and Postgres, both of which had grown in popularity. But this is what Horowitz did not want to do. He wanted to build a completely different approach to data, one that mapped to how developers programmed, not to some ERP system’s need for neat-and-tidy rows and columns.
The row-and-column approach to data schemas simply doesn’t resemble data as represented in application code, as Horowitz explained. In modern programming languages, the thing you want to store in the database (e.g., an order, a customer, etc.) is represented as a complete object, with all related attributes contained in a single data structure. This mismatch between developers and database administrators requires translation of that rich application structure to make it fit the rigid rules of the RDBMS. In this way, even the simplest of applications take on Frankenstein qualities in the RDBMS, requiring tens (or even thousands) of tables to capture the developer’s once-simple data modeling.
MongoDB, said Horowitz, offered developers a lifeline.
“If you take people who’ve never used a database before and you teach them MongoDB and then teach them a relational database, MongoDB is so much easier and more intuitive to them.” Yes, if you’re working with an accounting or ledger system, the RDBMS was designed for such applications, and they work well. “But for everything else, the relational model doesn’t work,” Horowitz declared.
If you’re using a programming language and a database, the thing that’s weird is that with MongoDB, in some ways, the [MongoDB] API feels much more like your programming language than the database. Therefore, it’s pretty straightforward to pick up. For many of the core concepts like indexing and queries, yes, it’s a different query language and the indexing is slightly different but, fundamentally, indexing is kind of the same. The basics of MongoDB are super easy to pick up.
Over the past 13 years, much of what his company has had to do, Horowitz said, is re-educate those who grew up on the RDBMS and need to learn a new way. But for those new to databases, “MongoDB is just so much more intuitive than other databases. It fits so much better into the way people think.” As such, Horowitz went on, “If you’re starting from scratch, you should almost always be using MongoDB, in my biased opinion.”
‘Cloud was always the vision’
Asked if he could pinpoint when he knew the MongoDB bet would pay off, Horowitz thought about it for a moment, then said, “April 2010.” That was when MongoDB hosted an event in San Francisco that sold out in under 48 hours. “People loved the core concepts, and everything became much easier.” Up until that point, Horowitz made clear, there were moments when he worried “Is this thing ever going to work?” But in April 2010, he knew the answer was a resounding “Yes.”
Which is not to say he hit cruise control. “Even then we knew it would take 10 years to add the features we wanted and that enterprises were going to need.”
One of those features was cloud.
Given MongoDB’s start as a would-be PaaS, it’s perhaps not surprising that Horowitz suggests cloud was on the roadmap from the first day. “Pretty shortly after we started building MongoDB, we also started building the MongoDB Monitoring Service, which provided the underpinnings for Atlas,” MongoDB’s database-as-a-service that now accounts for 42 percent of the company’s revenue. “Our goal was always to have a full database service.”
A big part of this ties into the company’s vision to make life easier for developers. “As developers, we knew that no one would want to manage the database themselves if they could get someone to do it for them that’s just as safe and reliable.” The full realization of this vision would have to wait, however, because even a well-funded startup like MongoDB couldn’t do everything all at once. “We had to invest all of our time and energy into getting the database right or it wouldn’t happen. That’s why we started playing with monitoring-as-a-service, just to make sure we understood how to run a cloud service at scale.” In addition, the company tinkered around the edges with things like credit card handling and support systems to “get practice at doing them so that when we were ready to launch Atlas for real, it wasn’t starting from scratch.”
Ultimately, Horowitz believes “the percentage of people running MongoDB using Atlas will be almost 100 percent,” though it’s unlikely to ever be 100 percent. With the “vast majority” of applications moving to the cloud, “There’s no reason not to use Atlas,” Horowitz maintains.
‘There’s no way you could argue we didn’t succeed’
Asked where the next MongoDB might come from, Horowitz didn’t identify a competitor so much as a guiding principle, the same one that prompted him and Merriman to build MongoDB: “You’ve got to do something fundamentally better than something else. If you came out with something that did everything that MongoDB or Postgres did, but was 10 times cheaper or 10 times faster, it would be pretty compelling.” That said, he added, “I don’t imagine how you can beat MongoDB in the data model right now.”
But what might be interesting, Horowitz posited, would be fundamentally different database architectures that can take advantage of the public cloud infrastructure to make things much cheaper. “Lots of people are working on this, but no one’s really done it. There’s nothing out there that is architected fundamentally differently.”
Which brings us back to where we started. “If you think about what we set out to do, which was make databases fundamentally easier and fundamentally better for developers, there’s no way you could argue we didn’t succeed,” Horowitz declared. “MongoDB is vastly superior to anything else that existed when we started.” Some might disagree, but few would disagree with his next statement: “The database world is forever changed because of what we did. That’s pretty awesome.”