Mark Porter has an interesting background. He’s the guy who used to run AWS’s Relational Database Service (RDS) and Aurora, and prior to that he spent over a decade at Oracle. Now he’s the CTO of MongoDB. Porter’s pedigree puts him in an interesting position to comment on the evolving database landscape, given that he’s worked at three database pioneers. So what’s he saying?
That former divisions between “back office and front office [are] dissolving.” That is, systems of record and systems of engagement, once so clearly separated, are merging, in Porter’s view.
If true, what does this mean for enterprises desperately trying to modernize their data strategies? According to Porter, it’s time for enterprises to “think beyond the database, and architect an actual ‘data platform’ that can process, store, secure, and analyze data in real-time, across all the relevant data sets.” But isn’t this just a fancy new way of trying to reimagine data warehouses and data lakes?
The machines have questions
For a long time data really has been different. Back-office systems relied on structured data, nicely fitted into rows and columns. Such relational databases were an amazing innovation at the time, and they continue to serve enterprises well to this day. However, as I wrote years ago,
The comfortably structured world of the relational database is increasingly challenged by mountains of unstructured or semistructured data. Much of this new data is created by what Geoffrey Moore calls systems of engagement, even as the last several decades have been built on systems of record (such as ERP and CRM systems).
Porter goes even further, adding a third type of system, “systems of insight.” As Porter explains:
For decades, enterprises have maintained systems of record and systems of engagement. Systems of record are foundational, mission-critical, sources of truth that are accessed primarily by internal programs and users. Systems of engagement are the digital interfaces with which customers and employees interact. And recently we have seen the addition of systems of insight, which combine data from various sources to inform decision-making across the enterprise. For a long time, each system lived on different computers, had different data management requirements, and were funded by different departments.
It’s no longer the case, however, that companies can get away with staid systems that plod along in a silo, failing to interact with other data and struggling to evolve. Things are moving too fast, writes Porter, and machines are starting to ask the questions:
[W]ith the rise of model training and inference, a different kind of analytics is arriving; one where it is programs that are asking the systems of insight questions and reacting to them in real time, rather than humans asking questions and then writing programs to implement them. This is a fundamental shift; so fundamental that you could liken it to the change from the IBM 7090s that powered SABRE to those that (will?) power SKYNET.
One data platform to rule them all
So what’s an enterprise to do? For the CTO of a database company, Porter’s answer seems curious:
[T]his [convergence in systems] comes at a time when most companies are undergoing radical digital transformation projects in order to become innovation-powered, software-driven, and cloud-based. In other words, even though everyone is already quite busy, there has never been a better time to think beyond the database, and architect an actual “data platform” that can process, store, secure, and analyze data in real-time, across all the relevant data sets — either without copying the data or making such copying invisible.
How is MongoDB thinking beyond the database? For now, the company is still billing MongoDB as “The database for modern applications.” Porter says he’ll be writing about what he means over the coming months, but there are clues already strewn throughout MongoDB’s website.
For example, there’s this presentation (“Kubernetes, MongoDB, and Your MongoDB Data Platform”) that talks about how companies can build on MongoDB’s portfolio of services including the Atlas managed cloud database, a data lake, search, and application development services. This is similar to what Snowflake pitches as its “cloud data platform.” In both cases, these companies are trying to give customers the ability to integrate and query data in one place. Other companies are doing the same.
Will it work? We’ll see. The industry has been scouting for such a holy grail for some time. We used to call it a “data lake” (that didn’t end well!) and now these platforms incorporate both data lakes and data warehouses, trying to out-meta them. But just because these efforts largely failed in the past doesn’t mean that they will fail in the future. And, given Porter’s background, it’s worth following his thinking on how databases, data warehouses, and data lakes will evolve to become data platforms that truly do go beyond the database — and the marketing blather.