As enterprises expand their machine learning (ML) capabilities to analyze data generated by increasingly complex applications, New Relic has updated its New Relic One full-stack observability application to include machine learning operations (MLOps) designed to help manage multiple data and ML models across different business units.
Along with application, network, infrastructure, browser monitoring, and log and error management, New Relic One is designed to allow data scientists and ML engineers to not only monitor ML model performance but also retrain models after raising alerts, said Guy Fighel, general manager of applied intelligence and group vice president of product engineering at New Relic.
Observability is a relatively new term in IT, used to describe the task of monitoring enterprise applications, data flow and distributed infrastructure. Systems that offer observability go beyond prior application performance monitoring (APM) programs, offering a high-level overview of IT infrastructure as well as granular metrics, to allow for efficient application, network, data, and security management.
According to a research report released by log-management application provider LogDNA, 75% of responding companies are still struggling to achieve true observability despite substantial investments in tools.
The study, which polled 200 senior engineering professionals across the US, showed that two-thirds of organizations currently spend $100K or more annually on observability tools, with 38% spending $300K or more annually.
MLOps aids system observability
The New Relic One update is designed to help alleviate several pain points for data scientists, chief among them the changing nature of ML or AI models, as they depend on underlying data and code that may become irrelevant as real-world conditions change.
“The ML models deteriorate over the course of time," said Andy Thurai, research vice president and principal analyst at Constellation Research. "So you need model monitoring to measure the model performance, skew, staleness/freshness of the model, model recall, model precision, and model accuracy metrics. Depending on the application and usage, the models can change in a matter of seconds or can be valid for days/weeks/years in rare cases.”
The New Relic One update allows software engineers and data scientists to either import their own data or integrate with data science platforms, as well as monitor machine learning models and interdependencies along with other application components, including infrastructure, Fighel said.
Currently, New Relic supports data science platforms such as AWS SageMaker, DataRobot, Aporia, Superwise, Comet, DAGsHub, Mona and TruEra among others.
The company said that enterprises can create custom dashboards to track accuracy of machine learning models and generate alerts for unusual changes before they have an impact on the business or customers.
Observability to break data silos, speed devops
Another problem for enterprises deploying ML applications, according to New Relic’s Fighel, is how different teams across enterprises cannot work with each other efficiently because of disparate dashboards and separate interfaces.
“There is a major gap between the model producers, AKA data scientists, versus model implementors, AKA data engineering, and devops teams. By having tools like this, a model can be productionized easily,” Thurai said.
The New Relic One platform can help bring the teams together even if the enterprise has already invested in separate data science platforms, by providing a common interface that lets data scientists and other users import data from, and view models built on, different ML platforms, Fighel said.
This capability can also help to address vendor lock-ins, Fighel said. According to the LogDNA research report, more than half of professionals surveyed said that enterprises can’t implement the tools they want because of vendor lock-in.
Pricing and availability
The new ML capabilities, which are in general availability, are being offered at no extra cost on the New Relic One platform with a 100GB per month capping. However, Fighel said that the new system will soon follow a consumption pricing model.
Some of New Relic’s competitors include companies such as Sumo Logic, AppDynamics, Dynatrace, ManageEngine and Microsoft Azure Application Insights suite.