ModelOps (model operations)

What is ModelOps (model operations)?

ModelOps (model operations) is a holistic approach to building analytics models that can quickly progress from the lab to production.

An important focus of ModelOps is to automate the deployment, monitoring, governance and continuous improvement of data analytics models running 24/7 within the enterprise.

ModelOps is a DevOps variation. While DevOps focuses on application development, ModelOps is designed for data analytics.

ModelOps -- which mirrors DevOps practices for ensuring regulatory compliance, security and manageability -- is crucial for predictive analytics at scale and enables the continuous delivery as well as smooth and efficient development and deployment of models.

With rapid machine learning and AI adoption, analytical assets and models are multiplying at a fast pace.

Although many organizations acknowledge the growing importance of a data-driven culture in their continuing digital transformation, applying changes based on the interpretation of data sets is challenging.

Organizations often struggle with the final stages of "implementing, operationalizing and putting analytics to work," according to Dan Vesset, group vice president of analytics and information management at IDC. As model development becomes more prevalent for solving business problems, deployment and model governance are often the last hurdle.

In fact, IDC estimated that just 35% of organizations say that they deploy models all the way to production.

How does ModelOps work?

A recent Gartner report stated that ModelOps "primarily focuses on the end-to-end governance and life cycle management of all analytics, artificial intelligence and decision models (including analytical models and models based on machine learning, knowledge graphs, rules, optimization, linguistics, agents and others)."

In a real-world application, the ModelOps team helps foster communication between data scientists or data science teams, data engineers, application owners and infrastructure owners.

They also monitor model deployment, coordinate proper handoffs and execute so that models can advance to the so-called "last mile."

Workflow automation, version management, promotions, compute resource management, monitoring, scaling and tuning are among ModelOps responsibilities.

Monitoring the effectiveness and performance of a ModelOps program is crucial. ModelOps involves development, testing, deployment and monitoring -- all of which can be effective only if model management is ensuring the scale and accuracy an organization needs.

As the models degrade, they need to be adjusted and redeployed. Performance parameters include the following:

• Set up and track accuracy goals for models through development, validation and deployment.
• Identify business metrics affected by the model in operation. Determine if the model is having the intended effects.
• Track metrics such as data size and frequency of update, locations, categories and types. These metrics can help determine if model performance problems are a result of changes in the data and its sources.
• Monitor how much computing resources or memory models consume.

Elements of a ModelOps approach

ModelOps aims to stimulate dynamic collaboration and improved productivity for analytics and IT operations teams, no matter which programming language is used, which data is accessed or where the model will be deployed.

To bring analytics from the lab into use, a ModelOps framework includes the following:

• accessing data from a trusted source and maintaining privacy and security standards;
• avoid rework by keeping a deployment scenario in mind when creating models;
• retaining data lineage and track-back information for governance and audit compliance;
• accomplishing ModelOps rapidly while ensuring close collaboration between data scientists and IT; and
• focusing on monitoring when deploying models so that analysts can monitor and retrain models as they degrade.

There are several business use cases for ModelOps, including software engineering, retail, finance, healthcare and more.

Benefits of ModelOps

ModelOps can help companies that face increasing challenges in scaling their analytics to move AI models and machine learning models from the data science lab into IT production.

By providing regular updates and deployments as these models are managed, scaled, monitored and retrained to push them into production, ModelOps provides great business value to the organization.

Benefits include:

• predictive capabilities that provide significant time and money savings;
• empowering improved decision-making and accelerated model deployment;
• correcting deviations from a defined baseline and providing real-time analytics;
• tying business key performance indicators (KPIs) to model outcomes by identifying patterns and creating insights; and
• cloud integrations to help make AI models more economical.

ModelOps best practices

McKinsey estimated that the total annual value generated by analytics and enterprise AI initiatives is between $9.5 trillion and $15.4 trillion. However, a large portion of this potential value could be lost if analytical models aren't pushed into production.

Due to inefficiencies that slow down the process, many analytics models never make it to "the last mile." Some of the technical challenges organizations face in deploying a model into production include:

• The analytics model must be compatible from the creation environment to the production environment. An agnostic scoring engine designed to take models created in any language and deploy them into production can help address the challenge of model compatibility across the model lifecycle.
• The model must be portable. Docker and other container technologies can help solve the application portability challenge by capturing the environmental dependencies for the analytic workload, providing a portable image.
• Monolithic and locked-in platforms may limit what organizations can do or offer services companies don't need. However, containerization technologies can help organizations to use native microservice software to address changing needs and limit service failures to isolated components.
• As the model progresses to production, it is typically exposed to larger volumes of data and data transport modes. The application and IT team will need tools for monitoring and solving performance and scalability challenges. Adopting a consistent, microservices-based approach to production analytics can help solve scalability challenges.