0 / 24

Model 41 of 74 · Technology & Data

MLOps Maturity Assessment

Assess how reliably your organisation builds, deploys, monitors, and maintains machine learning models in production.

Pillars & areas

Build

Make it repeatable

  1. 01Experimentation & Reproducibility
  2. 02Data & Feature Management

Deploy

Ship it safely

  1. 03Model Deployment & Release
  2. 04Production Monitoring

Operate

Keep it working

  1. 05Model Governance & Documentation
  2. 06Platform, Collaboration & Skills

Levels

  1. 1Notebook-BoundExperiments, not products
  2. 2Repeatable PipelinesVersioned & monitored
  3. 3Production-Grade MLReliable ML products

01 Experimentation & Reproducibility

Can model results be reproduced, compared and built upon by others?

0/4 answered

ER-1How is model development code managed?

Think about whether code lives in version control with reviews, or in personal notebooks and folders.

ER-2How are experiments tracked?

Consider whether parameters, data versions, metrics and results are recorded for each experiment.

ER-3Can your organisation reproduce a model exactly?

Think about whether you could retrain a production model from six months ago and get the same result.

ER-4How are models evaluated before they are considered ready?

Consider whether evaluation goes beyond a single accuracy figure to include business metrics, fairness, robustness and comparison with the current model.

02 Data & Feature Management

Is the data used to train and serve models versioned, validated and consistent?

0/4 answered

DF-1How is training data versioned and traced?

Think about whether you know exactly which data was used to train each model.

DF-2How is data validated before it is used for training or prediction?

Consider whether data is checked for missing values, schema changes, unexpected distributions and errors.

DF-3How consistent are features between training and production?

Think about whether features are calculated the same way when training a model and when it makes predictions live.

DF-4How are features reused across models and teams?

Consider whether teams build the same features again and again, or share them.

03 Model Deployment & Release

Can models be released to production quickly, safely and consistently?

0/4 answered

DR-1How are models deployed to production?

Think about how long it takes and how many people are involved in getting a trained model into production.

DR-2What testing happens before a model is released?

Consider tests of code, data, model performance, integration and behaviour on edge cases.

DR-3How are new model versions rolled out?

Think about whether new models replace old ones all at once, or are introduced gradually.

DR-4How quickly can a model be rolled back?

Consider what happens if a newly deployed model starts behaving badly.

04 Production Monitoring

Does the organisation know whether models in production are still performing well?

0/4 answered

PM-1How is model performance monitored in production?

Think about whether you know if a model's predictions are still accurate after it goes live.

PM-2How are changes in input data detected?

Consider whether you notice when the data a model receives starts to differ from the data it was trained on.

PM-3How are models retrained?

Think about whether retraining happens when needed, on a schedule, or only when someone remembers.

PM-4How is model performance linked to business outcomes?

Consider whether you know whether models are delivering the business value they were built for.

05 Model Governance & Documentation

Are models documented, inventoried and overseen in proportion to their risk?

0/4 answered

GD-1Does your organisation know what models it has in production?

Think about whether there is a complete inventory of models, their owners, versions and where they are used.

GD-2How are models documented?

Consider whether each model has documentation of its purpose, data, performance, limitations and intended use.

GD-3How are models reviewed and approved before use?

Think about whether higher-risk models — for example those affecting customers or financial decisions — receive more scrutiny.

GD-4How well can your organisation explain model decisions?

Consider whether you can explain why a model made a particular prediction to a customer, regulator or colleague.

06 Platform, Collaboration & Skills

Do teams have shared tools, effective collaboration and the skills to run ML in production?

0/4 answered

PC-1What platform supports ML work?

Think about whether teams share common tools for development, training, deployment and monitoring, or each builds its own.

PC-2How well do data scientists, engineers and product teams collaborate?

Consider whether models are thrown over the wall, or built by teams with shared goals.

PC-3Who owns models once they are in production?

Think about who is responsible when a model misbehaves at 2am, or when its original developer leaves.

PC-4How are ML skills and costs managed?

Consider training in MLOps practices and awareness of the cost of training and serving models.

Self-assessment report

MLOps

Answer the questions above and your report builds here: an overall score, a score and target for each area, and the three changes that matter most.

About this model

About the MLOps maturity model

Building a promising machine learning model in a notebook is the easy part. Getting it into production, keeping it accurate as the world changes, knowing what data it was trained on, being able to roll it back, and explaining its decisions months later — that is where most organisations struggle. Many models never reach production, and many that do quietly degrade until someone notices the predictions no longer make sense.

This model assesses MLOps as the set of practices that turn machine learning from one-off experiments into reliable, maintainable products. It covers reproducibility, data and feature management, deployment, monitoring, governance and the platforms and teams that make it all work. It is relevant to traditional predictive models and to systems built on large language and foundation models, and it complements the AI Readiness, AI Security and AI Governance models, which cover strategy, protection and oversight.

The MLOps maturity model contains 24 questions across 6 areas: experimentation and reproducibility, data and feature management, model deployment and release, production monitoring, model governance and documentation, and platform, collaboration and skills. Each question describes three levels of maturity through observable practices. It is designed for organisations that build, fine-tune or operate machine learning models, whether they have a handful of models or hundreds.

Who this assessment is for

  • Heads of data science, ML engineering and AI platforms
  • Data scientists and machine learning engineers
  • Data and platform engineering teams supporting ML workloads
  • Product owners of ML-powered products and features
  • Risk and model governance teams

The three maturity levels

  • Level 1 — Notebook-Bound: Machine learning work happens mainly in individual notebooks and laptops. Experiments are hard to reproduce, training data is copied around without versioning, and deploying a model is a slow, manual handover to engineers. Once a model is live, nobody systematically checks whether it is still accurate, and if the person who built it leaves, the model becomes a black box.
  • Level 2 — Repeatable Pipelines: Code, data and models are versioned, and experiments are tracked so results can be reproduced. Training and deployment follow repeatable pipelines with testing and approval steps, models in production are monitored for performance and drift, and a model registry records what is deployed where. Data scientists and engineers work together using shared tools.
  • Level 3 — Production-Grade ML: Machine learning is a dependable product capability. Teams can move from idea to safe production release quickly, every model's lineage and behaviour is traceable, and production performance is watched closely against business outcomes. Retraining and rollback are routine, governance is proportionate and built into workflows, and a shared platform lets teams focus on solving problems rather than plumbing.

The 6 areas assessed

1. Experimentation & Reproducibility

How model development work is organised. This area covers version control of code, experiment tracking, reproducibility of training runs, and the evaluation standards used to compare models.

2. Data & Feature Management

How data flows into machine learning. This area covers training data versioning and lineage, data validation, consistency between training and serving, and the reuse of features across models.

3. Model Deployment & Release

How models move into production. This area covers the deployment process, testing before release, safe rollout strategies, and the ability to roll back.

4. Production Monitoring

How models are watched once they are live. This area covers monitoring of prediction quality, data and concept drift, operational health, and the link between model performance and business outcomes, including retraining.

5. Model Governance & Documentation

How models are governed throughout their life. This area covers the model registry and inventory, model documentation, risk-based review and approval, and the explainability of model decisions.

6. Platform, Collaboration & Skills

The enabling environment for MLOps. This area covers the shared ML platform, collaboration between data science, engineering and product, ownership of models in production, and the skills and cost management of ML work.

How this model relates to other frameworks

This is an original, simplified maturity model and is not derived from or intended to replace any of the following. However, organisations that use this assessment may also find value in these established frameworks:

  • ISO/IEC 42001 (AI management systems) — a management system standard for organisations developing or using AI
  • ISO/IEC 5338 (AI system lifecycle processes) — guidance on lifecycle processes for AI systems
  • NIST AI Risk Management Framework — guidance for managing risks across the AI lifecycle
  • Cloud providers' MLOps guidance — published practices for operationalising machine learning from major cloud providers
  • Model cards and datasheets for datasets — widely used documentation practices for models and training data

Frequently asked questions

What is the MLOps maturity model?

Assess how reliably your organisation builds, deploys, monitors, and maintains machine learning models in production. It is an original, simplified model with three maturity levels (Notebook-Bound, Repeatable Pipelines and Production-Grade ML), assessed through 24 questions across 6 areas: Experimentation & Reproducibility, Data & Feature Management, Model Deployment & Release, Production Monitoring, Model Governance & Documentation and Platform, Collaboration & Skills.

How long does the MLOps assessment take?

Most people finish in 10–14 minutes. Each of the 24 questions has three concrete answer options. Your progress is saved in the page URL, so you can bookmark it and come back later.

How is the score calculated?

Each answer scores 1 (Notebook-Bound), 2 (Repeatable Pipelines) or 3 (Production-Grade ML). An area's score is the average of its answered questions, and the overall score is the average of the area scores, so every area counts equally. Scores run from 1.0 to 3.0.

Is my data stored or shared?

No. Your answers are never sent to a server. They are stored only in the page URL, which you control. Copy the link to share results with colleagues, or export a PDF or CSV report.

How should I use the results?

Start with "Where to focus" in your report. It lists the questions with the biggest gap to the target you set for each area, and shows what the next level looks like in practice. For a fuller picture, ask several people from different teams to complete the assessment and compare their links. Differences in perception are often as revealing as the scores themselves.