MLOps aka Operational AI (Part-3)

In this article, we will see the E2E flow in operationalizing AI on popular open-source tools. The idea is to make the architecture or toolsets more cloud-agnostic.

A typical Machine Learning lifecycle is broadly categorized into 3 classes:

1. Data Processing (Data Engineer Domain)
2. Model Training (Data Scientist Domain)
3. Model Serving (Data Engineer and DevOps Domain)

Data Processing:

In this stage, the focus is on building Data Workflow Management, Data Lineage, Data Labelling, Data Quality, and Feature generation.

Model Training:

In this stage, the focus is on Model Experiment tracking, Distributed training, Metadata Management, and Hyperparameter optimization.

Model Serving:

In this stage, the focus is on Packaging, Monitoring, Model explainability, and server audit trails.

The below flow diagram is broken into:

1. Business Understanding and Data handling
2. Feature Engineering (Custom Feature Store with Versioning using Deltalake time travel feature)
3. Model Training, Evaluation, and Validation
4. Model Management (Model Registry, Experimentation, Versioning, Artifacts, and Metadata tracking)
5. Model Explainability.
6. Model Deployment and Serving (Packaging, Scaling, throughput, and Monitoring)

Diagram1 — MLOps Flow

The preferred list of tools are as below:

1. Workflow Management: Apache Airflow 2.0 / Prefect
2. Storage: Any object or block storage like HDFS, S3, or Azure Blob
3. Database: Deltalake (Cloud agnostic)
4. Data Quality: Deequ/ Great Expectation (Cloud agnostic)
5. Feature Store: Custom Build on Deltalake (Cloud agnostic)/ FEAST
6. Model Management: MLflow (Cloud agnostic)/ Weights & Biases
7. Versioning: Git, GitLab
8. CICD: Jenkins/ Github Action
9. Model Serving on Kubernetes — Seldon (Recommended)/ KFServing (Cloud agnostic)
10. Model Monitoring — Seldon Alibi Detect (Cloud agnostic)

That's it!. In the next post, we will take some deep-dive on the model training area, challenges, and how to solve through MLOps practices.

Reference:

Docker Tips, Tricks, and Tutorials

Hands-On Guide To Weights and Biases (Wandb) | With Python Implementation

Machine Learning Model Serving Overview

Getting Started - alibi-detect 0.6.2 documentation

Tech - Seldon

Introducing Airflow 2.0

Picking A Kubernetes Orchestrator: Airflow, Argo, and Prefect

Evidently AI - Open-Source Machine Learning Monitoring