MLOps aka Operational AI (Part-3)

  1. Data Processing (Data Engineer Domain)
  2. Model Training (Data Scientist Domain)
  3. Model Serving (Data Engineer and DevOps Domain)
  1. Business Understanding and Data handling
  2. Feature Engineering (Custom Feature Store with Versioning using Deltalake time travel feature)
  3. Model Training, Evaluation, and Validation
  4. Model Management (Model Registry, Experimentation, Versioning, Artifacts, and Metadata tracking)
  5. Model Explainability.
  6. Model Deployment and Serving (Packaging, Scaling, throughput, and Monitoring)
Diagram1 — MLOps Flow
  1. Workflow Management: Apache Airflow 2.0 / Prefect
  2. Storage: Any object or block storage like HDFS, S3, or Azure Blob
  3. Database: Deltalake (Cloud agnostic)
  4. Data Quality: Deequ/ Great Expectation (Cloud agnostic)
  5. Feature Store: Custom Build on Deltalake (Cloud agnostic)/ FEAST
  6. Model Management: MLflow (Cloud agnostic)/ Weights & Biases
  7. Versioning: Git, GitLab
  8. CICD: Jenkins/ Github Action
  9. Model Serving on Kubernetes — Seldon (Recommended)/ KFServing (Cloud agnostic)
  10. Model Monitoring — Seldon Alibi Detect (Cloud agnostic)

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
AI & Data Engineering

AI & Data Engineering

A Data Enthusiast, Lead Architect with 17 yrs experience in the field of AI Engineering, BI, Data Warehousing, Dimensional modeling, ML and Big Data