🤖 GitHub Actions CI Pipeline for MLOps¶

Continuous Integration (CI) requires a central server to run checks on every codebase modification. In this module, we will explore the structure of our GitHub Actions workflow file (.github/workflows/ci.yml) and understand how the pipeline orchestrates linting, testing, and containerization.

The CI/CD Runner Orchestration¶

CI/CD systems automate quality control by scheduling independent task agents (Runners) that spawn on isolated machines (Docker containers or VMs). On each trigger (such as a commit push or PR open), the runner spins up, installs dependencies, executes pipeline steps (like code checks, model quality evaluations, and test cases), and compiles the deployment assets.

graph TD
     subgraph trigger_events ["Trigger Events"]
         A["Code Push / PR to main"] -->|"GitHub Webhook"| B["Queue Job"]
     end

     subgraph github_actions_runner_vm ["GitHub Actions Runner (VM)"]
         B -->|"Allocate Runner"| C["Checkout Code"]
         C -->|"astral-sh/setup-uv"| D["Setup uv Dependency Cache"]
         D -->|"Compile Python files"| E["Syntax Verification"]

         E -->|"Pass"| F["DVC Repro: Run prep/train/eval DAG"]
         F -->|"Generates metrics.json"| G["Run Gate Check: ci_ml_guide.py"]

         G -->|"Pass"| H["Run pytest test suite"]
         H -->|"Pass"| I["docker build -t app:latest ."]

         I -->|"Pass"| J["CI Success Checkmark"]
     end

     style E fill:#fff3e0,stroke:#ffb74d,stroke-width:1px
     style G fill:#fff3e0,stroke:#ffb74d,stroke-width:1px
     style J fill:#e8f5e9,stroke:#388e3c,stroke-width:2px

📋 1. Anatomy of a GitHub Actions Workflow¶

A GitHub Actions workflow is defined in a YAML configuration file inside the .github/workflows/ directory. Here is the structure of our workflow:

Name: Defines the title of the workflow shown in the GitHub actions UI.
```
name: Continuous Integration
```
Triggers (on): Defines what events trigger the pipeline. We configure it to run on pushes and pull requests targeting the main branch:
```
on:
  push:
    branches: [ main ]
  pull_request:
    branches: [ main ]
```
Jobs & Runners (runs-on): Defines the jobs to execute and the virtual operating system they run on. We use the latest Ubuntu runner:
```
jobs:
  lint-and-test:
    runs-on: ubuntu-latest
```

🪜 2. Core CI Stages and Steps¶

Within our job, we execute a sequence of actions and commands to build and verify our MLOps system:

Step 1: Checkout & Environment Setup¶

We check out the source repository code, set up the lightning-fast Python package resolver uv, and synchronize project dependencies:

- name: Checkout Code
  uses: actions/checkout@v4

- name: Setup uv
  uses: astral-sh/setup-uv@v5
  with:
    enable-cache: true
    version: "latest"

- name: Setup Python
  uses: actions/setup-python@v5
  with:
    python-version-file: ".python-version"

- name: Install Dependencies
  run: uv sync --frozen

Step 2: Syntax and Lint Verification¶

Before running tests, we ensure that the codebase has no syntax issues:

- name: Syntax and Lint Verification
  run: uv run python -m compileall src/ tests/

Step 3: Reproduce Pipeline (DVC)¶

We generate synthetic training data and run dvc repro. This builds the end-to-end pipeline (data preparation -> training -> evaluation) and outputs our model binary (data/model.pkl) and performance metrics (data/metrics.json):

- name: Generate Raw Data and Initialize DVC
  run: |
    git config --global user.email "ci@example.com"
    git config --global user.name "CI Runner"
    uv run python src/data_experimentation/module_04_data_versioning_dvc/dvc_guide.py

- name: Execute End-to-End Pipeline
  run: uv run dvc repro

Step 4: Model Quality Gating¶

We execute the gatekeeper check from the CI/ML Quality Gates guide to ensure the newly trained model satisfies performance staging requirements:

- name: Verify Model Performance Quality Gate
  run: uv run python src/automation_observability/module_10_ci_ml_automation/ci_ml_guide.py

Step 5: Test Execution & Container Compilation¶

We run our Pytest suite (using mocked dependencies for serving APIs and decoupled MLflow file stores) and build the Docker deployment image to ensure it compiles without failures:

- name: Run Test Suite
  env:
    MLFLOW_ALLOW_FILE_STORE: "true"
  run: uv run pytest -v

- name: Build Docker Container
  run: docker build -t mlops-housing-service:latest .

In [1]:

Copied!





# Ensure we run from the project root directory
import os
import sys

# Locate project root (searching upwards for mkdocs.yml)
current_dir = os.path.dirname(os.path.abspath(__file__)) if "__file__" in locals() else os.getcwd()
while current_dir != os.path.dirname(current_dir):
    if os.path.exists(os.path.join(current_dir, "mkdocs.yml")):
        break
    current_dir = os.path.dirname(current_dir)

if os.path.exists(os.path.join(current_dir, "mkdocs.yml")):
    os.chdir(current_dir)
    if current_dir not in sys.path:
        sys.path.insert(0, current_dir)
else:
    print("⚠️ Could not find project root containing mkdocs.yml.")

# Let's verify that the CI workflow config file exists and print its header structure
workflow_path = ".github/workflows/ci.yml"
if os.path.exists(workflow_path):
    print(f"✅ GitHub Actions workflow configuration found at: {workflow_path}\n")
    with open(workflow_path, "r") as f:
        # Read first 15 lines of the YAML file
        for _ in range(15):
            line = f.readline()
            if not line:
                break
            print(line.rstrip())
else:
    print(f"❌ Workflow configuration was not found at: {workflow_path}")
# Ensure we run from the project root directory
import os
import sys

# Locate project root (searching upwards for mkdocs.yml)
current_dir = os.path.dirname(os.path.abspath(__file__)) if "__file__" in locals() else os.getcwd()
while current_dir != os.path.dirname(current_dir):
    if os.path.exists(os.path.join(current_dir, "mkdocs.yml")):
        break
    current_dir = os.path.dirname(current_dir)

if os.path.exists(os.path.join(current_dir, "mkdocs.yml")):
    os.chdir(current_dir)
    if current_dir not in sys.path:
        sys.path.insert(0, current_dir)
else:
    print("⚠️ Could not find project root containing mkdocs.yml.")

# Let's verify that the CI workflow config file exists and print its header structure
workflow_path = ".github/workflows/ci.yml"
if os.path.exists(workflow_path):
    print(f"✅ GitHub Actions workflow configuration found at: {workflow_path}\n")
    with open(workflow_path, "r") as f:
        # Read first 15 lines of the YAML file
        for _ in range(15):
            line = f.readline()
            if not line:
                break
            print(line.rstrip())
else:
    print(f"❌ Workflow configuration was not found at: {workflow_path}")

✅ GitHub Actions workflow configuration found at: .github/workflows/ci.yml

name: Continuous Integration

on:
  push:
    branches: [ main ]
  pull_request:
    branches: [ main ]

env:
  MLFLOW_ALLOW_FILE_STORE: "true"

jobs:
  lint-and-test:
    name: Lint, Test, and Container Build
    runs-on: ubuntu-latest

🖥️ 3. Unified CI Verification via CLI¶

Module 11 extends our CLI by introducing mlops ci:

Run full CI pipeline locally via CLI:
```
uv run mlops ci run
```

Run via Docker:

docker run --rm -v $(pwd):/workspace -w /workspace mlops-cli ci run

Let's verify the help information for CI operations on our unified CLI:

In [2]:

Copied!

import subprocess
result = subprocess.run(["mlops", "ci", "--help"], capture_output=True, text=True)
print(result.stdout)
import subprocess
result = subprocess.run(["mlops", "ci", "--help"], capture_output=True, text=True)
print(result.stdout)

Usage: mlops ci [OPTIONS] {run}

  Local continuous integration validation runner

Options:
  --help  Show this message and exit.

Now that we've set up continuous integration, let's step into the Pipeline Orchestration & DAGs guide to learn about Airflow and Prefect!