Deploy a FastAPI ML Service

Your model scores 0.97 on the holdout set, and it still cannot take a single HTTP request. A model that only runs in a notebook is not a product. The gap between a working model and a deployed service is mostly infrastructure: choosing the right API framework, loading model artifacts safely, containerising the app, and making failure states visible to the orchestrator that runs it.

FastAPI is the standard choice for Python ML services because it is ASGI-native, produces OpenAPI docs automatically, and handles async I/O without the ceremony of older frameworks. This guide takes you from a raw model artifact to a containerised service with health checks the orchestrator can act on.

TLDR

Load model artifacts in a lifespan hook at startup, never at import time.
Validate request and response shapes with Pydantic so bad input fails before inference.
Build a lean image with a multi-stage Dockerfile and run as a non-root user.
Split /health (liveness) from /ready (readiness) so the orchestrator routes traffic only when the model is loaded.

What you will build#

By the end of this guide you will have a containerised FastAPI service that:

Loads a scikit-learn or embedding model at startup using a lifespan context manager
Accepts prediction requests with Pydantic-validated input schemas
Returns structured JSON responses with error handling
Exposes /health (liveness) and /ready (readiness) endpoints
Runs Uvicorn in production with the correct worker configuration
Uses a multi-stage Docker build for a minimal final image

The patterns apply to any Python model. The specific example uses a scikit-learn classifier, but the structure is identical for PyTorch models, embedding models, LLM wrappers, or RAG retrieval services.

Prerequisites#

Requirement	Why
Python 3.11+	Match the version you will use in Docker
Docker Desktop	Build and run the container locally
A saved model artifact	`.joblib`, `.pkl`, `.pt`, or similar
`pip install fastapi uvicorn[standard] scikit-learn joblib`	Core runtime dependencies

If you do not have a saved model, create a simple one for this exercise:

python

# save_model.py
from sklearn.ensemble import RandomForestClassifier
from sklearn.datasets import load_iris
import joblib

X, y = load_iris(return_X_y=True)
clf = RandomForestClassifier(n_estimators=50, random_state=42)
clf.fit(X, y)
joblib.dump(clf, "model/classifier.joblib")
print("Model saved.")

Architecture overview#

A request enters through Uvicorn, the ASGI server, which hands it to the FastAPI app. The app exposes three routes: two probes for the orchestrator and one inference endpoint. The model artifact loads once at startup through a lifespan hook and stays in memory for every prediction.

Separating liveness from readiness matters for orchestrated deployments. A liveness failure tells Kubernetes the container is dead and should be replaced. A readiness failure tells it the container is alive but not yet able to serve traffic, which is useful during model load time or warm-up.

Step-by-step implementation#

•

Create the Project Layout

A clean directory structure makes Docker layer caching work correctly and separates configuration from code.

text

ml-service/
├── app/
│   ├── __init__.py
│   ├── main.py          # FastAPI app, lifespan, routes
│   ├── schemas.py       # Pydantic request/response models
│   └── model.py         # Model loading logic
├── model/
│   └── classifier.joblib
├── Dockerfile
├── .dockerignore
└── requirements.txt

Keep the model/ directory outside app/ so the model path can be configured independently from the application code. In production, model artifacts typically live in object storage (S3, GCS), not baked into the container.

•

Define Request and Response Schemas

Pydantic schemas give you automatic input validation, clear error messages, and auto-generated OpenAPI documentation with no extra work.

python

# app/schemas.py
from pydantic import BaseModel, Field, field_validator

class PredictRequest(BaseModel):
    features: list[float] = Field(
        ...,
        min_length=4,
        max_length=4,
        description="Four iris feature values: sepal length, sepal width, petal length, petal width",
        examples=[[5.1, 3.5, 1.4, 0.2]],
    )

    @field_validator("features")
    @classmethod
    def check_positive(cls, v: list[float]) -> list[float]:
        if any(f < 0 for f in v):
            raise ValueError("All feature values must be non-negative")
        return v

class PredictResponse(BaseModel):
    label: int
    label_name: str
    confidence: float

class HealthResponse(BaseModel):
    status: str

class ReadyResponse(BaseModel):
    status: str
    model_loaded: bool

Explicit field validators catch domain-specific errors before they reach model inference. A request with a negative feature value that passes JSON parsing but fails domain validation is much easier to debug when the error comes from the schema layer.

•

Write the Model Loading Module

Separate model loading from the FastAPI app. This makes the loading logic independently testable and avoids accidental imports that trigger loading at module import time.

python

# app/model.py
import logging
import os
from pathlib import Path
import joblib
import numpy as np

logger = logging.getLogger(__name__)

LABEL_NAMES = {0: "setosa", 1: "versicolor", 2: "virginica"}

class ClassifierModel:
    def __init__(self):
        self._clf = None
        self._ready = False

    def load(self, model_path: str) -> None:
        path = Path(model_path)
        if not path.exists():
            raise FileNotFoundError(f"Model artifact not found: {path}")
        logger.info("Loading model from %s", path)
        self._clf = joblib.load(path)
        self._ready = True
        logger.info("Model loaded successfully")

    @property
    def is_ready(self) -> bool:
        return self._ready

    def predict(self, features: list[float]) -> dict:
        if not self._ready:
            raise RuntimeError("Model is not loaded")
        X = np.array(features).reshape(1, -1)
        label = int(self._clf.predict(X)[0])
        proba = float(self._clf.predict_proba(X).max())
        return {
            "label": label,
            "label_name": LABEL_NAMES[label],
            "confidence": round(proba, 4),
        }

# Module-level singleton — initialised but not loaded at import time
model = ClassifierModel()

The ClassifierModel class owns the loading state explicitly. The is_ready property drives the /ready endpoint without coupling the health check to implementation details inside FastAPI.

•

Build the FastAPI App with Lifespan

Use FastAPI's lifespan context manager to load the model at startup. The lifespan pattern replaces the deprecated @app.on_event("startup") approach (FastAPI lifespan docs).

python

# app/main.py
import logging
import os
from contextlib import asynccontextmanager

from fastapi import FastAPI, HTTPException

from .model import model
from .schemas import HealthResponse, PredictRequest, PredictResponse, ReadyResponse

logging.basicConfig(
    level=logging.INFO,
    format="%(asctime)s %(levelname)s %(name)s: %(message)s",
)
logger = logging.getLogger(__name__)

MODEL_PATH = os.environ.get("MODEL_PATH", "model/classifier.joblib")

@asynccontextmanager
async def lifespan(app: FastAPI):
    # Startup: load model artifacts
    try:
        model.load(MODEL_PATH)
    except Exception as e:
        logger.error("Failed to load model: %s", e)
        # Allow startup to complete so /health returns 200 but /ready returns false
    yield
    # Shutdown: release resources if needed
    logger.info("Shutting down service")

app = FastAPI(
    title="Iris Classifier API",
    version="1.0.0",
    lifespan=lifespan,
)

@app.get("/health", response_model=HealthResponse)
def health() -> HealthResponse:
    """Liveness probe. Returns 200 if the process is running."""
    return HealthResponse(status="ok")

@app.get("/ready", response_model=ReadyResponse)
def ready() -> ReadyResponse:
    """Readiness probe. Returns 200 only when the model is loaded."""
    if not model.is_ready:
        raise HTTPException(status_code=503, detail="Model not ready")
    return ReadyResponse(status="ok", model_loaded=True)

@app.post("/predict", response_model=PredictResponse)
def predict(request: PredictRequest) -> PredictResponse:
    """Run inference on a single feature vector."""
    if not model.is_ready:
        raise HTTPException(status_code=503, detail="Model not ready")
    try:
        result = model.predict(request.features)
        return PredictResponse(**result)
    except Exception as e:
        logger.exception("Prediction failed")
        raise HTTPException(status_code=500, detail="Prediction error") from e

MODEL_PATH from an environment variable lets you point to different model artifacts without rebuilding the image. During development it defaults to the local path; in production it can point to a mounted volume or a pre-downloaded artifact.

Do not load models at import time

Placing model = joblib.load(...) at module level means every test import, lint run, and Uvicorn worker reload triggers a full model load. Use lifespan or an explicit load() method called at startup. This makes the service faster to start in test environments and easier to mock in unit tests.

•

Write the Dockerfile

Use a two-stage build: a builder stage that installs dependencies, and a slim runtime stage that copies only what is needed to run. This keeps the final image lean and avoids shipping build tools, caches, and test dependencies into production.

dockerfile

# Dockerfile

# ── Builder ──────────────────────────────────────────────────────────────────
FROM python:3.11-slim AS builder

WORKDIR /build

COPY requirements.txt .
RUN pip install --no-cache-dir --prefix=/install -r requirements.txt

# ── Runtime ───────────────────────────────────────────────────────────────────
FROM python:3.11-slim AS runtime

# Non-root user for security
RUN useradd --create-home appuser
WORKDIR /home/appuser

# Copy installed packages from builder
COPY --from=builder /install /usr/local

# Copy application code and model artifact
COPY app/ ./app/
COPY model/ ./model/

# Do not run as root
USER appuser

ENV MODEL_PATH=model/classifier.joblib
ENV PORT=8000

EXPOSE 8000

HEALTHCHECK --interval=15s --timeout=5s --start-period=10s --retries=3 \
    CMD python -c "import urllib.request; urllib.request.urlopen('http://localhost:8000/health')"

CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000"]

The --prefix=/install trick in the builder stage copies all installed packages to a single directory, making the COPY --from=builder step precise (FastAPI Docker docs). The HEALTHCHECK instruction lets Docker itself track the container health status independently of the orchestrator.

Create a .dockerignore file to keep the build context small:

text

__pycache__
*.pyc
*.pyo
.git
.env
tests/
*.md

•

Configure for Production with Uvicorn

For local development and single-container deployments, running Uvicorn directly is sufficient:

bash

uvicorn app.main:app --host 0.0.0.0 --port 8000 --workers 1

For multi-core machines or containers with multiple CPUs, add workers:

bash

uvicorn app.main:app --host 0.0.0.0 --port 8000 --workers 4

FastAPI's official guidance is to use multiple workers with the --workers flag to take advantage of multi-core CPUs and run processes in parallel (FastAPI server workers docs). A common starting point is 2x the number of CPU cores plus 1 for CPU-bound work, or 1 worker per CPU for async-heavy services.

Workers and model memory

Each Uvicorn worker is a separate process and loads the model independently. A 500 MB model with 4 workers consumes 2 GB of RAM. On Kubernetes, run one Uvicorn worker per container and scale horizontally via replica count instead of increasing workers inside a single container.

For Gunicorn + Uvicorn workers (needed when you want Gunicorn's process management features):

bash

gunicorn app.main:app \
  -k uvicorn.workers.UvicornWorker \
  --workers 4 \
  --bind 0.0.0.0:8000 \
  --timeout 60

Test and verify the result#

A single prediction request walks through validation, the readiness check, and inference before a response goes back to the client. The sequence below is what each curl call exercises end to end.

Build and run the container locally:

bash

docker build -t ml-service:latest .
docker run -p 8000:8000 ml-service:latest

Verify each endpoint:

bash

# Liveness
curl http://localhost:8000/health
# {"status":"ok"}

# Readiness (should return 200 after lifespan startup completes)
curl http://localhost:8000/ready
# {"status":"ok","model_loaded":true}

# Prediction
curl -X POST http://localhost:8000/predict \
  -H "Content-Type: application/json" \
  -d '{"features": [5.1, 3.5, 1.4, 0.2]}'
# {"label":0,"label_name":"setosa","confidence":0.98}

# OpenAPI docs
open http://localhost:8000/docs

Check the Docker health status:

bash

docker ps
# CONTAINER ID  ...  STATUS
# abc123        ...  Up 30 seconds (healthy)

If the status shows (health: starting) for more than 10 seconds, check the --start-period in HEALTHCHECK and ensure the model loads within that window.

Common mistakes#

Loading the model at module import time. classifier = joblib.load("model.joblib") at the top of main.py means every Uvicorn worker reload, every test run, and every linter invocation loads the model. Move loading inside the lifespan hook.

No readiness probe. A liveness probe alone is not enough. If the container starts but the model has not finished loading, the orchestrator will route traffic to it and requests will fail with 500 errors. The /ready probe holds traffic back until the service is actually ready.

Hardcoded file paths. model/classifier.joblib as a hardcoded string breaks when the container is deployed with a different model. Use environment variables so paths can be changed without a rebuild.

Single large Docker layer for dependencies. Copying requirements.txt after the application code breaks Docker layer caching. Always COPY requirements.txt and pip install before COPY app/ so the dependency layer is reused between code-only rebuilds.

Not logging exceptions. An unhandled ValueError inside predict() that returns a silent 500 is nearly impossible to diagnose in production. Use logger.exception() inside the except block so the full traceback appears in your log aggregator.

Production considerations#

Model artifact delivery. Do not bake model files into the container image in most production setups. Large artifacts slow down image pulls and couple model updates to container rebuilds. Instead, download the artifact at startup from S3 or GCS using the object's version ID, and validate its checksum before loading.

python

import boto3, hashlib

def download_model(bucket: str, key: str, dest: str) -> None:
    s3 = boto3.client("s3")
    s3.download_file(bucket, key, dest)

Graceful shutdown. Uvicorn handles SIGTERM gracefully, so in-flight requests complete before the process exits. Set a --timeout-graceful-shutdown value (e.g., 30 seconds) to prevent long-running inference requests from being cut off during rolling deployments.

Observability. Add structured logging for every prediction: input shape, predicted label, confidence score, and latency. Export Prometheus metrics via prometheus-fastapi-instrumentator so dashboards show request rate, error rate, and p99 latency without manual instrumentation. See observability for AI systems for a production monitoring pattern.

Input validation at scale. Pydantic validation is synchronous and fast for simple schemas, but complex validation against a database (e.g., "is this user ID valid?") should be async and cached. Avoid blocking the event loop inside validators.

Versioning. Expose a /version endpoint that returns the model artifact version, git SHA, and build timestamp. This makes debugging production regressions dramatically faster. You can confirm which model version is running without shell access to the container.

System design. A single FastAPI ML service is a unit. For a larger system that chains retrieval, inference, and post-processing, see system design for AI products for how these services compose at the infrastructure level.

Key takeaways#

FastAPI and Uvicorn are the right defaults for Python ML serving: ASGI-native, async-capable, and well-documented. The patterns that separate a tutorial service from a production one are not exotic: lifespan model loading, split liveness/readiness probes, multi-stage Docker builds, and structured logging.

The most common failure point is model loading. Anything that loads a model at import time, ignores startup errors, or lacks a readiness probe will cause subtle traffic-routing bugs that are hard to reproduce locally. Make model state explicit, test startup failure paths, and let the orchestrator see your health status before routing requests.

FAQ#

Should I use Gunicorn or Uvicorn directly?

For simple container deployments (one process per container, horizontal scaling via replicas), Uvicorn's --workers flag is sufficient. Gunicorn adds process lifecycle management and automatic worker restart on crash, which is useful on bare-metal servers or when you cannot rely on the orchestrator to restart crashed containers. The FastAPI docs recommend Gunicorn + UvicornWorker for traditional VM deployments and Uvicorn alone on Kubernetes (FastAPI server workers docs).

How do I handle a model that takes 30+ seconds to load?

Set --start-period in HEALTHCHECK and the Kubernetes initialDelaySeconds for the readiness probe to a value larger than your worst-case load time. Keep the /health liveness probe always returning 200 so the container is not killed during startup. Track load time as a metric to detect model size regressions.

Can I serve multiple models from one FastAPI app?

Yes. Create one ClassifierModel instance per model in model.py, load all of them in the lifespan hook, and expose separate routes (e.g., /predict/sentiment, /predict/classifier). The readiness probe should check all models. Be careful with memory, since each loaded model consumes RAM for every worker process.

How do I run database migrations before the app starts?

Do not run migrations inside the FastAPI lifespan. They belong in a separate init container (Kubernetes) or a pre-start script that runs before Uvicorn starts. Mixing migration and serving in the same process creates race conditions when multiple replicas start simultaneously.

What is the difference between /health and /ready?

/health (liveness) answers: "Is this process alive?" It should always return 200 as long as the Python process is running. /ready (readiness) answers: "Can this process serve requests right now?" It returns 503 when the model has not loaded, the database is unreachable, or any other dependency is unavailable. Orchestrators use liveness to restart containers and readiness to route traffic.

TLDR

Load model artifacts in a lifespan hook at startup, never at import time.
Validate request and response shapes with Pydantic so bad input fails before inference.
Build a lean image with a multi-stage Dockerfile and run as a non-root user.
Split /health (liveness) from /ready (readiness) so the orchestrator routes traffic only when the model is loaded.

What you will build#

By the end of this guide you will have a containerised FastAPI service that:

Loads a scikit-learn or embedding model at startup using a lifespan context manager
Accepts prediction requests with Pydantic-validated input schemas
Returns structured JSON responses with error handling
Exposes /health (liveness) and /ready (readiness) endpoints
Runs Uvicorn in production with the correct worker configuration
Uses a multi-stage Docker build for a minimal final image

Prerequisites#

Requirement	Why
Python 3.11+	Match the version you will use in Docker
Docker Desktop	Build and run the container locally
A saved model artifact	`.joblib`, `.pkl`, `.pt`, or similar
`pip install fastapi uvicorn[standard] scikit-learn joblib`	Core runtime dependencies

If you do not have a saved model, create a simple one for this exercise:

python

# save_model.py
from sklearn.ensemble import RandomForestClassifier
from sklearn.datasets import load_iris
import joblib

X, y = load_iris(return_X_y=True)
clf = RandomForestClassifier(n_estimators=50, random_state=42)
clf.fit(X, y)
joblib.dump(clf, "model/classifier.joblib")
print("Model saved.")

Architecture overview#

Step-by-step implementation#

•

Create the Project Layout

A clean directory structure makes Docker layer caching work correctly and separates configuration from code.

text

ml-service/
├── app/
│   ├── __init__.py
│   ├── main.py          # FastAPI app, lifespan, routes
│   ├── schemas.py       # Pydantic request/response models
│   └── model.py         # Model loading logic
├── model/
│   └── classifier.joblib
├── Dockerfile
├── .dockerignore
└── requirements.txt

•

Define Request and Response Schemas

Pydantic schemas give you automatic input validation, clear error messages, and auto-generated OpenAPI documentation with no extra work.

python

# app/schemas.py
from pydantic import BaseModel, Field, field_validator

class PredictRequest(BaseModel):
    features: list[float] = Field(
        ...,
        min_length=4,
        max_length=4,
        description="Four iris feature values: sepal length, sepal width, petal length, petal width",
        examples=[[5.1, 3.5, 1.4, 0.2]],
    )

    @field_validator("features")
    @classmethod
    def check_positive(cls, v: list[float]) -> list[float]:
        if any(f < 0 for f in v):
            raise ValueError("All feature values must be non-negative")
        return v

class PredictResponse(BaseModel):
    label: int
    label_name: str
    confidence: float

class HealthResponse(BaseModel):
    status: str

class ReadyResponse(BaseModel):
    status: str
    model_loaded: bool

•

Write the Model Loading Module

Separate model loading from the FastAPI app. This makes the loading logic independently testable and avoids accidental imports that trigger loading at module import time.

python

# app/model.py
import logging
import os
from pathlib import Path
import joblib
import numpy as np

logger = logging.getLogger(__name__)

LABEL_NAMES = {0: "setosa", 1: "versicolor", 2: "virginica"}

class ClassifierModel:
    def __init__(self):
        self._clf = None
        self._ready = False

    def load(self, model_path: str) -> None:
        path = Path(model_path)
        if not path.exists():
            raise FileNotFoundError(f"Model artifact not found: {path}")
        logger.info("Loading model from %s", path)
        self._clf = joblib.load(path)
        self._ready = True
        logger.info("Model loaded successfully")

    @property
    def is_ready(self) -> bool:
        return self._ready

    def predict(self, features: list[float]) -> dict:
        if not self._ready:
            raise RuntimeError("Model is not loaded")
        X = np.array(features).reshape(1, -1)
        label = int(self._clf.predict(X)[0])
        proba = float(self._clf.predict_proba(X).max())
        return {
            "label": label,
            "label_name": LABEL_NAMES[label],
            "confidence": round(proba, 4),
        }

# Module-level singleton — initialised but not loaded at import time
model = ClassifierModel()

The ClassifierModel class owns the loading state explicitly. The is_ready property drives the /ready endpoint without coupling the health check to implementation details inside FastAPI.

•

Build the FastAPI App with Lifespan

Use FastAPI's lifespan context manager to load the model at startup. The lifespan pattern replaces the deprecated @app.on_event("startup") approach (FastAPI lifespan docs).

python

# app/main.py
import logging
import os
from contextlib import asynccontextmanager

from fastapi import FastAPI, HTTPException

from .model import model
from .schemas import HealthResponse, PredictRequest, PredictResponse, ReadyResponse

logging.basicConfig(
    level=logging.INFO,
    format="%(asctime)s %(levelname)s %(name)s: %(message)s",
)
logger = logging.getLogger(__name__)

MODEL_PATH = os.environ.get("MODEL_PATH", "model/classifier.joblib")

@asynccontextmanager
async def lifespan(app: FastAPI):
    # Startup: load model artifacts
    try:
        model.load(MODEL_PATH)
    except Exception as e:
        logger.error("Failed to load model: %s", e)
        # Allow startup to complete so /health returns 200 but /ready returns false
    yield
    # Shutdown: release resources if needed
    logger.info("Shutting down service")

app = FastAPI(
    title="Iris Classifier API",
    version="1.0.0",
    lifespan=lifespan,
)

@app.get("/health", response_model=HealthResponse)
def health() -> HealthResponse:
    """Liveness probe. Returns 200 if the process is running."""
    return HealthResponse(status="ok")

@app.get("/ready", response_model=ReadyResponse)
def ready() -> ReadyResponse:
    """Readiness probe. Returns 200 only when the model is loaded."""
    if not model.is_ready:
        raise HTTPException(status_code=503, detail="Model not ready")
    return ReadyResponse(status="ok", model_loaded=True)

@app.post("/predict", response_model=PredictResponse)
def predict(request: PredictRequest) -> PredictResponse:
    """Run inference on a single feature vector."""
    if not model.is_ready:
        raise HTTPException(status_code=503, detail="Model not ready")
    try:
        result = model.predict(request.features)
        return PredictResponse(**result)
    except Exception as e:
        logger.exception("Prediction failed")
        raise HTTPException(status_code=500, detail="Prediction error") from e

Do not load models at import time

•

Write the Dockerfile

dockerfile

# Dockerfile

# ── Builder ──────────────────────────────────────────────────────────────────
FROM python:3.11-slim AS builder

WORKDIR /build

COPY requirements.txt .
RUN pip install --no-cache-dir --prefix=/install -r requirements.txt

# ── Runtime ───────────────────────────────────────────────────────────────────
FROM python:3.11-slim AS runtime

# Non-root user for security
RUN useradd --create-home appuser
WORKDIR /home/appuser

# Copy installed packages from builder
COPY --from=builder /install /usr/local

# Copy application code and model artifact
COPY app/ ./app/
COPY model/ ./model/

# Do not run as root
USER appuser

ENV MODEL_PATH=model/classifier.joblib
ENV PORT=8000

EXPOSE 8000

HEALTHCHECK --interval=15s --timeout=5s --start-period=10s --retries=3 \
    CMD python -c "import urllib.request; urllib.request.urlopen('http://localhost:8000/health')"

CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000"]

Create a .dockerignore file to keep the build context small:

text

__pycache__
*.pyc
*.pyo
.git
.env
tests/
*.md

•

Configure for Production with Uvicorn

For local development and single-container deployments, running Uvicorn directly is sufficient:

bash

uvicorn app.main:app --host 0.0.0.0 --port 8000 --workers 1

For multi-core machines or containers with multiple CPUs, add workers:

bash

uvicorn app.main:app --host 0.0.0.0 --port 8000 --workers 4

Workers and model memory

For Gunicorn + Uvicorn workers (needed when you want Gunicorn's process management features):

bash

gunicorn app.main:app \
  -k uvicorn.workers.UvicornWorker \
  --workers 4 \
  --bind 0.0.0.0:8000 \
  --timeout 60

Test and verify the result#

A single prediction request walks through validation, the readiness check, and inference before a response goes back to the client. The sequence below is what each curl call exercises end to end.

Build and run the container locally:

bash

docker build -t ml-service:latest .
docker run -p 8000:8000 ml-service:latest

Verify each endpoint:

bash

# Liveness
curl http://localhost:8000/health
# {"status":"ok"}

# Readiness (should return 200 after lifespan startup completes)
curl http://localhost:8000/ready
# {"status":"ok","model_loaded":true}

# Prediction
curl -X POST http://localhost:8000/predict \
  -H "Content-Type: application/json" \
  -d '{"features": [5.1, 3.5, 1.4, 0.2]}'
# {"label":0,"label_name":"setosa","confidence":0.98}

# OpenAPI docs
open http://localhost:8000/docs

Check the Docker health status:

bash

docker ps
# CONTAINER ID  ...  STATUS
# abc123        ...  Up 30 seconds (healthy)

If the status shows (health: starting) for more than 10 seconds, check the --start-period in HEALTHCHECK and ensure the model loads within that window.

Common mistakes#

Production considerations#

python

import boto3, hashlib

def download_model(bucket: str, key: str, dest: str) -> None:
    s3 = boto3.client("s3")
    s3.download_file(bucket, key, dest)

Key takeaways#

FAQ#

Should I use Gunicorn or Uvicorn directly?

How do I handle a model that takes 30+ seconds to load?

Can I serve multiple models from one FastAPI app?

How do I run database migrations before the app starts?

What is the difference between /health and /ready?

Deploy a FastAPI ML Service

What you will build#

Prerequisites#

Architecture overview#

Step-by-step implementation#

Create the Project Layout

Define Request and Response Schemas

Write the Model Loading Module

Build the FastAPI App with Lifespan

Write the Dockerfile

Configure for Production with Uvicorn

Test and verify the result#

Common mistakes#

Production considerations#

Key takeaways#

FAQ#

Related Articles

RAG From Development to Post-Deployment Monitoring: Part 2, Production and Deployment

How to Debug Slow API Endpoints

Production AI Agents with LangGraph and Langfuse: Part 2, Tools, Human-in-the-Loop, and Traceable Loops

Building from this idea?

Deploy a FastAPI ML Service

What you will build#

Prerequisites#

Architecture overview#

Step-by-step implementation#

Create the Project Layout

Define Request and Response Schemas

Write the Model Loading Module

Build the FastAPI App with Lifespan

Write the Dockerfile

Configure for Production with Uvicorn

Test and verify the result#

Common mistakes#

Production considerations#

Key takeaways#

FAQ#

Related Articles

RAG From Development to Post-Deployment Monitoring: Part 2, Production and Deployment

How to Debug Slow API Endpoints

Production AI Agents with LangGraph and Langfuse: Part 2, Tools, Human-in-the-Loop, and Traceable Loops

Building from this idea?