Skip to content

fastapi-observer

CI PyPI Python License

Zero-tracing production diagnostics and one-call observability for FastAPI. Catch N+1 SQL/HTTP loops (with the calling file, line, and ORM relationship), slow queries, DB connections held across outbound HTTP or idle work, blocked event loops, full pools, and GC pauses on every request — alongside structured JSON logs, Prometheus /metrics, OpenTelemetry traces, and default field redaction. No vendor backend: send it to whatever you already run.

from fastapi import FastAPI
from fastapiobserver import install_observability

app = FastAPI()
install_observability(app)  # pass db_engine=engine for SQL diagnostics without tracing

Every request writes one diagnostic JSON line, followed by targeted warnings pointing straight to the root cause (without logging raw SQL or secrets):

{
  "timestamp": "2026-11-20T09:10:20.345831+00:00",
  "level": "INFO",
  "logger": "fastapiobserver.middleware",
  "message": "request.completed",
  "app_name": "orders-api",
  "service": "orders",
  "environment": "production",
  "version": "2.5.0",
  "request_id": "d496f541-5e5e-4f14-9174-e9bd697faa62",
  "trace_id": "11d5b6f088bfdba501927e55463be4cd",
  "span_id": "50a12c2d9937eacb",
  "event": {
    "method": "GET",
    "path": "/orders/42",
    "http.route": "/orders/{order_id}",
    "status_code": 200,
    "duration_ms": 472.814,
    "app_duration_ms": 473.12,
    "response_completed": true,
    "client_ip": "203.0.113.7",
    "error_type": "ok",
    "diagnostics": ["db.connection_held_during_http", "db.n_plus_one", "gc.pause"],
    "gc_pause_ms": 142,
    "db": {
      "count": 13,
      "ms": 18.4,
      "sqlalchemy": {
        "attempted": 13,
        "succeeded": 13,
        "failed": 0,
        "in_flight": 0,
        "failed_ms": 0.0,
        "errors": {},
        "conn_held_ms": 312.6
      }
    },
    "http": {
      "count": 1,
      "ms": 294.1
    }
  }
}
{
  "level": "WARNING",
  "logger": "fastapiobserver.middleware",
  "message": "db.n_plus_one",
  "request_id": "d496f541-5e5e-4f14-9174-e9bd697faa62",
  "event": {
    "method": "GET",
    "route": "/orders/{order_id}",
    "statement_fingerprint": "66cbb3a40d4bbd150b75825ad291a6545399f3098fc1079e4d8b5bb061a6a481",
    "count": 12,
    "relationship": "Order.items",
    "code.file.path": "/app/orders.py",
    "code.line.number": 41,
    "code.function.name": "orders.get_order"
  }
}

Try it in 60 seconds

Watch every diagnostic fire locally in one command — no Docker or collector required:

python3 examples/full_stack/failure_lab.py --local

Or boot the three-service Docker Compose stack with Prometheus + ready-to-import alerts.yml, Loki, Tempo, and the Grafana dashboard:

docker compose -f examples/full_stack/docker-compose.yml up --build -d --wait && python3 examples/full_stack/failure_lab.py

Documentation: vitaee.github.io/FastapiObserver

Already have a JSON logger and your own redaction? Keep them:

install_observability(app, configure_logging=False)

Request diagnostics use your existing handlers, filters, formatters and levels. Your application owns log redaction and export; no logging handler or log exporter is added. See the integration guide and runnable example.

Install

pip install fastapi-observer                 # JSON logs, request IDs, redaction
pip install "fastapi-observer[prometheus]"   # + Prometheus /metrics
pip install "fastapi-observer[otel]"         # + OpenTelemetry traces and OTLP logs
pip install "fastapi-observer[all]"          # everything

Requires Python 3.11 or newer.

Configure with environment variables

install_observability(app) reads its settings from the environment: its own start with OBS_, and OpenTelemetry's standard OTEL_* variables work as usual. These are the ones most apps set:

Variable Example What it does
OBS_APP_NAME, OBS_SERVICE_NAME, OBS_ENVIRONMENT, OBS_APP_VERSION orders-api, orders, production, 2.3.1 Added to every log line
OBS_METRICS_ENABLED true Serves Prometheus metrics at /metrics (needs [prometheus])
OBS_OTEL_ENABLED, OTEL_EXPORTER_OTLP_ENDPOINT true, http://collector:4317 Sends traces over OTLP and adds trace_id to logs (needs [otel])
OBS_REDACTION_PRESET strict, pci, gdpr Chooses which fields and headers are masked
OBS_PROFILE development, production Sensible defaults: production sets INFO logs, a larger log queue, the strict preset and the event-loop watchdog

Explicit variables and Python arguments always win over profile defaults. Every setting is listed in the configuration guide.

What you get

  • One JSON line per request with method, path, sent status, response duration, total application lifetime and client IP, and your service's name, environment and version on every line, including your own logging calls. HTTP latency excludes background work; later failures still appear on the correlated request line.
  • Request IDs taken from X-Request-ID when the caller is a trusted proxy, generated otherwise, returned in the response and passed on to the services you call through instrument_httpx_client(), instrument_requests_session(), inject_trace_headers() or OpenTelemetry-instrumented clients, so one ID finds a request's logs in every service. See request ID forwarding.
  • Field-name redaction on by default. The library's outputs mask values under configured sensitive field and header names, including passwords, tokens, API keys, Authorization and cookies. URL credentials are masked in exception messages and stack traces. Arbitrary free text is not scanned for secrets or personal data; keep your own sanitizer for that. Presets (strict, pci, gdpr) select fields and masking rules, not compliance guarantees. See the redaction limits.
  • Prometheus metrics labelled by route template (/orders/{order_id}). Requests no route matches, such as scanners, CORS preflights and requests an auth middleware rejects, share one <unmatched> label, and made-up methods share _OTHER, so neither IDs nor anonymous clients explode cardinality.
  • OpenTelemetry traces, from FastAPI's own support on 0.142 and later, with trace_id and span_id on log records when trace context is available, and optional log export over OTLP using the same field-name redaction.
  • Non-blocking log writes. Lines are written from a background thread, so a slow stdout never stalls the event loop.
  • Help finding what slows a request down. A warning with the stack of any sync call that blocks the event loop, on in the production profile. A warning when the thread pool or a database connection pool is full, the usual cause when latency climbs while CPU stays low. Requests that garbage collection stopped for 100 ms or more are tagged, and so are requests that kept a database connection out of the pool while doing other work. For the SQLAlchemy engines you pass in, each request line counts its queries, and a query run once per item (N+1) gets a warning. HTTPX/HTTPX2 clients passed to instrument_httpx_client(), and Requests instrumented with instrument_requests_session(), report outbound counts, duration and failure categories too. These diagnostics work without tracing. Opt-in: DEBUG lines written only for requests that fail. With metrics on, /metrics shows how many bytes of logs each route writes (in a single process), and OBS_LOG_SAMPLE_RATE keeps a share of successful requests, chosen by trace ID, while failed and slow requests are always written in full.

Optional extras: runtime log-level control, SQLAlchemy, httpx and requests tracing, Strawberry GraphQL and a Loguru bridge. See the documentation.

Grafana dashboard built from fastapi-observer metrics, traces and logs

The Grafana dashboard from examples/full_stack, a Docker Compose stack you can run locally.

How it compares

fastapi-observer FastAPI 0.142+ built in structlog or loguru OpenTelemetry auto-instrumentation Logfire
JSON log line per request Built in No, only errors as OTLP log records You write the middleware No A span per request
Zero-tracing diagnostics N+1 SQL/HTTP (file/line/relationship), slow SQL, idle/HTTP-held DB, blocked loop, full pools, GC pauses, 422 fields No You build it Trace spans only Trace spans + profiler
Request ID on every log line Built in No You wire it up No Trace IDs
Redaction by default Field/header names and exception URL credentials; presets Signed-URL query parameters only You write a processor No Yes
Prometheus /metrics Built in Through an exporter No Through an exporter No
Traces FastAPI's own on 0.142+, OpenTelemetry's instrumentation before Yes, with spans for dependencies, the endpoint and serialization No Yes Yes
Where data goes stdout, Prometheus and any OTLP backend Any OTLP backend stdout or files Any OTLP backend Logfire's platform

Since 0.142, FastAPI traces requests on its own once an OpenTelemetry provider is set up. fastapi-observer uses that for traces and adds the rest of the first column.

When not to use it:

  • You only want logs and need the lowest possible overhead: structlog alone does less work per line.
  • You already send everything to Logfire, Datadog or Sentry: their own FastAPI integrations fit better.
  • Your app isn't built on FastAPI or Starlette.

What it costs: with default settings, about 42 µs of CPU per request on an Apple ARM64 Mac (including writing the line on a background thread), about 26 µs when the application owns logging (configure_logging=False), and about +16 µs per outbound HTTP call. See the benchmarks, including a comparison with wiring the same stack by hand.

Works with

  • Servers: uvicorn, gunicorn with UvicornWorker, and Granian. With default logging, Uvicorn's duplicate access line is dropped automatically; application-owned logging keeps its existing access-log routing. To get the servers' own startup lines as JSON too, see running with gunicorn and Granian.
  • Backends: Prometheus, Grafana (Loki, Tempo), Jaeger and anything that accepts OTLP. See backends.
  • Error tracking: runs alongside Sentry. See the Sentry notes.

Why you can trust it

  • Tested in CI on Python 3.11 to 3.15, against both the newest and the oldest dependency versions it supports, type-checked with mypy --strict, and used in production by its maintainer.
  • Releases are published with PyPI Trusted Publishing and signed attestations, and only after their tests pass. CI runs pip-audit and builds the package on every pull request.
  • Security fixes are called out in the changelog. To report a vulnerability, see SECURITY.md.
  • Benchmarks are reproducible from examples/benchmarks.

Compatibility

Component Supported
Python >=3.11 (CI: 3.11 to 3.15)
FastAPI >=0.133.0
Starlette >=1.3.1
Prometheus extra prometheus-client>=0.24.1
OpenTelemetry extra opentelemetry-api/sdk/exporter>=1.45.1

Documentation

Support the project

If this library saves you time, you can buy me a coffee.