Observability (OpenTelemetry & Prometheus)¶

ObservabilityConfig integrates litestar-queues with OpenTelemetry distributed tracing and metrics, Prometheus collectors, SQLSpec correlation IDs, and Google SQLCommenter query attribution.

Runnable Configuration Example¶

"""Configuring OpenTelemetry, Prometheus, and SQLCommenter integration with ObservabilityConfig."""

from litestar_queues import QueueConfig, WorkerConfig
from litestar_queues.observability import DEFAULT_DURATION_BUCKETS, ObservabilityConfig


def build_observability_queue_config() -> QueueConfig:
    """Configure queue observability with automatic plugin detection and custom metric prefix."""
    return QueueConfig(
        queue_backend="memory",
        execution_backend="local",
        worker=WorkerConfig(placement="external"),
        observability=ObservabilityConfig(
            enable_otel=None,
            enable_prometheus=None,
            enable_sqlcommenter=True,
            metric_prefix="billing_queues",
            duration_buckets=DEFAULT_DURATION_BUCKETS,
        ),
    )


async def main() -> None:
    """Verify ObservabilityConfig policy evaluation."""
    config = build_observability_queue_config()
    assert config.observability is not None
    assert config.observability.should_enable_sqlcommenter() is True
    assert config.observability.metric_prefix == "billing_queues"

Automatic Litestar Plugin Detection¶

By default, enable_otel, enable_prometheus, and enable_sqlcommenter are set to None (auto-detect):

W3C Trace Context & Correlation ID Propagation¶

When QueueService.enqueue() or enqueue_many() persists a QueuedTaskRecord:

1. OpenTelemetry Context (_otel_context): QueueObservabilityRuntime.inject_trace_context() serializes the active W3C traceparent / tracestate headers into QueuedTaskRecord.metadata["_otel_context"] and creates a PRODUCER span (litestar_queues.enqueue).
2. Consumer Span Linking: When a worker, Cloud Tasks route, Cloud Run Job, or broker consumer executes the record, extract_trace_context() restores the parent context and starts a linked CONSUMER span (litestar_queues.process) carrying messaging.system="litestar_queues", messaging.destination.name, queue.task.name, queue.execution.backend, queue.task.attempt, and queue.task.status.
3. SQLSpec Correlation ID (_correlation_id): If SQLSpec's CorrelationContext is active during enqueue(), the correlation ID is captured into QueuedTaskRecord.metadata["_correlation_id"] and automatically rebound around the task body during execution.

Google SQLCommenter Query Attribution¶

When enable_sqlcommenter resolves to True on the SQLSpec storage backend, SQLSpecQueueBackend enables SQLCommenter annotations on its managed adapter configuration so database slow-query logs and pg_stat_statements attribute queue SQL statements to the active task and trace context. If your application supplies a shared SQLSpec instance with its own custom observability configuration, litestar-queues coexists cleanly without overwriting adopter-configured telemetry hooks.

Exported Metrics & Bounded Cardinality¶

All OpenTelemetry and Prometheus metrics strictly bound label cardinality—high-cardinality identifiers (task_id, worker_id, raw error strings, and arguments) are recorded only on trace spans and structured logs, never as metric labels.