Telemetry metric catalog#
Every metric name, attribute, and label value queue telemetry can emit. Turn telemetry on first with Observability.
Bounded Attributes#
Queue telemetry limits attributes and metric labels to the following known values. This prevents unbounded label counts:
messaging.systemmessaging.operation.namemessaging.destination.namemessaging.message.idon spans onlyqueue.task.namequeue.task.statusqueue.task.attemptqueue.backendqueue.operationon batch-size histograms, currentlyenqueue_manyorclaim_manyqueue.transporton wakeup, listener, and event-delivery metricsqueue.outcomeon listener and event-delivery metricsqueue.execution.backendqueue.execution.profilequeue.execution.statuson dispatch metrics, one ofdispatched,fallback,error,skipped,cancelled,ownership_lost,scheduled, oralready_existsqueue.delivery.outcomeon the delivery metric, one ofacknowledged,duplicate,retry_scheduled, ortransient_errorqueue.repair.outcomeon the repair metric, one ofpresent,recreated, orerrorqueue.stale.outcomeon stale-recovery metrics, one ofrequeued,failed,skipped, orhandler_neededqueue.expiry.outcomeon the expiry metric, currentlyexpiredworker.error.typeworker.wait.kindon worker delay and wait metrics, eithernativeorpollingqueue.worker.idon spans only, when a worker id already existsscopeon plugin-owned stream metricsreasonon stream authorization-denial metrics only
Counts never appear as label values. Stale recovery reports each outcome as its
own sample rather than encoding the tallies into labels. The
litestar_queues.expiry counter records the number of records transitioned
by a worker sweep as its sample value with the bounded expired outcome.
Each metric name has exactly one emitter. Dispatch, reconcile, and repair counters belong to the execution backend, and the heartbeat failure counter belongs to the heartbeat manager, so a single metric never arrives with two different label sets.
Queue transport metrics#
Transport instrumentation is emitted at logical ownership boundaries rather than once per record. Batch sizes are actual batch sizes, wakeup counters belong to the backend notification method, wait timing belongs to the worker, and event flush timing belongs to the event publisher:
Metric |
Labels |
Meaning |
|---|---|---|
|
|
Records accepted by one |
|
|
Wakeup hints actually sent by a backend. |
|
|
Per-record hints avoided by a coalesced notification. |
|
|
Worker cycles that found no local or externally claimed work. |
|
|
Configured delay before the next polling or native wait cycle. |
|
|
Time actually spent waiting for work. |
|
|
Time from a native notification to the following claim attempt. |
|
|
Native listener reconnection attempts after a read failure. |
|
|
Native listener failures; the current outcome is |
|
|
Actual records returned by one |
|
|
Events in one successful or failed live-delivery attempt. |
|
|
Time spent in one live-delivery attempt. |
|
|
Buffered events dropped with the bounded |
Event flush outcomes are success or failed. Transport and backend names
come from the configured built-in vocabulary; arbitrary channel names, payload
fields, task arguments, record identifiers, and exception messages never become
metric labels.
That constraint is why re-checking lost deliveries reports into
litestar_queues.execution.repair rather than joining
litestar_queues.execution.reconcile. Both describe bringing a record back in
line with its executor, but they answer different questions and so carry
different label keys, and a collector fixes its label names when the metric is
first registered. Two vocabularies on one name would make whichever backend
recorded second raise instead of counting.
Managed transports#
A backend that hands records to a transport instead of a worker emits three families:
Metric |
Outcome label |
What it tells you |
|---|---|---|
|
|
Whether a delivery was created for a record that became due. |
|
|
What each arriving delivery did. A queue whose deliveries are mostly
|
|
|
How often maintenance finds a delivery the transport lost. On a queue
nobody polls, a non-zero |
The delivery metric carries only the execution backend and its outcome. The route holds a record id, not a record, and labelling it further would mean reading storage again on the one path that otherwise needs nothing.
Unset Attributes#
Spans omit an attribute that has no value. A task with no execution profile
carries no queue.execution.profile attribute at all.
Metrics cannot do that. Prometheus binds label names when a collector is first
constructed, and a later sample with a different key set is rejected, so every
sample of a metric must carry every label. Unset labels therefore carry an empty
value, which is exactly how Prometheus encodes “not set”: a label with an empty
value is equivalent to the label being absent, and
queue_execution_profile="" matches series that never had the label.
Prometheus Names#
Prometheus collectors register with the configured registry, or the default
prometheus_client registry when none is given. Counter instruments carry no
.count suffix – the instrument type already conveys it – and names follow
Prometheus convention rather than mirroring the OpenTelemetry instrument names:
Instrument |
Exported Prometheus name |
|---|---|
|
|
|
|
|
|
|
|
Duration histograms use buckets that span sub-millisecond enqueues through
half-hour task executions. Override them with duration_buckets when your
workload needs a different resolution.
The package never uses task arguments, results, arbitrary metadata, tenant IDs, user IDs, job IDs, exception messages, or Cloud Run execution references as metric labels.
Stream Metrics#
When QueueConfig.events.stream and observability are enabled, plugin-owned
WebSocket and SSE streams report metrics through the same runtime. Stream
labels use only scope and, for denied access, reason. They never include
task IDs, queue names, tenant IDs, user IDs, exception messages, or payload
fields.
Metric |
Type |
Labels |
Meaning |
|---|---|---|---|
|
Counter |
|
Stream connections accepted by scope. |
|
Gauge / OTel UpDownCounter |
|
Active stream connections, incremented on connect and decremented on disconnect. |
|
Counter |
|
Queue events sent to stream clients. |
|
Counter |
|
Duplicate events dropped within one connection by |
|
Counter |
|
WebSocket ping frames or SSE keepalive comments sent. |
|
Counter |
|
Stream subscription denials. |
|
Histogram |
|
Stream connection lifetime in seconds. |