Events#
Configuration#
Consolidated queue event configuration.
- class litestar_queues.events.config.EventDeliveryConfig(buffer: EventBufferConfig | None = <factory>, sinks: tuple[QueueEventSink, ...] = (), max_payload_bytes: int | None = None, payload_size_estimator: QueueEventSizeEstimator | None = None, strict: bool = False, publish_task_channel: bool = True, publish_queue_channel: bool = True, publish_global_lifecycle: bool = False)[source]#
Bases:
objectConfiguration for live queue event delivery.
- buffer: EventBufferConfig | None#
Optional producer buffer;
Nonedelivers every event immediately.
- sinks: tuple[QueueEventSink, ...]#
Additional live-delivery sinks, invoked in tuple order.
- payload_size_estimator: QueueEventSizeEstimator | None#
Optional encoded-size estimator used for Channels payload chunking.
- __init__(buffer: EventBufferConfig | None = <factory>, sinks: tuple[QueueEventSink, ...] = (), max_payload_bytes: int | None = None, payload_size_estimator: QueueEventSizeEstimator | None = None, strict: bool = False, publish_task_channel: bool = True, publish_queue_channel: bool = True, publish_global_lifecycle: bool = False) None#
- class litestar_queues.events.config.QueueEventsConfig(channels: ChannelsLike | None = None, delivery: EventDeliveryConfig | None = None, stream: EventStreamConfig | None = None, history: EventHistoryConfig | None = None)[source]#
Bases:
objectGroup queue event delivery, streaming, and history capabilities.
- __init__(channels: ChannelsLike | None = None, delivery: EventDeliveryConfig | None = None, stream: EventStreamConfig | None = None, history: EventHistoryConfig | None = None) None#
- delivery: EventDeliveryConfig | None#
Live delivery configuration;
Nonedisables live publishing.
- stream: EventStreamConfig | None#
Application stream endpoints;
Noneregisters no endpoints.
- history: EventHistoryConfig | None#
Backend-owned event history;
Nonedisables persistence.
Models#
Typed realtime event models for queue tasks.
- class litestar_queues.events.models.QueueEvent(*, type: str, scope: ~typing.Literal['task', 'queue', 'worker', 'global', 'custom'], id: str = <factory>, scope_key: str | None = None, task_id: str | None = None, task_name: str | None = None, queue: str | None = None, worker_id: str | None = None, execution_backend: str | None = None, execution_profile: str | None = None, attempt: int | None = None, sequence: int | None = None, level: str | None = None, message: str | None = None, progress_current: int | float | None = None, progress_total: int | float | None = None, progress_percent: float | None = None, actor: ~litestar_queues.events.models.QueueEventActor | None = None, entity: ~litestar_queues.events.models.QueueEventEntityRef | None = None, payload: dict[str, ~typing.Any] = <factory>, occurred_at: ~datetime.datetime = <factory>, schema_version: int = 1, event_key: str | None = None)[source]#
Bases:
StructStable event envelope for queue lifecycle, progress, log, and custom events.
The wire format is camelCase. Null-valued top-level fields are preserved so subscribers can rely on a stable schema for intermediate progress and log events. Payload contents are passed through verbatim.
- actor: QueueEventActor | None#
- entity: QueueEventEntityRef | None#
- to_dict() dict[str, Any][source]#
Return the stable camelCase JSON-compatible event envelope.
Null-valued top-level fields are preserved so subscribers can rely on a stable schema for intermediate progress and log events. Payload contents are passed through verbatim.
- class litestar_queues.events.models.QueueEventActor(*, type: str | None = None, id: str | None = None, name: str | None = None)[source]#
Bases:
StructActor reference for a queue event.
Publisher#
Queue event publisher.
- class litestar_queues.events.publisher.EventBufferConfig(batch_size: int = 20, flush_interval: float = 0.5, max_pending: int = 2000, overflow: Literal['drop_oldest', 'drop_newest', 'block', 'error'] = 'drop_oldest')[source]#
Bases:
objectProducer-side micro-batch buffer for live event delivery.
- class litestar_queues.events.publisher.QueueEventPublisher(sink: QueueEventSink | None = None, *, event_log: _QueueEventHistoryWriter | None = None, event_log_strict: bool = False, buffer_config: EventBufferConfig | None = None, strict: bool = False, publish_task_channel: bool = True, publish_queue_channel: bool = True, publish_global_lifecycle: bool = False, namespace: QueueNamespace | str | None = None, observability_runtime: QueueObservabilityRuntimeProtocol | None = None, transport: str | None = None)[source]#
Bases:
objectPublish queue events through a configured sink.
- __init__(sink: QueueEventSink | None = None, *, event_log: _QueueEventHistoryWriter | None = None, event_log_strict: bool = False, buffer_config: EventBufferConfig | None = None, strict: bool = False, publish_task_channel: bool = True, publish_queue_channel: bool = True, publish_global_lifecycle: bool = False, namespace: QueueNamespace | str | None = None, observability_runtime: QueueObservabilityRuntimeProtocol | None = None, transport: str | None = None) None[source]#
- strict#
- publish_task_channel#
- publish_queue_channel#
- publish_global_lifecycle#
- property sink: QueueEventSink#
Configured event sink.
- set_event_log(event_log: _QueueEventHistoryWriter, *, strict: bool = False) None[source]#
Attach backend-owned durable event history to this publisher.
- set_observability_runtime(runtime: QueueObservabilityRuntimeProtocol) None[source]#
Attach the service-owned runtime used for live delivery metrics.
Buffering and Chunking#
Producer-side live event buffering.
- class litestar_queues.events.buffer.LiveEventBuffer(config: EventBufferConfig, *, sink_publish: SinkPublish, record_drop: RecordDrop, runtime_logger: logging.Logger | None = None)[source]#
Bases:
objectBounded producer-side buffer for live queue event delivery.
- __init__(config: EventBufferConfig, *, sink_publish: SinkPublish, record_drop: RecordDrop, runtime_logger: logging.Logger | None = None) None[source]#
- async add(event: QueueEvent, channels: Sequence[str]) None[source]#
Add an event to the buffer, applying configured overflow behavior.
- litestar_queues.events.buffer.event_buffer_key(event: QueueEvent) EventBufferKey[source]#
Return the buffer key used for scoped flushes.
Live queue-event transport sizing helpers.
- litestar_queues.events.chunking.estimate_event_payload_bytes(event: QueueEvent) int[source]#
Return the direct JSON payload size for one queue event.
- litestar_queues.events.chunking.split_event_batch_by_size(event: ~litestar_queues.events.models.QueueEvent, *, max_bytes: int, size_estimator: ~collections.abc.Callable[[QueueEvent], int] = <function estimate_event_payload_bytes>) tuple[QueueEvent, ...][source]#
Split package-owned batch events into complete QueueEvent payloads.
- Returns:
The original event when no split is needed, otherwise complete event payloads that each fit inside the configured limit.
Task Context#
Task execution context and helper APIs for queue event publishing.
- class litestar_queues.events.context.TaskBeatSink(*args, **kwargs)[source]#
Bases:
ProtocolReceives last-value-wins beat progress for a running task.
- record_beat(task_id: str, detail: str | None) None[source]#
Record the latest beat detail reported by
task_id.
- __init__(*args, **kwargs)#
- class litestar_queues.events.context.TaskExecutionContext(task_id: str, task_name: str, queue: str, worker_id: str | None, execution_backend: str, execution_profile: str | None, attempt: int, event_publisher: QueueEventPublisher, scope_key: str | None = None, actor: QueueEventActor | None = None, entity: QueueEventEntityRef | None = None)[source]#
Bases:
objectContext bound while a queue task is executing.
- event_publisher: QueueEventPublisher#
- actor: QueueEventActor | None#
- entity: QueueEventEntityRef | None#
- async progress(*, current: float | None = None, total: float | None = None, percent: float | None = None, message: str | None = None, payload: dict[str, Any] | None = None, channels: Sequence[str] | None = None, scope_key: str | None = None, actor: QueueEventActor | None = None, entity: QueueEventEntityRef | None = None, immediate: bool = False) None[source]#
Publish a task progress event.
- async log(message: str, *, level: str = 'info', payload: dict[str, Any] | None = None, channels: Sequence[str] | None = None, scope_key: str | None = None, actor: QueueEventActor | None = None, entity: QueueEventEntityRef | None = None, immediate: bool = False) None[source]#
Publish a task log event.
- async event(event_type: str, *, message: str | None = None, payload: dict[str, Any] | None = None, channels: Sequence[str] | None = None, scope_key: str | None = None, actor: QueueEventActor | None = None, entity: QueueEventEntityRef | None = None, immediate: bool = False) None[source]#
Publish a custom task event.
- async lifecycle(event_type: str, *, message: str | None = None, payload: dict[str, Any] | None = None) None[source]#
Publish a worker-owned lifecycle event.
- beat(detail: str | None = None) None[source]#
Record last-value-wins progress for the next heartbeat tick.
- async publish(event_type: str, *, level: str | None = None, message: str | None = None, progress_current: float | None = None, progress_total: float | None = None, progress_percent: float | None = None, payload: dict[str, Any] | None = None, channels: Sequence[str] | None = None, scope: QueueEventScope = 'task', scope_key: str | None = None, actor: QueueEventActor | None = None, entity: QueueEventEntityRef | None = None, immediate: bool = False) QueueEvent[source]#
Build and publish an event for this task context.
- Returns:
The published queue event.
- litestar_queues.events.context.beat(detail: str | None = None) None[source]#
Record progress through the currently bound task context, if any.
- litestar_queues.events.context.bind_beat_sink(sink: TaskBeatSink) Iterator[TaskBeatSink][source]#
Bind
sinkto receiveTaskExecutionContext.beat()calls.- Yields:
The bound beat sink.
- litestar_queues.events.context.bind_task_context(context: TaskExecutionContext) Iterator[TaskExecutionContext][source]#
Bind
contextas the current task execution context.This is the supported entry point for external runtimes adopting the events subpackage standalone. While bound,
require_current_task_context()and the module-level publish helpers resolve tocontext.- Yields:
The bound task execution context.
- litestar_queues.events.context.get_current_task_context() TaskExecutionContext | None[source]#
Return the task execution context for the current async context.
- async litestar_queues.events.context.publish_task_event(event_type: str, *, message: str | None = None, payload: dict[str, Any] | None = None, channels: Sequence[str] | None = None, scope_key: str | None = None, actor: QueueEventActor | None = None, entity: QueueEventEntityRef | None = None, immediate: bool = False) None[source]#
Publish a custom event through the currently bound task context.
- async litestar_queues.events.context.publish_task_log(message: str, *, level: str = 'info', payload: dict[str, Any] | None = None, channels: Sequence[str] | None = None, scope_key: str | None = None, actor: QueueEventActor | None = None, entity: QueueEventEntityRef | None = None, immediate: bool = False) None[source]#
Publish a log event through the currently bound task context.
- async litestar_queues.events.context.publish_task_progress(*, current: float | None = None, total: float | None = None, percent: float | None = None, message: str | None = None, payload: dict[str, Any] | None = None, channels: Sequence[str] | None = None, scope_key: str | None = None, actor: QueueEventActor | None = None, entity: QueueEventEntityRef | None = None, immediate: bool = False) None[source]#
Publish progress through the currently bound task context.
- litestar_queues.events.context.require_current_task_context() TaskExecutionContext[source]#
Return the current task context or raise if none is bound.
- Raises:
RuntimeError – If no task context is bound.
Channels#
Channel naming helpers for queue events.
- class litestar_queues.events.channels.QueueChannels[source]#
Bases:
objectCanonical channel name factories for queue event scopes.
- classmethod task(task_id: str, *, topic: str = 'events', namespace: QueueNamespace | str | None = None) str[source]#
Return the channel for task-scoped events.
- classmethod queue(queue: str, *, topic: str = 'events', namespace: QueueNamespace | str | None = None) str[source]#
Return the channel for queue-scoped events.
- classmethod worker(worker_id: str, *, topic: str = 'events', namespace: QueueNamespace | str | None = None) str[source]#
Return the channel for worker-scoped events.
Sinks#
Queue event sink protocols and core implementations.
- class litestar_queues.events.sinks.CompositeQueueEventSink(sinks: Sequence[QueueEventSink], *, strict: bool = False, namespace: QueueNamespace | str | None = None)[source]#
Bases:
objectDeliver events to multiple sinks in deterministic order.
- __init__(sinks: Sequence[QueueEventSink], *, strict: bool = False, namespace: QueueNamespace | str | None = None) None[source]#
- property sinks: tuple[QueueEventSink, ...]#
Configured sinks in delivery order.
- async publish(event: QueueEvent, *, channels: Sequence[str]) None[source]#
Publish to every sink, continuing after non-strict failures.
- class litestar_queues.events.sinks.InMemoryQueueEventSink[source]#
Bases:
objectIn-process event sink for tests, examples, and local demos.
- property events: list[QueueEvent]#
Published events in publish order.
- events_for(channel: str) list[QueueEvent][source]#
Return events published to a channel.
- async publish(event: QueueEvent, *, channels: Sequence[str]) None[source]#
Store an event in process.
- class litestar_queues.events.sinks.NoopQueueEventSink[source]#
Bases:
objectEvent sink that accepts events and drops them.
- async publish(event: QueueEvent, *, channels: Sequence[str]) None[source]#
Drop an event publish.
- class litestar_queues.events.sinks.QueueEventSink(*args, **kwargs)[source]#
Bases:
ProtocolTransport boundary for queue event delivery.
- async publish(event: QueueEvent, *, channels: Sequence[str]) None[source]#
Publish an event to the requested channels.
- __init__(*args, **kwargs)#
- async litestar_queues.events.sinks.default_publish_many(sink: QueueEventSink, batch: Sequence[tuple[QueueEvent, Sequence[str]]]) None[source]#
Publish a batch by looping over a sink’s single-event publish method.
Event History#
Backend-owned queue event history contracts.
- litestar_queues.events.history.RESERVED_EVENT_HISTORY_COLUMNS = frozenset({'actor', 'entity', 'scope', 'scope_key'})#
Names held for built-in event-history scoping dimensions.
These are not columns on the table yet. They are reserved so an adopter-declared extra column cannot claim a name the package intends to own.
- class litestar_queues.events.history.EventHistoryConfig(batch_size: int = 20, flush_interval: float = 1.0, strict: bool = False, memory_capacity: int = 1000, extra_columns: tuple[~litestar_queues.events.history.EventHistoryExtraColumn, ...] = <factory>)[source]#
Bases:
objectConfiguration for backend-managed queue event history.
- extra_columns: tuple[EventHistoryExtraColumn, ...]#
Adopter-declared scoping columns on the event-history table.
- class litestar_queues.events.history.EventHistoryExtraColumn(name: str, source: str, indexed: bool = False)[source]#
Bases:
objectAdopter-declared scoping column on the SQLSpec event-history table.
- class litestar_queues.events.history.QueueEventLog(*args, **kwargs)[source]#
Bases:
ProtocolBackend-owned queue event history writer and query interface.
- __init__(*args, **kwargs)#
- async publish_event(event: QueueEvent) None[source]#
- async query_events(query: QueueEventQuery | None = None, *, extra: Mapping[str, str] | None = None) OffsetPagination[QueueEventLogRecord][source]#
- async summarize_stages(query: QueueEventQuery | None = None) list[QueueEventStageSummary][source]#
- class litestar_queues.events.history.QueueEventLogRecord(event_id: str, event_type: str, task_id: str | None, task_name: str | None, queue: str | None, worker_id: str | None, execution_backend: str | None, execution_profile: str | None, actor_type: str | None, actor_id: str | None, stage: str | None, level: str | None, message: str | None, detail: dict[str, Any], progress_current: float | None, progress_total: float | None, progress_percent: float | None, duration_ms: float | None, sequence: int | None, occurred_at: datetime, created_at: datetime, scope: str | None = None, scope_key: str | None = None, actor: str | None = None, entity: str | None = None, extra: dict[str, str] = <factory>)[source]#
Bases:
objectA durable queue event history record.
- occurred_at: datetime#
- created_at: datetime#
- __init__(event_id: str, event_type: str, task_id: str | None, task_name: str | None, queue: str | None, worker_id: str | None, execution_backend: str | None, execution_profile: str | None, actor_type: str | None, actor_id: str | None, stage: str | None, level: str | None, message: str | None, detail: dict[str, Any], progress_current: float | None, progress_total: float | None, progress_percent: float | None, duration_ms: float | None, sequence: int | None, occurred_at: datetime, created_at: datetime, scope: str | None = None, scope_key: str | None = None, actor: str | None = None, entity: str | None = None, extra: dict[str, str] = <factory>) None#
- class litestar_queues.events.history.QueueEventStageSummary(stage: str | None, event_count: int, total_duration_ms: float, first_event_at: datetime | None, last_event_at: datetime | None, latest_sequence: int | None = None, latest_message: str | None = None, worst_level: str | None = None)[source]#
Bases:
objectAggregated queue event history data for a single stage.
- litestar_queues.events.history.extract_event_extras(payload: Mapping[str, Any] | None, declared_columns: Sequence[EventHistoryExtraColumn]) dict[str, str][source]#
Extract declared extra columns from an event payload dict.
- Returns:
Mapping of physical column names to extracted string values.
- litestar_queues.events.history.validate_event_extra_filter(filter_map: Mapping[str, str] | None, declared_columns: Sequence[EventHistoryExtraColumn]) dict[str, str][source]#
Validate and resolve extra column filter key-value pairs against declared columns.
- Returns:
Mapping of resolved physical column names to expected filter values.
- Raises:
QueueConfigurationError – If any filter key is not declared in declared_columns.
- litestar_queues.events.history.validate_event_history_extra_columns(columns: Sequence[EventHistoryExtraColumn]) tuple[EventHistoryExtraColumn, ...][source]#
Validate adopter-declared extra event-history columns.
- Returns:
The validated declarations as a tuple.
- Raises:
QueueConfigurationError – If a name is not a valid unquoted SQL identifier, collides with a package-owned column, uses a reserved scoping-dimension name, repeats another declaration, or the payload source key is empty.
External Producers#
Producer facade for queue event publishing.
- class litestar_queues.events.producer.QueueEventProducer(publisher: QueueEventPublisher)[source]#
Bases:
objectThin facade over a queue event publisher.
- __init__(publisher: QueueEventPublisher) None[source]#
- litestar_queues.events.producer.create_event_producer(config: QueueConfig) _ExternalProducer[source]#
Return an external producer context manager for queue event publishing.
Litestar Channels#
Litestar Channels event sink for queue events.
- class litestar_queues.events.channels_sink.ChannelsQueueEventSink(channels_backend: ChannelsLike, *, manage_lifecycle: bool = False, max_payload_bytes: int | None = None, payload_size_estimator: QueueEventSizeEstimator | None = None)[source]#
Bases:
objectEvent sink that publishes to an app-owned Litestar Channels object.
- __init__(channels_backend: ChannelsLike, *, manage_lifecycle: bool = False, max_payload_bytes: int | None = None, payload_size_estimator: QueueEventSizeEstimator | None = None) None[source]#
- property channels_backend: ChannelsLike#
Wrapped Channels backend or plugin.
- async publish(event: QueueEvent, *, channels: Sequence[str]) None[source]#
Publish an event to Litestar Channels.
Stream Configuration and Routes#
Configuration for plugin-owned queue event streaming endpoints.
- litestar_queues.events.stream_config.ChannelAuthorizer#
alias of
Callable[[Any,Literal[‘task’, ‘queue’, ‘worker’, ‘global’, ‘custom’],str|None],bool|Awaitable[bool]]
- class litestar_queues.events.stream_config.EventStreamConfig(transports: set[~typing.Literal['sse', 'websocket']] = <factory>, path: str = <object object>, guards: list[~collections.abc.Callable[[~typing.Any, ~typing.Any], ~typing.Any]] | None = None, channel_authorizer: ~collections.abc.Callable[[~typing.Any, ~typing.Literal['task', 'queue', 'worker', 'global', 'custom'], str | None], bool | ~collections.abc.Awaitable[bool]] | None = None, unauthenticated_access: ~typing.Literal['warn', 'allow', 'error'] = 'warn', scopes: set[~typing.Literal['task', 'queue', 'worker', 'global', 'custom']] = <factory>, heartbeat_interval: float = 25.0, replay_limit: int = 0, include_in_schema: bool = False, opt: dict[str, ~typing.Any] | None = None)[source]#
Bases:
objectConfiguration for plugin-registered WebSocket queue-event streaming.
- guards: list[Callable[[Any, Any], Any]] | None#
Litestar route guards applied to stream endpoints;
Noneadds none.
- channel_authorizer: Callable[[Any, Literal['task', 'queue', 'worker', 'global', 'custom'], str | None], bool | Awaitable[bool]] | None#
Per-subscription authorizer;
Noneleaves channel selection unrestricted.
- unauthenticated_access: Literal['warn', 'allow', 'error']#
Policy used when stream endpoints have neither guards nor an authorizer.
- __init__(transports: set[~typing.Literal['sse', 'websocket']] = <factory>, path: str = <object object>, guards: list[~collections.abc.Callable[[~typing.Any, ~typing.Any], ~typing.Any]] | None = None, channel_authorizer: ~collections.abc.Callable[[~typing.Any, ~typing.Literal['task', 'queue', 'worker', 'global', 'custom'], str | None], bool | ~collections.abc.Awaitable[bool]] | None = None, unauthenticated_access: ~typing.Literal['warn', 'allow', 'error'] = 'warn', scopes: set[~typing.Literal['task', 'queue', 'worker', 'global', 'custom']] = <factory>, heartbeat_interval: float = 25.0, replay_limit: int = 0, include_in_schema: bool = False, opt: dict[str, ~typing.Any] | None = None) None#
- scopes: set[Literal['task', 'queue', 'worker', 'global', 'custom']]#
Task-event scopes clients may subscribe to.
- replay_limit: int#
Maximum retained Channels messages replayed on subscription; zero disables replay.
- resolve(namespace: QueueNamespace | None = None) EventStreamConfig[source]#
Resolve the namespace-owned default path without mutating this reusable config.
Plugin-owned WebSocket and SSE streaming endpoints for queue events.
Imported only when a stream config is present and enabled, so base package imports stay free of routing and Channels-driver imports.
The router and both relays are private: QueuePlugin is their only caller,
and applications configure streaming through
EventStreamConfig instead.
StreamMetrics stays public because it describes the observability surface
an application can record against.
- class litestar_queues.events.streaming.StreamMetrics(*args, **kwargs)[source]#
Bases:
ProtocolOptional metric callbacks used by the WebSocket and SSE stream relays.
- on_connect(scope: Literal['task', 'queue', 'worker', 'global', 'custom']) None[source]#
Record a stream connection.
- on_event(scope: Literal['task', 'queue', 'worker', 'global', 'custom']) None[source]#
Record an event sent to a stream client.
- on_heartbeat(scope: Literal['task', 'queue', 'worker', 'global', 'custom']) None[source]#
Record a heartbeat sent to a stream client.
- on_dedup_drop(scope: Literal['task', 'queue', 'worker', 'global', 'custom']) None[source]#
Record a deduplicated event dropped by the stream relay.
- on_denial(scope: Literal['task', 'queue', 'worker', 'global', 'custom'], reason: str) None[source]#
Record an authorization denial.
- on_disconnect(scope: Literal['task', 'queue', 'worker', 'global', 'custom'], duration_seconds: float) None[source]#
Record stream connection lifetime.
- __init__(*args, **kwargs)#