Skip to content

Execution, Workers & Maintenance

API reference for execution backend contracts, serverless and broker execution configurations, worker supervisors, bounded maintenance, and single-record task consumers.

Execution Backend Registry & Base Contract

get_execution_backend

get_execution_backend(backend='immediate', config=None)

Get an instantiated execution backend.

Returns:

Type Description
BaseExecutionBackend

A configured execution backend instance.

Raises:

Type Description
TypeError

If a typed execution config selects a backend class that does not accept execution_config.

get_execution_backend_class

get_execution_backend_class(backend_path)

Get an execution backend class by short name or import path.

Returns:

Type Description
type[BaseExecutionBackend]

The resolved execution backend class.

Raises:

Type Description
ValueError

If a short backend name is unknown.

list_execution_backends

list_execution_backends()

Return registered execution backend names.

execution_backend

execution_backend(name)

Decorator to register an execution backend class with a short name.

Returns:

Type Description
Callable[[type[BaseExecutionBackend]], type[BaseExecutionBackend]]

A decorator that registers the backend class.

BaseExecutionBackend

Base class for queue execution backends.

Attributes:

Name Type Description
is_externalbool

Whether this backend dispatches records to another process.

schedules_on_enqueuebool

Whether the backend schedules a persisted record without a Worker.

max_schedule_horizontimedelta | None

How far ahead this backend will hold a scheduled delivery.

schedule

schedule(service, record)

Schedule one already-persisted record for external delivery.

Returns:

Type Description
str | None

The external delivery reference, if one was created.

repair

repair(service, *, limit)

Recreate deliveries this backend owns that its transport no longer holds.

A no-op for every backend whose records are found by polling: nothing can go missing from a store the worker reads directly. Only a managed transport that took ownership of the record can lose it silently.

Parameters:

Name Type Description Default
service QueueService

The queue service whose records to repair.

required
limit int

Ceiling on how many records one pass may examine. Bounded maintenance is the only caller, and it always passes a positive budget it needs back.

required

Returns:

Type Description
DispatchRepairResult

An empty result.

open

open()

Open execution resources.

Returns:

Type Description
bool

True when resources are ready.

close

close()

Close execution resources.

execute

execute(service, record, *, worker_id=None)

Execute a queue record.

dispatch

dispatch(service, record)

Dispatch a queue record to an external executor.

Returns:

Type Description
str | None

The external execution reference, if one was created.

reconcile

reconcile(service, record)

Reconcile an externally running queue record.

Returns:

Type Description
QueuedTaskRecord | None

The updated record when reconciliation changes state.

cancel_execution

cancel_execution(service, record)

Cancel the provider resource backing one external attempt.

Called before any durable transition to cancelled for a record whose execution_ref names a live provider resource. Implementations must contain their client's exceptions and answer with a result: a not-found resource is already_cancelled, anything transient or refused is retryable. Raising is reserved for programming errors.

Returns:

Type Description
ExecutionCancelResult

unsupported for a transport with no cancellation control plane.

BaseConsumerExecutionBackend

Bases: BaseExecutionBackend

Base for external backends that continuously receive broker deliveries.

run_consumer

run_consumer(service, *, max_concurrency, drain_timeout)

Receive and execute deliveries until cancelled.

DispatchRepairResult

Outcome of one bounded delivery-repair pass.

examined is how much of the caller's budget the pass consumed, whether or not a candidate needed anything done, so the caller can spend what is left on its other work.

Built-in Cloud Tasks repair partitions examined into changed, failed and unchanged. Legacy backends may populate only the first two original fields; their unchanged counts are not inferred. A full selection allowance sets limit_reached conservatively: more work may remain, but no backlog count is performed.

Attributes:

Name Type Description
examinedint

Number of candidate records inspected within the repair budget.

changedint

Number of candidate records whose external delivery was created or updated.

failedint

Number of candidate records whose external delivery repair attempt failed.

unchangedint

Number of candidate records that required no external delivery change.

limit_reachedbool

Whether the candidate scan hit the caller's examination limit.

ExternalReconciliationResult

Repair evidence and completed reconciliation under one shared budget.

Attributes:

Name Type Description
repairDispatchRepairResult

Delivery-repair summary produced before or during external reconciliation.

reconciledint

Number of externally running records transitioned to a terminal queue status.

changedint

Total records repaired or reconciled.

ExecutionCancelResult

Outcome of one provider-level cancellation attempt.

detail carries the provider's own words for the log line and the lifecycle event; it is never parsed.

Attributes:

Name Type Description
statusExecutionCancelStatus

Provider cancellation status ("accepted", "already_cancelled", "retryable", or "unsupported").

detailstr | None

Optional unparsed diagnostic message returned by the external provider.

permits_durable_cancelbool

Whether this outcome may win the durable transition to cancelled.

accepted

accepted(detail=None)

Return an accepted cancellation result.

Returns:

Type Description
Self

A result whose status is accepted.

already_cancelled

already_cancelled(detail=None)

Return an idempotent already-cancelled result.

Returns:

Type Description
Self

A result whose status is already_cancelled.

retryable

retryable(detail=None)

Return a transient or rejected cancellation result.

Returns:

Type Description
Self

A result whose status is retryable.

unsupported

unsupported(detail=None)

Return a result for a transport with no cancellation control plane.

Returns:

Type Description
Self

A result whose status is unsupported.

Local & Immediate Execution

LocalExecutionBackend

Bases: BaseExecutionBackend

Execution backend for in-process workers.

execute

execute(service, record, *, worker_id=None)

Execute one claimed task in a local worker.

Returns:

Type Description
QueuedTaskRecord

The updated queue record.

ImmediateExecutionBackend

Bases: BaseExecutionBackend

Execution backend that runs records inline.

execute

execute(service, record, *, worker_id=None)

Execute a task immediately in the current event loop.

Returns:

Type Description
QueuedTaskRecord

The updated queue record.

Google Cloud Tasks & Cloud Run Jobs

CloudTasksExecutionConfig

Configuration for Google Cloud Tasks managed dispatch.

Cloud Tasks delivers each persisted record over HTTP to a private consumer service, so this carries both the queue coordinates and the delivery target.

Attributes:

Name Type Description
backend_namestr

Registered execution backend identifier ("cloudtasks").

project_idstr

Google Cloud project owning the Cloud Tasks queue.

locationstr

Cloud Tasks queue location, such as us-central1.

queue_idstr

Cloud Tasks queue that receives every dispatched record.

service_urlstr

HTTPS origin of the private consumer service.

service_account_emailstr

Service account Cloud Tasks mints the OIDC delivery token for.

audiencestr | None

OIDC audience; None resolves to the service_url origin.

route_pathstr

Path the consumer route is mounted on.

delivery_name_prefixstr | None

Cloud Tasks task-name prefix; None derives it from QueueConfig.namespace.

dispatch_deadlineint

Seconds Cloud Tasks waits for a delivery response before abandoning it.

response_marginfloat

Seconds reserved for the consumer to answer inside the dispatch deadline.

default_task_timeoutfloat

Timeout applied to records that declare none of their own.

api_timeoutfloat

Timeout for a single Cloud Tasks API call.

mount_routebool

Whether QueuePlugin mounts the Cloud Tasks HTTP delivery route on the app.

verify_oidcbool

Whether the delivery route verifies Google OIDC Bearer tokens in-process.

trust_platform_authbool

Whether Cloud Run IAM alone is accepted as protection for the route.

guardstuple[Guard, ...]

Application guards applied to the consumer route.

target_urlstr

Absolute URL Cloud Tasks posts each record to.

queue_pathstr

Fully qualified Cloud Tasks queue resource name.

resolve

resolve(namespace=None)

Resolve namespace-owned defaults without mutating this reusable config.

CloudTasksExecutionBackend

Bases: BaseExecutionBackend

Execution backend that hands persisted records to Google Cloud Tasks.

__init__

__init__(config=None, *, execution_config=None, client=None)

Initialize the Cloud Tasks execution backend.

cancel_execution

cancel_execution(service, record)

Delete the Cloud Tasks delivery holding this record's attempt.

Cloud Tasks holds a future delivery rather than a running process, so cancellation is a delete: after it, nothing will be delivered. A delivery that is already in flight has been claimed by the consumer, and the delete answers not-found -- idempotent success, with the running body left to cooperative cancellation.

Returns:

Type Description
ExecutionCancelResult

The provider's answer, mapped to the shared cancellation contract.

repair

repair(service, *, limit)

Recreate deliveries Cloud Tasks no longer holds for still-active records.

A delivery can go missing while its record stays active: a create call that failed after Google had already accepted it, an operator purging the queue, a retention window closing before the schedule time arrived. Nothing polls these records, so without this pass they wait forever.

Every candidate is attempted at most once per pass, so a queue whose target is broken cannot spin inside one maintenance window.

Parameters:

Name Type Description Default
service QueueService

The queue service whose records to repair.

required
limit int

Ceiling on how many records this pass may examine.

required

Returns:

Type Description
DispatchRepairResult

How many records the pass looked at, and how many it re-delivered.

close

close()

Release a client this backend created.

An injected client belongs to whoever built it and is left untouched.

CloudTasksOIDCGuard

Litestar route guard that verifies Google-signed OIDC Bearer tokens.

Attributes:

Name Type Description
audiencestr | None

Expected OIDC aud claim (typically the consumer service origin).

service_account_emailstr | None

Expected verified email claim matching the Cloud Tasks OIDC service account.

queue_idstr | None

Optional expected X-CloudTasks-QueueName request header value.

token_verifierCallable[[str, str | None], Mapping[str, Any]] | None

Optional custom token verification callable used in place of google-auth.

CloudRunExecutionConfig

Configuration for Cloud Run Jobs execution.

Attributes:

Name Type Description
backend_namestr

Registered execution backend identifier ("cloudrun").

project_idstr

Google Cloud project containing the target jobs.

regionstr

Google Cloud region containing the target jobs.

job_namestr | None

Default Cloud Run Job name; None requires a matching profile.

profilesdict[str, str]

Execution-profile names mapped to Cloud Run Job names.

timeoutint

Cloud Run API operation timeout in seconds.

env_prefixstr | None

Explicit environment prefix; None derives it from QueueConfig.namespace.

extra_envdict[str, str]

Additional environment variables passed to every Cloud Run execution.

fallback_execution_backendstr | None

Backend used after dispatch failure; None propagates the failure.

schedules_on_enqueuebool

Whether QueueService.enqueue() dispatches due records without a polling worker.

resolve_job_name

resolve_job_name(profile=None)

Return the Cloud Run Job name for a profile.

Returns:

Type Description
str

The resolved Cloud Run Job name.

Raises:

Type Description
QueueConfigurationError

If no job name can be resolved.

env_name

env_name(suffix, *, namespace=None)

Return an environment variable name using the configured prefix.

CloudRunExecutionBackend

Bases: BaseExecutionBackend

Execution backend that dispatches queued records to Cloud Run Jobs.

__init__

__init__(
    config=None, *, execution_config=None, jobs_client=None, executions_client=None
)

dispatch

dispatch(service, record)

Dispatch a queue record to Cloud Run Jobs.

Returns:

Type Description
str | None

The Cloud Run execution reference, if dispatch succeeds.

cancel_execution

cancel_execution(service, record)

Cancel the Cloud Run execution backing this record's attempt.

Returns:

Type Description
ExecutionCancelResult

The provider's answer, mapped to the shared cancellation contract.

reconcile

reconcile(service, record)

Reconcile a Cloud Run execution with the queue record.

Returns:

Type Description
QueuedTaskRecord | None

The terminal queue record when reconciliation completed it.

repair

repair(service, *, limit)

Dispatch due Cloud Run candidates that have no active execution reference.

Returns:

Type Description
DispatchRepairResult

Counts of examined, dispatched, failed, and unchanged candidates.

Raises:

Type Description
QueueConfigurationError

If limit is negative.

Broker Transports (Pub/Sub, SQS, RabbitMQ, Kafka)

PubSubExecutionConfig

Google Cloud Pub/Sub execution-dispatch configuration.

Attributes:

Name Type Description
backend_namestr

Registered execution backend identifier ("pubsub").

project_idstr

Google Cloud project containing the topic and subscription.

topic_idstr

Topic receiving task-id dispatch messages.

subscription_idstr

Pull subscription consumed by litestar queues run-consumer.

ack_deadlineint

Initial acknowledgment deadline in seconds; never the queue lease.

ack_extension_intervalint

Courtesy acknowledgment-deadline extension cadence during execution.

dispatch_stale_afterint

Age after which a reserved attempt can be rotated and republished.

api_timeoutfloat

Timeout applied to unary Pub/Sub API calls.

api_endpointstr | None

Optional API host override, including the official emulator host.

api_insecurebool

Use plaintext gRPC for a configured local emulator endpoint.

topic_pathstr

Return the fully qualified Pub/Sub topic path.

subscription_pathstr

Return the fully qualified Pub/Sub subscription path.

PubSubExecutionBackend

Bases: BaseConsumerExecutionBackend

Dispatch bare task identifiers through Google Cloud Pub/Sub.

__init__

__init__(config=None, *, execution_config=None, publisher=None, subscriber=None)

dispatch

dispatch(service, record)

Reserve and publish one task-id delivery.

run_consumer

run_consumer(service, *, max_concurrency, drain_timeout)

Consume streaming-pull deliveries until cancelled.

close

close()

Close clients created by this backend.

SqsExecutionConfig

Amazon SQS execution-dispatch configuration.

Attributes:

Name Type Description
backend_namestr

Registered execution backend identifier ("sqs").

queue_urlstr

Absolute URL of the target SQS queue.

region_namestr | None

Optional AWS region override; the normal SDK chain supplies the default.

endpoint_urlstr | None

Optional endpoint override, primarily for LocalStack.

fifobool

Whether to emit FIFO-only group and deduplication fields.

message_group_idstr | None

Explicit FIFO group; when omitted a stable queue-name group is derived.

wait_time_secondsint

SQS long-poll duration.

receive_batch_sizeint

Maximum number of messages requested per receive.

visibility_timeoutint

Initial SQS delivery visibility, independent from queue heartbeat leases.

visibility_extension_intervalint

Courtesy visibility-extension cadence during local execution.

dispatch_stale_afterint

Age after which an owned attempt can be atomically rotated and republished.

api_timeoutfloat

Timeout applied to individual SQS API operations.

SqsExecutionBackend

Bases: BaseConsumerExecutionBackend

Dispatch bare task identifiers through Amazon SQS.

__init__

__init__(config=None, *, execution_config=None, client=None)

dispatch

dispatch(service, record)

Reserve external dispatch and publish the task ID to the configured SQS queue.

run_consumer

run_consumer(service, *, max_concurrency, drain_timeout)

Run the SQS long-poll consumer loop and execute received task IDs via consume_one().

repair

repair(service, *, limit)

Rotate stale SQS dispatch reservations and republish their task IDs.

close

close()

Close the owned aiobotocore SQS client context.

RabbitMQExecutionConfig

RabbitMQ execution-dispatch configuration.

Attributes:

Name Type Description
backend_namestr

Registered execution backend identifier ("rabbitmq").

amqp_urlstr

AMQP or AMQPS connection URL; excluded from representations and telemetry.

queue_namestr | None

Broker queue name, derived from the queue namespace when omitted.

declare_queuebool

Declare the quorum queue when true; otherwise verify it passively.

dispatch_stale_afterint

Age in seconds after which an owned attempt can be rotated and republished.

api_timeoutfloat

Timeout in seconds for connection and publish operations.

delayed_retry_typeLiteral['disabled', 'all', 'failed', 'returned']

RabbitMQ 4.3 quorum-queue delayed retry category.

delayed_retry_min_msint

Minimum broker-managed redelivery delay in milliseconds.

delayed_retry_max_msint

Maximum broker-managed redelivery delay in milliseconds.

consumer_timeout_msint | None

Optional acknowledgement timeout; must exceed legitimate task duration.

RabbitMQExecutionBackend

Bases: BaseConsumerExecutionBackend

Dispatch bare task identifiers through a RabbitMQ quorum queue.

__init__

__init__(config=None, *, execution_config=None, connection=None)

dispatch

dispatch(service, record)

Reserve external dispatch and publish the task ID to the RabbitMQ quorum queue.

run_consumer

run_consumer(service, *, max_concurrency, drain_timeout)

Run the RabbitMQ consumer loop and execute received task IDs via consume_one().

close

close()

Close open RabbitMQ publisher/consumer channels and the AMQP connection.

KafkaExecutionConfig

Kafka execution-dispatch configuration.

Attributes:

Name Type Description
backend_namestr

Registered execution backend identifier ("kafka").

bootstrap_serversstr

Comma-separated Kafka bootstrap servers.

topicstr

Topic receiving task-id dispatch records.

consumer_groupstr

Consumer group used by litestar queues run-consumer.

dispatch_stale_afterint

Age after which a reserved attempt can be rotated and republished.

api_timeoutfloat

Timeout applied to producer delivery acknowledgements.

producer_optionsdict[str, Any]

Additional AIOKafkaProducer options, such as TLS or SASL settings.

consumer_optionsdict[str, Any]

Additional AIOKafkaConsumer options, such as TLS or SASL settings.

KafkaExecutionBackend

Bases: BaseConsumerExecutionBackend

Dispatch bare task identifiers through a Kafka consumer group.

__init__

__init__(config=None, *, execution_config=None, producer=None, consumer=None)

dispatch

dispatch(service, record)

Reserve external dispatch and publish the task ID to the configured Kafka topic.

run_consumer

run_consumer(service, *, max_concurrency, drain_timeout)

Run the Kafka consumer group loop and execute received task IDs via consume_one().

close

close()

Stop owned Kafka producer and consumer clients and clear in-flight state.

Worker, Consumer & Maintenance

Worker

Local in-process queue worker.

Attributes:

Name Type Description
is_runningbool

Whether the worker loop is active.

worker_idstr

Worker identity used for events and logs.

wait_started

wait_started()

Wait until heartbeat startup succeeds or propagate its failure.

start

start()

Run the worker loop until stopped or cancelled.

stop

stop(*, force=False)

Stop the worker loop and drain or cancel in-flight work.

Returns:

Type Description
bool

True when graceful drain escalated to cancellation.

run_once

run_once()

Process one batch of due tasks.

Returns:

Type Description
int

Number of claimed task records.

Raises:

Type Description
QueueConfigurationError

If the execution backend schedules its own delivery, leaving a worker nothing it may safely claim.

reconcile_external

reconcile_external(*, limit=None)

Reconcile externally dispatched records by delegating to the service.

The reconciliation contract (state transitions, unknown-backend skipping, and metrics) lives on QueueService.reconcile_external(); the worker keeps only the periodic cadence and fleet lock.

Returns:

Type Description
int

Number of records that reached a terminal queue status.

WorkerPlacement

WorkerPlacement = Literal['server', 'asgi', 'external']

Which process owns the queue worker.

server The Litestar CLI server lifespan owns exactly one fresh worker process per litestar run invocation. This is the default. asgi Each ASGI worker owns one queue worker inside its own application lifespan. Deliberately multiplicative with the web-worker count. external Nothing is started automatically; a separate process manager runs litestar queues run, or the caller executes tasks inline.

consume_one

consume_one(queue, task_id, *, expected_retry_count=None, expected_execution_ref=None)

Claim, execute, and report one queued record identified by its id.

The live record in the queue backend is authoritative; the id only locates it. Redelivery is fenced by the live expected_retry_count at claim time.

The claim is taken before the task name is resolved, because retiring a record this process cannot run is itself a write, and a persistent backend only accepts one over a record that is running and owned.

Returns:

Type Description
TaskExitCode

A deterministic task exit code.

run_task

run_task(
    *,
    config=None,
    service=None,
    service_factory=None,
    task_id=None,
    config_factory=None,
    task_modules=None,
    env=None,
)

Resolve a service and run one queued task by id.

The prefix-aware environment is the default source for every input; the override arguments take precedence over it. config_factory replaces the CONFIG_FACTORY env var, task_id replaces the TASK_ID value, and task_modules replaces TASK_MODULES.

Returns:

Type Description
TaskExitCode

A deterministic task exit code.

TaskExitCode

Bases: IntEnum

Deterministic external-consumer process exit codes.

CANCELLED reports a record whose durable execution result is cancelled. A consumer whose own caller was cancelled reports nothing at all: the cancellation propagates instead, because the record is still running and no outcome has been reached.

QueueMaintenanceConfig

Bounded maintenance thresholds and limits.

Durations and retention values are seconds. Every limit and duration must be positive and coordination_timeout must exceed time_budget so ownership outlives the whole run. stale_after, terminal_retention, and event_retention_rules default to None (or empty) which disables their phase; there are no destructive defaults.

Attributes:

Name Type Description
time_budgetfloat

Budget checked before each phase; in-flight operations can run longer.

coordination_timeoutfloat

Distributed ownership duration in seconds; must exceed time_budget.

expire_limitint

Maximum overdue pending or scheduled tasks expired in one run.

external_limitint

Maximum external executions reconciled in one run.

stale_afterfloat | None

Running-task age threshold in seconds; None disables stale recovery.

stale_limitint

Maximum stale running tasks recovered in one run.

schedules_limitint

Maximum recurring schedules asserted in one run.

terminal_retentionfloat | None

Terminal-task retention age in seconds; None disables deletion.

terminal_limitint

Maximum expired terminal tasks deleted in one run.

event_retention_rulestuple[QueueEventRetentionRule, ...]

Ordered event-history retention rules; empty disables the events phase.

event_limitint

Maximum expired task-event records deleted in one run.

mount_routebool

Whether QueuePlugin mounts an HTTP endpoint that runs maintenance once.

route_pathstr

Path the HTTP maintenance endpoint is mounted on.

guardstuple[Guard, ...]

Application guards applied to the HTTP maintenance endpoint.

trust_platform_authbool

Whether platform IAM alone is accepted as protection for the HTTP maintenance endpoint.

verify_oidcbool

Whether the HTTP maintenance endpoint verifies Google OIDC Bearer tokens in-process.

oidc_service_account_emailstr | None

Expected Google OIDC service account email when verify_oidc=True.

oidc_audiencestr | None

Expected Google OIDC audience when verify_oidc=True.

resolve

resolve(namespace=None)

Resolve namespace-owned defaults without mutating this reusable config.

QueueMaintenanceService

Run bounded maintenance phases under token-fenced coordination and a time budget.

run

run(phases=None)

Claim maintenance ownership and run each selected phase once.

Parameters:

Name Type Description Default
phases Collection[MaintenancePhase] | None

Optional narrowing of the phases to run. Filtering only narrows configuration; it never enables a disabled retention threshold. None considers every phase in the fixed order.

None

Returns:

Type Description
QueueMaintenanceSummary

A summary whose outcome is already_running when ownership is denied,

QueueMaintenanceSummary

failed when any phase or repair failed, partial when a scan

QueueMaintenanceSummary

exhausted its allowance or the time budget skipped an enabled phase,

QueueMaintenanceSummary

else completed.

Raises:

Type Description
QueueConfigurationError

If a requested phase name is unknown or the backend does not support distributed maintenance coordination.

QueueMaintenanceSummary

Result of a whole maintenance run.

Attributes:

Name Type Description
outcomeMaintenanceOutcome

Overall maintenance run outcome ("completed", "busy", "partial", or "failed").

acquiredbool

Whether this caller acquired the distributed maintenance coordination lease.

duration_msfloat

Total elapsed maintenance execution time in milliseconds.

phaseslist[QueueMaintenancePhaseResult]

Ordered per-phase execution summaries.

to_payload

to_payload()

Return a JSON-native mapping of the whole summary.

QueueMaintenancePhaseResult

Result of one bounded maintenance phase.

Attributes:

Name Type Description
phaseMaintenancePhase

Name of the maintenance phase executed.

statusMaintenancePhaseStatus

Outcome status of this phase ("completed", "skipped", "failed", or "budget_exhausted").

changedint

Count of records created, updated, or deleted during this phase.

duration_msfloat

Elapsed execution duration of this phase in milliseconds.

errorstr | None

Sanitized error message if the phase failed, otherwise None.

repairDispatchRepairResult | None

Structured external dispatch repair summary produced by this phase, if applicable.

to_payload

to_payload()

Return a JSON-native mapping of this phase result.