Choose backends#
Make these five choices separately. Most applications need only task storage and execution placement. The other choices can reduce worker delay or deliver events to users.
Need |
Queue persistence |
Execution placement |
Worker wakeup strategy |
User-facing event stream |
Next guide |
|---|---|---|---|---|---|
One process or tests |
|
|
Polling/in-process hint |
Memory Channels (process-local) |
|
SQLSpec application |
|
|
Native transport by default when supported, otherwise polling |
Configure Channels separately |
|
SQLAlchemy application |
|
|
Optional PostgreSQL hint, otherwise polling |
Configure Channels separately |
|
Redis infrastructure |
|
|
Redis pub/sub hint |
Optional Redis Channels |
|
Valkey infrastructure |
|
|
Valkey pub/sub hint |
Optional Valkey-compatible Channels |
The queue backend stores task records. local runs work in a worker process.
immediate runs it during the enqueue call. Cloud Run runs it remotely; it
does not store the queue. Backend notifications wake workers, but they do not
send task events to browsers.
Worker-wakeup capability matrix#
This matrix is checked against the runtime’s explicit SQLSpec adapter mapping, canonical transport set, and Advanced Alchemy driver set:
Backend or adapter |
Enabling setting |
Effective strategy |
Durable wakeup |
Ownership |
|---|---|---|---|---|
Memory |
Always |
asyncio-event |
No |
Process-local hint |
SQLSpec: asyncpg, psycopg, psqlpy |
Default |
notify_queue |
Yes |
Durable queue plus PostgreSQL push |
SQLSpec: DuckDB |
Default |
poll_queue |
Yes |
Durable embedded queue |
SQLSpec: other adapters |
Default |
Polling |
N/A |
Durable task-state polling |
SQLSpec: Oracle |
Explicit only |
aq or txeventq |
Yes |
Application-provisioned Oracle queue |
Advanced Alchemy: asyncpg, psycopg |
worker_wakeups=True |
postgres-listen-notify |
No |
Transient marker |
Advanced Alchemy: other drivers |
Any |
Polling |
N/A |
Durable task-state polling |
Redis |
Default |
Redis pub/sub |
No |
Transient marker |
Valkey |
Default |
Valkey pub/sub |
No |
Transient marker |
Durable wakeup describes the notification transport, not the queue record.
Queue records remain authoritative in every row.
Install extras#
Integration |
Install extra |
Shared scope |
|---|---|---|
Memory |
Core package |
Current Python process only |
SQLSpec |
|
Shared database |
Advanced Alchemy |
|
Shared database |
Redis |
|
Shared Redis service |
Valkey |
|
Shared Valkey service |
Cloud Run execution |
|
Shared persistent queue required |
Event-history support#
Queue backend |
History support |
Ownership |
|---|---|---|
Memory |
Supported, bounded and ephemeral |
|
SQLSpec |
Supported |
The app owns the queue schema and migrations; SQLSpec manages the sessions. |
Advanced Alchemy |
Supported |
The app owns the model and migrations. |
Redis / Valkey |
Supported |
You choose how long records stay and whether to back them up. |
Event history saves records for later queries. It does not deliver live events. See Event history and SSE and WebSockets.
Topology and security#
Topology |
Queue records |
Live events |
Security boundary |
|---|---|---|---|
Memory examples |
Process-local memory |
|
Local demo; do not expose as a multi-replica service. |
Separate Redis/Valkey worker |
Shared Redis or Valkey |
Explicit shared Channels backend and distinct key prefix |
Authenticate the service, isolate prefixes, and authorize stream routes. |
SQL/AA workers |
Shared database |
Separately configured Channels transport |
Protect database credentials and authorize subscriber scopes. |
Adding a wakeup or control transport#
The rows above are what the shipped backends provide. If you are writing a new queue backend, two optional hooks on the backend base class carry best-effort hints, and both are lossy by contract — a dropped hint costs latency, never correctness:
Hook pair |
Purpose |
|---|---|
|
Tell an idle worker that new work exists, so it stops waiting early. |
|
Tell the owning worker to reconcile now, used by running cancellation. |
A backend that implements neither inherits pure polling and stays correct: the worker reconciles durable queue and task state on a fixed cadence regardless. No wakeup or cancellation outcome may depend on a hint arriving.
Continue with Run workers or the focused backend guide selected above.