Google Cloud Run Jobs Execution¶

CloudRunExecutionBackend (configured via CloudRunExecutionConfig) executes each queued task inside an isolated, one-shot Google Cloud Run Job container that scales to zero immediately upon exit.

Runnable Configuration Example¶

"""Configuring CloudRunExecutionConfig with execution profiles and schedules_on_enqueue."""

from litestar_queues import QueueConfig, WorkerConfig, task
from litestar_queues.backends.redis import RedisBackendConfig
from litestar_queues.execution.cloudrun import CloudRunExecutionConfig


@task("reports.render_annual_pdf", queue="reports", execution_backend="cloudrun", execution_profile="high-memory")
async def render_annual_pdf(organization_id: str) -> str:
    """Render a large PDF inside a high-memory Cloud Run Job execution."""
    return f"pdf:{organization_id}"


def build_cloud_run_config() -> QueueConfig:
    """Configure Cloud Run Jobs execution with profile mapping and direct enqueue dispatch."""
    execution_config = CloudRunExecutionConfig(
        project_id="my-gcp-project",
        region="us-central1",
        job_name="litestar-worker-standard",
        profiles={"default": "litestar-worker-standard", "high-memory": "litestar-worker-highmem"},
        schedules_on_enqueue=True,
    )
    return QueueConfig(
        queue_backend=RedisBackendConfig(url="redis://redis.internal:6379/0"),
        execution_backend=execution_config,
        worker=WorkerConfig(placement="external"),
    )


async def main() -> None:
    """Verify Cloud Run profile resolution and task routing."""
    config = build_cloud_run_config()
    assert isinstance(config.execution_backend, CloudRunExecutionConfig)
    assert config.execution_backend.schedules_on_enqueue is True
    assert config.execution_backend.resolve_job_name("high-memory") == "litestar-worker-highmem"
    assert config.execution_backend.resolve_job_name(None) == "litestar-worker-standard"
    assert render_annual_pdf.execution_profile == "high-memory"

Cloud Run Deployment Topologies¶

Job Container Command & Environment Contract¶

Configure your Cloud Run Job container entrypoint to invoke the one-shot task consumer CLI (litestar queues run-task or python -m litestar_queues.consumer):

ENTRYPOINT ["litestar", "--app", "my_app.main:app", "queues", "run-task"]

Or without loading a Litestar HTTP application object, point QUEUES_CONFIG_FACTORY to a callable returning QueueConfig:

ENV QUEUES_CONFIG_FACTORY="my_app.queues:build_queue_config"
ENV LITESTAR_QUEUES_TASK_MODULES="my_app.tasks"
ENTRYPOINT ["python", "-m", "litestar_queues.consumer"]

When CloudRunExecutionBackend triggers run_job(), it passes the ID-only override environment variable QUEUES_TASK_ID=<uuid> (plus any static CloudRunExecutionConfig.extra_env variables) to the container overrides. Task arguments and metadata are never passed in container environment overrides; run-task reads QUEUES_TASK_ID and claims the authoritative record from your storage backend.

Execution Profiles & Hardware Sizing¶

Map logical profile names to distinct Cloud Run Job resources (for example standard CPU vs high-memory vs GPU jobs) using CloudRunExecutionConfig.profiles. When resolving which Cloud Run Job to launch for a QueuedTaskRecord, resolve_job_name() evaluates in strict order:

1. profiles[record.execution_profile] when record.execution_profile is set and present in profiles.
2. CloudRunExecutionConfig.job_name when configured.
3. profiles["default"] when present.

If none of the three resolves a job name, QueueConfigurationError is raised.

Required IAM Permissions & Cloud SQL Connectivity¶

Grant the service account executing QueueService.enqueue() (and QueueMaintenanceService) permission to launch, inspect, and cancel Cloud Run Job executions:

• run.jobs.run and run.jobs.runWithOverrides (included in roles/run.developer)
• run.executions.get (used by reconcile_external_tasks() to inspect execution completion or failure)
• run.executions.cancel (used by QueueService.cancel_task() to terminate running job executions)

When connecting both the web service and Cloud Run Jobs to Cloud SQL or Memorystore Redis/Valkey, attach the Cloud SQL Auth Proxy volume (--add-cloudsql-instances) or Direct VPC Egress (--network / --subnet) to both the Cloud Run Service and every target Cloud Run Job definition.

Dispatch Failure Handling & Deterministic Exit Codes¶

• Fallback Backend (fallback_execution_backend): If CloudRunExecutionConfig.fallback_execution_backend is set (for example "local"), dispatch failures fall back to the named execution backend; when None (default), dispatch errors leave the record eligible for bounded reconcile_external_tasks() repair.
• Deterministic Container Exit Codes (TaskExitCode): litestar queues run-task and python -m litestar_queues.consumer exit with distinct codes so Cloud Run Job execution logs reflect the exact outcome: