Skip to content
synaptiaiPublic

About

Elastic, priority-aware task scheduler for Rust, Python and gRPC: aging priorities, deadlines, cron and fixed starts, DAGs, retries, rate limits, queue-pressure autoscaling and Postgres-backed HA.

Topics

Resources

Code of conduct

Contributing

Security policy

Stars

0 stars

Watchers

0 watching

Forks

cronoq

An elastic, priority-aware task scheduler for Rust, Python and any language over gRPC.
Priorities with aging · deadlines · cron with reserved capacity · DAGs · rate limits · queue-pressure autoscaling · Postgres or SQLite HA
Ordering keys · replace and update of queued work · pause and resume · embeddable in Go, Swift and C (experimental)

CI crates.io docs.rs PyPI MSRV License

Documentation · Use cases · Quickstart · Contributing


cronoq decides when work runs, in what order, and how much capacity it needs. You decide where it runs: async functions in your process, Python coroutines, or remote workers connected over gRPC. One engine powers all three, with the same scheduling rules and metrics. An experimental C library embeds the same engine in Go, Swift and C programs (Embedding from other languages).

Why cronoq

Plain queues treat every job the same. Real workloads do not:

  • 🧑‍💻 User-facing work stuck behind batch jobs → four priority classes, so urgent work is dispatched ahead of batch work; aging lets long-waiting background work overtake a steady stream of ordinary work, while high-priority work still goes first.
  • ⏰ SLAs missed with idle capacity earlier in the day → deadlines raise urgency as the latest start time approaches; earliest-deadline-first within a class.
  • 📅 Cron jobs starting late because workers were busy → fixed-start tasks reserve capacity ahead of their start and get a worker held just before it; a run that cannot get capacity is reported as a conflict instead of starting late unnoticed.
  • 🚦 429 storms against rate-limited APIs → shared token buckets and semaphores are enforced before dispatch; queued tasks wait for a limit without occupying a worker; retry_after is honoured.
  • 💸 Paying for idle workers at night, saturated at noon → queue-pressure autoscaling per pool, pre-warmed for upcoming reservations, with cost reporting.
  • 🔗 Hand-rolled job chaining → DAG edges on success, failure or completion; atomic batches; idempotent keys.
  • 🏢 One tenant's burst delays everyone else → weighted fair sharing between tenants within each priority class; a token can be pinned to its tenant so a client can't spend another tenant's share.
  • 🎛️ GPU jobs landing on CPU-only workers → worker labels: a task that requires gpu=a100 runs only on workers that have it, without blocking the rest of the pool.
  • 🧾 Work for one account, document or device running out of order → ordering keys: tasks that share a key run one at a time, in submission order, while other keys run in parallel.
  • ♻️ Stale updates wasting capacity → replace by key: the newest submission of a key wins, and a task that already runs is followed by the newest value instead of being aborted. A queued task's priority, deadline and tags can be updated in place, and a key reused for different work is reported as a conflict.
  • ⏸️ No way to hold a queue → pause a pool, one class or one ordering key, with a reason; held work keeps its place. A worker that cannot run a task now releases it without using a retry.
  • 📱 No engine for a client app → an experimental C library runs the same engine inside a Go, Swift or C program, without a server, for example as the send queue of a desktop or mobile app.
  • 💥 Lost jobs when a worker dies → leases, heartbeats, timeouts, retries with backoff, dead letters, and failover backed by Postgres or a local SQLite file. Attempt numbers never repeat, so workers can use them as fencing tokens for effectively-once side effects.

It is fast enough for the hot path: ~1 million scheduling decisions per second per core. A dispatch compares the heads of a few sorted queues instead of scanning the backlog, so a deep queue does not slow it down noticeably.

Not sure it fits your system? Benefits at a glance maps common symptoms to features, shows where to start, and lists the trade-offs.

Install

What How
Rust library cargo add cronoq (Rust 1.95+)
Python pip install cronoq for the in-process scheduler; pip install "cronoq[client]" adds the gRPC client and worker (CPython 3.14+)
Server (container) docker pull ghcr.io/synaptiai/cronoq:0.3.0 (includes the cronoq CLI)
Server (binaries) Linux, macOS and Windows archives on the release page, each with cronoq-server and cronoq
Kubernetes helm install cronoq oci://ghcr.io/synaptiai/charts/cronoq --version 0.3.0; the operator chart is oci://ghcr.io/synaptiai/charts/cronoq-operator
Command-line tool cargo install cronoq-cli
Go go get github.com/synaptiai/cronoq/sdks/go@v0.3.0 (Go 1.26+)
TypeScript @synaptiai/cronoq on GitHub Packages, or the tarball on the release page (Node 22+; how to install)
Embedded C library (experimental) An archive per platform on the release page, cronoq-embed-v<version>-<target> (and cronoq-embed-sqlite-… with the SQLite store), with the static and dynamic libcronoq_ffi and the header cronoq.h. Or build it from this repository: cargo build --release -p cronoq-ffi.
Go, embedded (experimental) The package github.com/synaptiai/cronoq/sdks/go-embed, in sdks/go-embed. It needs the C library: by default it loads the dynamic library at run time without cgo; with the build tag cronoq_static it links the static library. go get github.com/synaptiai/cronoq/sdks/go-embed@v0.3.0.
Swift, embedded (experimental) The package Cronoq, in sdks/swift (macOS 12+, iOS 15+). It is not in a package registry. It links the static C library: build the library and pass -Xlinker -L<dir with libcronoq_ffi.a>, or use the binary framework CCronoq.xcframework.zip from the release page (how).

Wheels, binaries, images and the TypeScript package carry build provenance attestations: gh attestation verify <file> --repo synaptiai/cronoq, or oci://ghcr.io/synaptiai/cronoq:0.3.0 in place of the file for an image.

Quickstart

Rust

[dependencies]
cronoq = "0.3"
tokio = { version = "1", features = ["macros", "rt-multi-thread"] }
use std::{sync::Arc, time::Duration};
use cronoq::{Pool, Priority, ResourceConfig, Scheduler, Task, TaskError};

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let sched = Scheduler::<String, usize>::builder()
        .resource(ResourceConfig::token_bucket("llm-api", 20.0, 5)) // 20 req/s, burst 5
        .pool(Pool::new("agents").workers(1, 2, 32).executor_fn(|_ctx, prompt: Arc<String>| async move {
            // call a model; return Err(TaskError::retry_after("429", d)) to back off
            Ok::<_, TaskError>(prompt.len())
        }))
        .build()
        .await?;

    let handle = sched
        .submit(Task::new("hello".to_string())
            .pool("agents")
            .priority(Priority::High)
            .deadline_in(Duration::from_secs(5))
            .resource("llm-api", 1))
        .await?;
    println!("{}", handle.await?);

    sched.shutdown(Duration::from_secs(5)).await?;
    Ok(())
}

Python

After pip install cronoq:

import asyncio, cronoq

async def summarize(ctx, text):
    return text[:20]

async def main():
    async with cronoq.Scheduler([cronoq.Pool("default", summarize, min=1, max=8)]) as sched:
        handle = await sched.submit("a long document ...", priority="high", deadline_in=5.0)
        print(await handle)

asyncio.run(main())

Server with remote workers

# One random token per role (at least 16 characters; 32+ recommended).
for role in CLIENT WORKER METRICS ADMIN; do export CRONOQ_${role}_TOKEN=$(openssl rand -hex 24); done
docker run --name cronoq -p 7070:7070 -p 7071:7071 \
  -e CRONOQ_CLIENT_TOKEN -e CRONOQ_WORKER_TOKEN -e CRONOQ_METRICS_TOKEN -e CRONOQ_ADMIN_TOKEN \
  ghcr.io/synaptiai/cronoq:0.3.0

The server listens on port 7070 (gRPC) and 7071 (HTTP), with an in-memory store and the pools of the example configuration, including default. Start a worker in another shell with the same tokens exported (pip install "cronoq[client]"):

import asyncio, os
from cronoq.client import Worker

async def handle(lease):
    return {"echo": lease.json()}

asyncio.run(Worker("localhost:7070", "default", handle, slots=4,
                   token=os.environ["CRONOQ_WORKER_TOKEN"]).run())

Then submit a task and wait for its result, over HTTP or with the cronoq CLI in the image:

curl -X POST localhost:7071/v1/tasks -H "authorization: Bearer $CRONOQ_CLIENT_TOKEN" \
     -H 'content-type: application/json' \
     -d '{"payload": {"hello": "world"}, "priority": "high", "wait": true}'
docker exec -e CRONOQ_TOKEN=$CRONOQ_CLIENT_TOKEN cronoq \
  cronoq submit '{"hello": "world"}' --json --priority high --wait

The HTTP response carries the worker's answer in result.output, and the CLI prints it: {"echo": {"hello": "world"}}. The server quickstart explains each step, and leads on to deployment with Postgres, Kubernetes and TLS.

Embedded in a Go, Swift or C program (experimental)

No server: the scheduler runs inside your program. Build the C library first (cargo build --release -p cronoq-ffi). In Go:

import embedded "github.com/synaptiai/cronoq/sdks/go-embed"

s, err := embedded.Open(embedded.Config{})
if err != nil { log.Fatal(err) }
defer s.Close()

// Messages of one chat are sent in order; a newer title replaces the one that waits.
s.Submit([]byte("send event 17"), &embedded.Task{Priority: embedded.High, OrderingKey: "chat-1"})
s.Submit([]byte("title v2"), &embedded.Task{Key: "title", OnKeyConflict: "replace_queued"})

lease, err := s.Next("", 5*time.Second) // waits up to 5 s for a task
if err != nil { log.Fatal(err) }        // embedded.ErrTimeout, ErrPaused, ErrShutdown, ...
if err := send(lease.Payload); err != nil {
    lease.Fail(err.Error(), true, 0)    // retryable
} else {
    lease.Complete()
}

Run the full example from inside the Go module (on macOS the library is libcronoq_ffi.dylib):

cargo build --release -p cronoq-ffi
(cd sdks/go-embed && CRONOQ_LIBRARY="$PWD/../../target/release/libcronoq_ffi.so" go run ./example)

The same from Swift and C, the options, and what to do when a phone sleeps: Embedding from other languages.

Use cases

LLM and agent workloads Rate limits, 429 backoff, chat before batch, autoscaling
Interactive + batch on shared capacity One fleet instead of two, batch still finishes
Deadline-driven work Reorder by urgency as deadlines approach
Scheduled jobs with guaranteed capacity Cron with capacity reserved ahead of each run
Pipelines and DAGs Fan-out/fan-in, failure handlers, atomic batches
GPUs and scarce resources Shared semaphores, no head-of-line blocking
High-throughput microtasks A few microseconds of scheduling overhead per task
Polyglot worker fleets gRPC workers in any language, HA
Cost-aware autoscaling Queue-pressure scaling, a desired-capacity metric for KEDA or HPA
Ordered work per entity One task per account, document or device at a time, in order; entities in parallel
Latest-wins updates State sync, cache refresh, index rebuilds: only the newest value runs
Idempotent producers Safe retries by key; a key reused for different work is reported
Reprioritising live work Promote queued work a user is now waiting for
Holding work without losing it Pause for maintenance windows; release when a downstream service is unreachable
Offline-first clients and embedded queues A send queue inside a desktop or mobile app, in Swift or Go (experimental)

Features

Area Highlights
Ordering 4 priority classes, bounded aging, FIFO within a class (or weighted fair sharing between tenants), deadline urgency, earliest deadline first (by latest start time)
Order per key Ordering keys: tasks that share one run one at a time, in submission order; optional halt on failure; a class floor per key
Changing queued work Update priority, deadline or tags in place; replace by key (latest wins); conflict detection for reused keys, by fingerprint
Holding work Pause and resume a pool, a class or an ordering key, with a reason; durable; workers release attempts without using a retry
Placement Worker labels with required labels per task; tasks without requirements prefer the least specialized workers
Time Fixed starts with reservations and worker holds, miss policies, cron with time zones
Dependencies DAG edges by id or key, on_success / on_failure / on_complete, atomic batches, dedup keys
Reliability Leases, heartbeats, timeouts, retries with exponential backoff and retry_after, dead letters, cancellation
Limits Semaphores and token buckets shared across pools (within a shard), bounded queues (wait / reject / shed)
Elasticity Per-pool queue-pressure autoscaling, wait-time SLOs, reservation pre-warming, cost reporting
Durability A Postgres store, or SQLite with standbys on the same host; leader election, epoch fencing, recovery, at-least-once execution, attempt numbers for fencing side effects
Scale-out Pools split between shards, each with its own leader and store; client calls to any server reach the right shard, and workers are redirected to it
Interfaces Rust (async + blocking), Python (asyncio), Go, TypeScript (Node and browsers), gRPC, gRPC-Web, HTTP/JSON, the cronoq CLI with a live top view
Embedding Experimental: a C library with Go and Swift wrappers runs the engine inside a program without a server; fixed starts, dependencies, tenants, labels and recurring schedules (UTC); optional SQLite file; clock resynchronisation after sleep
Operations Helm charts, a Kubernetes operator that scales worker Deployments, a webhook scaler and a desired-capacity metric for KEDA, token roles limited to pools and pinned to tenants, TLS and mTLS
Observability Prometheus metrics, windowed quality metrics, and in the Rust library OpenTelemetry metrics and trace propagation

Performance

Measured on a shared 4-vCPU VM
Engine: submit → dispatch → complete ~1.0–1.2 million cycles/s per core
End to end through tokio, no-op tasks submitted in bulk ~250,000 tasks/s
Sequential submit → await round trip ~70 µs

A benchmark workflow runs on every push to main (cargo bench -p cronoq-core, cargo bench -p cronoq) and fails if the engine, with 64 workers and a backlog of 10,000, falls below 700,000 cycles/s, or the end-to-end run below 175,000 tasks/s. It does not check the round-trip time.

Architecture

flowchart LR
    subgraph surfaces[Interfaces]
        R[Rust API]
        P[Python API]
        G[gRPC / HTTP server]
    end
    R & P & G -->|commands| D[Driver task]
    D <-->|inputs / actions| C[cronoq-core<br/>sans-IO engine]
    D -->|dispatch| E[Executors / remote workers]
    E -->|outcomes, heartbeats| D
    D -->|scale requests| S[Scalers]
    D -->|group commit| DB[(Postgres or SQLite store,<br/>or in memory)]
Loading

The engine is a deterministic state machine with no clock or I/O, tested by randomized simulation under virtual time. The embedded C library runs the same engine and driver inside a host program, whose threads take the place of the workers. Details: Architecture.

Documentation

Repository layout

Path Description
crates/cronoq-core Sans-IO scheduling engine
crates/cronoq Tokio runtime and Rust API
crates/cronoq-store-postgres Durable store with leader election and fencing
crates/cronoq-store-sqlite Durable store in a local file, with standby processes and fencing
crates/cronoq-store-tests Conformance tests every durable store passes (not published)
crates/cronoq-proto Protobuf definitions and generated gRPC code
crates/cronoq-server gRPC/HTTP server, remote workers, Rust worker client
crates/cronoq-cli cronoq command-line client with a live top view
crates/cronoq-py Python bindings and gRPC client (cronoq on PyPI)
crates/cronoq-operator Kubernetes operator that scales worker Deployments (published as an image)
crates/cronoq-ffi C library and header that embed the scheduler in other languages (experimental)
sdks/go Go client and worker (module github.com/synaptiai/cronoq/sdks/go)
sdks/go-embed Go wrapper of the embedded library (module github.com/synaptiai/cronoq/sdks/go-embed)
sdks/swift Swift wrapper of the embedded library (package Cronoq)
sdks/typescript TypeScript client (Node and browsers) and Node worker (package @synaptiai/cronoq)
deploy Example server configuration, and the Helm charts for the server and the operator with their end-to-end tests
docs The book's source, the design document and the original Python prototype

Community

License

Licensed under either of

at your option. Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.

About

Elastic, priority-aware task scheduler for Rust, Python and gRPC: aging priorities, deadlines, cron and fixed starts, DAGs, retries, rate limits, queue-pressure autoscaling and Postgres-backed HA.

Topics

Resources

Code of conduct

Contributing

Security policy

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages