Spindrel checkpoints every step of an ordinary async function. When a process dies mid-run, another worker picks it up at the step that was in flight, with every earlier result replayed from storage.
npm i @spindrel/sdkNode 20+, Python 3.10+, Go 1.22+. Apache-2.0 engine, hosted or self-run.
Choose when the worker dies, then read the last line.
import { workflow, step } from "@spindrel/sdk";import { payments, inventory, mail, shipping } from "./clients";export const checkout = workflow("checkout", async (order: Order) => { const hold = await step.run("reserveInventory", () => inventory.reserve(order.items), ); // Retried with the same key, so the card is charged once. const charge = await step.run("chargeCard", () => payments.charge(order.total, { idempotencyKey: step.key }), ); await step.run("sendReceipt", () => mail.receipt(order.email, charge.id)); await step.run("scheduleShipment", () => shipping.book(hold.warehouse, order.address), ); return { orderId: order.id, chargeId: charge.id };});spindrel run checkout --input @order.json→ run_01JDX4R7 queued · leased by node-3 ✓ reserveInventory 41 ms hold_3Wk · warehouse RNO-2 ✓ chargeCard 212 ms ch_9fK2 · $86.40 ▸ sendReceipt running… ✗ worker node-3 lost (SIGKILL) during sendReceipt … lease expired after 5.0 s · node-1 picks up run_01JDX4R7 ↺ reserveInventory 0 ms replayed from checkpoint ↺ chargeCard 0 ms replayed from checkpoint ✓ sendReceipt 88 ms attempt 2 ✓ scheduleShipment 63 ms label 1Z-5550-0127✓ completed in 5.49 s · 4 steps · 1 retry · 2 replayed, 0 re-executedThe dashboard is the same data your CLI reads. Five panels from a real-looking Tuesday.
okfailed, retriedsleeping, no worker held
One run, every step, including the retry and the 2-second sleep between them.
| Latency | Executions |
|---|---|
| <10 | 3,120 |
| 10–25 | 11,840 |
| 25–50 | 14,210 |
| 50–100 | 7,930 |
| 100–250 | 3,560 |
| 250–500 | 912 |
| 0.5–1s | 271 |
| >1s | 63 |
41,906 step executions across 7 workflows.
| Column | Type | Note |
|---|---|---|
| run_id | text | pk |
| seq | int4 | pk |
| kind | event_kind | StepStarted… |
| step | text | null for run events |
| payload | jsonb | ≤ 4 MB |
| at | timestamptz |
Append-only. Export it to Parquet or query it in Postgres if you self-host.
Logs are tied to run and step, so you grep by order, not by pod.
tenant:harrow-print
4/4 running212 queued
tenant:kilnworks
2/4 running0 queued
tenant:odd-lot
3/4 running0 queued
tenant:pellmore
1/4 running0 queued
One noisy tenant queues behind its own limit, not everyone else's.
await step.sleep("trial-ends", "14d");The worker returns. No process, timer or open connection waits for two weeks. On day 14 the run is queued again and the next free worker continues from the following line.
await step.run("syncLedger", push, { retries: 6, backoff: "exp" });Failures retry at 1, 2, 4, 8, 16 and 32 seconds, then the run parks as failed with the full error and input. Fix the bug, press Resume, and it continues from this step.
const ok = await step.waitForSignal("approved", { timeout: "72h" });The run parks until something calls spindrel.signal(runId, "approved"): an admin button, a webhook, a chat bot. If nobody answers in 72 hours, ok is false and your code decides.
We ship on Thursdays. Breaking changes get a major version and a migration script.
v2.14.0
Pass a function to concurrency.key, e.g. (input) => `tenant:${input.orgId}`.
v2.13.2
Workers on Node 22 could lose a lease during a stop-the-world pause over 4 s. Renewal now runs on a separate timer thread.
v2.13.0
sp.Run is generic over its return type; no more interface{} casts.
v2.12.0
Nightly export of spindrel.events to Parquet in your own bucket.
A step is one call to step.run, step.sleep or step.waitForSignal. Replays from checkpoint are not billed.
Estimated monthly bill
$42.40
Team plan
| Hobby$0100k steps / monthSide projects and the first prototype. No card. | Teammost teams$40per month, 2M steps included, then $0.30 per 100kProduction workloads for one company. | Scale$0.12per 100k steps above 300M / monthDedicated cluster, your region, a named engineer. | |
|---|---|---|---|
| Included steps | 100k | 2M | Committed volume |
| History retention | 3 days | 14 days | 90 days |
| Environments | 1 | 5 | Unlimited |
| Concurrent runs | 20 | 2,000 | Contracted |
| Max step payload | 1 MB | 4 MB | 4 MB + references |
| SSO (SAML, OIDC) | No | No | Yes |
| Support | Forum | Email, 1 business day | Shared channel, 1 hour |
| Start building, Hobby plan | Start building, Team plan | Talk to an engineer, Scale plan |
6 questions, short answers
One call to step.run, step.sleep or step.waitForSignal. Retries count as steps. Replays from checkpoint after a crash do not.
Wrap the side effects you want checkpointed in step.run. Everything else stays a normal function. Most teams port a cron job in an afternoon.
Steps already running finish and buffer their results on the worker for up to 10 minutes. New runs queue in the SDK and are sent when the API answers again.
Yes. The engine is a single Apache-2.0 binary plus Postgres 15 or newer. The hosted dashboard is the only closed part.
Virginia or Frankfurt, your choice per project. Step payloads are encrypted with a per-project key you can rotate.
Only between steps. Put clocks, random numbers and network calls inside step.run and the replay will match.
Free up to 100k steps a month. Install takes one command and the first workflow about eleven minutes.