- n8n / Zapier / Make / Power Automate / Pipedream — event-driven glue with connectors; ideal for business automations and quick internal tools. - Airflow / Prefect / Kestra / Dagster — scheduled data pipelines with backfills and lineage; right for batch embedding and enrichment jobs. - Temporal — durable execution: workflow code survives process crashes, with retries and timeouts as first-class concepts. Right for long-running, human-in-the-loop AI workflows.
ts
// Temporal workflow: durable, resumable, deterministic
export async function enrichLead(lead: Lead) {
const profile = await activities.fetchProfile(lead.domain); // retried automatically
const summary = await activities.summarise(profile); // AI activity, idempotent
const approved = await condition(() => signals.approved, "72h"); // waits days if needed
if (approved) await activities.writeCrm(lead.id, summary);
}Rules for AI steps inside any runner: make them idempotent with a natural key so a retry cannot double charge or double email; cap concurrency to stay inside rate limits; persist intermediate output so a retry does not re-run an expensive generation; and check a paused flag first when credits are exhausted.
Exercise: build a nightly job that embeds new documents in batches of 50 with an idempotency key per document, then re-run it and verify zero duplicate embeddings and zero extra spend.