We evaluated four workflow engines, here's what we found.
Early in Suga's development, deployments were synchronous. A user would trigger a deploy from the dashboard, and our API would handle it inline. This works in theory, but in practice, a user can easily just refresh their browser mid-deployment, and all the context would be lost. From their perspective, the deployment would have just stopped. No state, no progress, no way to tell if it had succeeded or was still running somewhere in the background.
We needed a proper workflow engine. Something that could maintain execution state independently of the client, survive server restarts, and let a user close their laptop, make a coffee, and come back to find their deployment exactly where they left it.
We evaluated four options: Temporal, Restate, Vercel Workflows, and Hatchet.
What we were looking for
Suga is a container-based PaaS. We needed something that could be self-hosted with low operational overhead, because we're a small team and can't afford to babysit a complex cluster of services just to run our orchestration layer. Authentication had to be baked in, not bolted on. And we wanted a model that matched how engineers actually think about deployment pipelines, where the shape of the work is knowable upfront.
Vercel Workflows
Vercel's Workflow Development Kit (WDK) has the most approachable API of the four. It uses JavaScript directives to make ordinary async functions durable with no new abstractions:
export async function deployWorkflow(input: DeployInput) {
"use workflow";
await provisionNamespace(input);
await buildImage(input);
await pushToRegistry(input);
await applyManifests(input);
}
async function provisionNamespace(input: DeployInput) {
"use step";
// provision the tenant namespace
}
If you're already on Vercel, this is easily the quickest path to durable execution available to you right now.
The directives do have a rough edge though. They're just string literal expressions. TypeScript has no idea what "use step" means. Correctness is enforced by a build-time esbuild transform rather than the type system, which has two practical consequences. First, calling a step function outside of a workflow context won't produce a type error. Second, and more subtle, steps are identified by their position in the compiled output rather than an explicit key, which means refactoring or deploying mid-execution can break running workflows in ways that are hard to observe. Inngest, who built a similar directive-based system early on, wrote about exactly this problem after moving to an explicit API model.
I'm just realizing as I write this that we completely left Inngest out of our evaluation 🤦
We're also building Suga as an independent platform. The WDK's persistence, queuing, and routing are all managed by Vercel's infrastructure, and while we do host some of our infrastructure with Vercel, that may not be the case forever.
Restate
Restate caught our attention early, we'd heard about it from a user of the Nitric framework who was building out their own durable workflows with it. The demo on their homepage was a fun way to demonstrate workflow/step replay and I am a sucker for mini-games on homepages.
The push-based model was also compelling for us. Restate acts like a reverse proxy that invokes your service handlers directly, which meant we could potentially reuse our existing Vercel-hosted backend without a major architectural shift. The programming model is relatively clean too. Durable steps are just ctx.run() calls inline in a regular handler:
const deploy = restate.workflow({
name: "deploy",
handlers: {
run: async (ctx: restate.WorkflowContext, input: DeployInput) => {
await ctx.run("provision", () => provisionNamespace(input));
await ctx.run("build", () => buildImage(input));
await ctx.run("push", () => pushToRegistry(input));
await ctx.run("apply", () => applyManifests(input));
},
},
});
Two things killed it. The admin plane has no user authentication or RBAC story for self-hosted deployments. The RESTATE_AUTH_TOKEN in the docs is a Restate Cloud concept. Running it yourself, the admin API and UI are wide open, and for a platform with multiple tenants that's not a gap we could paper over easily. The UI was also immature at the time of our evaluation. The graphical dashboard only arrived in Restate 1.2, and observability features we'd consider table stakes were still being added in 1.5.
To be clear their trajectory looks good, and there remains a strong case for push based workflows orchestration as it serves serverless platforms. So it's good to see an open-source alternative to something like Vercel Workflows that serves users in this space. It's just not quite what we needed for our use case.
Temporal
Temporal is the mature choice. Large ecosystem, sophisticated versioning mechanisms, proven in serious production environments. We got it running, and it works.
The problem was what the programming model demanded from our team.
Temporal workflows must be deterministic. Every time a workflow function executes, it must produce the same sequence of API calls in the same order, because that's how Temporal replays execution history after a failure. In practice this means your workflow code runs in a constrained sandbox with no Node built-ins, no fs, no path. You can't call an activity directly either. You call a proxy that schedules it in the system:
const { buildImage, pushToRegistry } = proxyActivities<typeof activities>({
startToCloseTimeout: '10m',
});
export async function deployWorkflow(input: DeployInput): Promise<void> {
await buildImage(input);
await pushToRegistry(input);
}
The workflow/activity split means two separate conceptual layers, two separate files, and the proxy pattern is non-obvious to anyone who hasn't internalized the replay model. The versioning story compounds it: if you change the order of API calls in a live workflow, in-flight executions fail with a non-determinism error, so any meaningful code change requires versioning annotations.
All of that makes sense for the problems Temporal was built to solve. Workflows that run for months, complex branching with replay guarantees across code changes. But for deployment pipelines with a knowable structure and bounded execution times, it's a lot of ceremony for problems we don't have. Temporal is extremely powerful, and it felt over-engineered for what we needed.
Authentication is also solvable but not simple. Temporal provides a pluggable ClaimMapper and Authorizer system, configured as Go server options. A default JWT ClaimMapper exists, but without an Authorizer configured the server runs with a no-op that allows all requests. Getting to production means wiring up an OIDC provider, configuring the Authorizer, and optionally setting up mTLS. It's all well documented, but it's more plumbing we'd need to own alongside the rest of the deployment.
Honestly dealing with the proxy activity pattern in practice was a painful experience. The additional complexity created by this indirection just didn't feel worth the advantages, there are other ways to achieve idempotency that don't involve what feels like over-engineering.
Hatchet
Hatchet's model maps directly onto how we think about deployment pipelines. Workflows are declared as explicit DAGs, tasks with typed inputs, typed outputs, and dependencies expressed as direct object references:
const deploy = hatchet.workflow<DeployInput>({ name: 'deploy' });
const provision = deploy.task({
name: 'provision-namespace',
fn: async (input) => { /* ... */ },
});
const build = deploy.task({
name: 'build-image',
parents: [provision],
fn: async (input) => { /* ... */ },
});
const apply = deploy.task({
name: 'apply-manifests',
parents: [build],
fn: async (input) => { /* ... */ },
});
I evaluated Hatchet after Temporal and was blown away by it's simplicity after having spent a more time than I'd like to admit wrestling with Temporal's mental model.
This is just TypeScript. No proxy pattern, no sandbox constraints, no determinism rules to internalize. You can unit test a task function by calling it directly. The parents array is typed, so if you rename a task the reference breaks at compile time rather than at runtime with a cryptic replay error.
Self-hosting is a single hatchet-lite container backed by Postgres. Authentication is first-class configuration. The Helm chart exposes email/password auth, cookie domain settings, and admin seeding as standard values with no custom code required.
We ended up on Hatchet Cloud rather than self-hosting in production, but the strength of the self-hosting story was a big part of what got us there. Knowing we could run it ourselves if we needed to, gave us a confidence in the product that the others didn't.
After choosing Hatchet we had a proof of concept running the same day. The path from prototype to production was short enough that we could ship reliable deployment pipelines without a dedicated sprint on workflow infrastructure.
The deployment that originally lost its state mid-browser-refresh now survives server restarts, client disconnections, and infrastructure blips. A user can trigger a deploy, close their laptop, and come back to find a precise record of every step: what ran, what succeeded, what failed, and why, in a dashboard that reflects exactly what their workflow was supposed to do.
Temporal is the right tool for certain problems. If you're building workflows that run for months, need fine-grained replay semantics across code changes, or require multi-database backend support, it's probably worth the investment.
For us, Hatchet gave us the durability and observability we needed without asking our team to adopt a new programming model to get there. For a small team with a lot of ground to cover, that tradeoff matters.