How to Automate Fly.io Operations with Kestrel Workflows: Incident Response, Machine Provisioning, CI/CD, and Security
Fly.io makes running apps close to your users effortless; operating those apps is still on you. The teams that run Fly.io well automate the recurring 80% — incident response, machine provisioning, developer requests, CI/CD, and security — as deterministic, approval-gated workflows. Here's how to build all five with Kestrel Workflows, with copy-paste prompts for each.
Every team running production apps on Fly.io carries the same operational load. A Machine crashes at 2am and someone spends an hour digging through events to find out why. All of an app's machines go down and the one person with flyctl access is asleep. A deploy leaves a machine stuck in a crash loop. The secret rotation that's been on the backlog for two quarters still hasn't happened. And the staging machines that nobody stopped are quietly burning money overnight.
None of this is a Fly.io problem — it's an automation problem. Each of those tasks is a known, repeatable sequence of steps that happens to be executed by a human with terminal access. The way to make Fly.io easier to operate is not more flyctl access; it's codifying those sequences as workflows that run the same way every time, with a human approving anything that has blast radius.
This post shows how to automate the five domains of Fly.io operations with Kestrel Workflows: incident response, machine provisioning, developer requests, CI/CD, and security. Each section includes three prompts written the way you'd hand them to the Workflow Agent — paste one in, review the workflow it builds, and activate it.
How it works: plain English in, deterministic workflows out
You describe the automation in plain English and the Workflow Agent assembles it from real building blocks — Fly.io's machine operations (Restart, Start, Stop, and Suspend Machine, Get Machine, Get Machine Events, List Machines, Set Secrets, Cordon and Uncordon Machine, and AI-powered Investigate Fly.io) plus the stack around it: GitHub pull requests and Actions, Cloudflare DNS, causal PR search, Slack, PagerDuty, and Jira. The AI builds the workflow once; the workflow itself runs deterministically — the exact same steps on every execution, not an agent improvising at runtime.
Fly.io has no outbound webhooks, so Kestrel detects Machine lifecycle events by polling the Fly Machines API on an interval you configure — there is nothing to set up inside Fly.io itself. Workflows start from the trigger that fits the job: Machine Crashed, Machine Stopped, Machine Started, App Down, CI events, schedules, or plain-English developer requests. And every workflow runs inside Kestrel's access control and approval model — scope it to specific Fly apps, restrict who can invoke it, and drop approval gates anywhere in the graph so nothing touches production until the right person signs off in the dashboard, in Slack, or by merging a PR.
The Workflow Agent building a production-ready workflow from a plain-English description.
How to automate incident response in Fly.io
Fly.io incidents come in three shapes: the Machine that crashes and needs a root cause, the app that goes fully down when every machine stops, and the machine that quietly stops for a reason nobody looks into until it matters. All three follow the same split — the investigation is safe to automate fully, and the remediation gets an approval gate.
When a Fly Machine crashes, fetch the machine's state and its recent event history, run an investigation to determine the root cause of the crash, and post the analysis to #incidents in Slack. Send a machine restart to a Slack approval gate for the on-call lead; on approval, restart the machine, fetch its state to confirm it is back to started and healthy, and add a note to the PagerDuty alert.
When a Fly app goes down — all of its machines stopped or unhealthy — list the app's machines, investigate why they went down, and post the findings to #incidents in Slack. Send a recovery to a Slack approval gate; on approval, start the stopped machines and list the machines again to confirm the app is back up. If machines are still not running after the recovery attempt, page the on-call engineer through PagerDuty with the investigation findings attached.
When a Fly Machine stops unexpectedly, fetch the machine's event history and classify why it stopped — an out-of-memory kill, a host migration, or Fly's auto-stop — and post a plain-English explanation to #platform in Slack. If the cause was an out-of-memory kill, create a Jira ticket to track the memory issue for the owning team. Read-only, no approval gate.
How to automate machine provisioning in Fly.io
Provisioning on Fly.io lives in three places: machine counts and regions in fly.toml, app credentials in secrets, and cost in whatever is still running when nobody's using it. The automation should follow that shape — capacity changes go through pull requests and CI, configuration goes through approval-gated secret updates, and cost control goes on a schedule.
Every weekday at 8pm, list the machines in our staging Fly apps and stop the ones that are running; every weekday at 7am, start them again and confirm they come back to a started state. Post the summary — how many machines were stopped or started, per app — to #platform in Slack. No approval gate; these are non-production apps.
When a team requests more capacity or a new region for a Fly app, generate the edit to the app's fly.toml — machine count, sizes, or the added region — and open a pull request with the change. After the PR is merged, trigger the GitHub Actions workflow that runs flyctl deploy, wait for it to complete, then list the app's machines to confirm the new machines are started in the right regions and reply in Slack with the result.
When a team requests new configuration for a Fly app — API keys for a new integration, a changed connection string — send the change to a Slack approval gate for the platform team. On approval, set the secrets on the app, restart the app's machines so they pick up the new values, confirm the machines come back healthy, and reply to the requester with the confirmation. Never post the secret values themselves to Slack.
The best way to automate developer requests in Fly.io
Machine requests are the most time-sensitive requests a platform team gets — a stuck app needs a restart now, and finding out why a machine crashed last night shouldn't require pinging the one person with flyctl access. The fix is a plain-English front door. Developers ask in Slack with /kestrel-workflow, from the CLI, or on the Developer Requests page; Kestrel matches the request to an active request-triggered workflow, extracts the parameters from the phrasing, and runs it behind the approval gates and app scoping you defined.
When a developer asks to restart a stuck or misbehaving Fly app, restart the machines for the named app, fetch each machine's state to confirm it is back to started and healthy, and post the result to the #deploys channel and the requester in Slack. Require approval from the on-call lead for production apps, and restrict this to the apps the requester's team owns.
When a developer asks why their app crashed or restarted, fetch the event history for the app's machines and run an investigation across machine state, events, and lifecycle history, then post a plain-English explanation of the root cause back to the requester. Read-only, so allow it for all developers with no approval gate.
When a developer asks to drain a Fly Machine for maintenance, cordon the named machine so it stops receiving traffic, confirm its state, and reply to the requester that it is safe to work on. When they ask to bring it back, uncordon the machine, confirm it is receiving traffic again, and post the result to #platform in Slack. Require approval for production apps.
This is the same pattern our internal developer platform guide builds across the whole stack — Kubernetes, AWS, PlanetScale, and more. Here it's scoped to Fly.io, but the request workflows compose: the same front door serves every integration you connect.
How to automate CI/CD in Fly.io
Most Fly.io teams deploy with flyctl deploy inside GitHub Actions. The gaps are everything around that: verifying the deploy actually left every machine healthy, recovering when the deploy workflow breaks, and rolling out config changes without a blast-radius moment. These three workflows close those gaps.
When our flyctl deploy GitHub Actions workflow completes on the main branch, list the machines for the deployed Fly app and fetch each machine's state. If every machine is started and healthy, post a deploy-verified summary to #deploys in Slack. If any machine is crashed or stopped, investigate why, post the findings to #deploys, and send a machine restart to a Slack approval gate for the on-call lead.
When our deploy workflow fails in GitHub Actions, fetch the failed run's logs, generate a code or configuration fix for the failure, and open a pull request with the fix and the failure analysis attached. Post the PR link to the owning team's Slack channel. No approval gate — the fix goes through normal code review.
When a config release PR is merged for our production Fly app, roll the change out machine by machine: cordon the first machine to drain its traffic, restart it, confirm it comes back healthy, uncordon it, and repeat for each remaining machine. Post the rollout progress and final result to #deploys in Slack. Require a Slack approval from the release owner before the first machine is touched.
How to automate Fly.io security
Fly.io security work is mostly hygiene that never gets done by hand: the app secret that hasn't rotated since launch, the machine that appeared in a region nobody expected, and the DNS record still pointing at an app that was deleted last quarter. These three workflows turn that hygiene into a schedule.
Every 90 days, rotate the secrets for our production Fly apps: generate new values for the database passwords and API tokens, send the rotation plan to a Slack approval gate for the security on-call, and on approval set the new secrets on each app, restart the app's machines so they pick up the new values, and confirm every machine comes back healthy. Post the completed rotation summary to #security and log it in Jira.
Every Monday at 9am, investigate our Fly apps for security posture: machines running in regions outside our approved list, machine creation events that don't correspond to a deploy, and machines running images older than our patch policy allows. Post the findings to #security in Slack and create a Jira ticket for each new finding that isn't already tracked. Read-only.
Every Monday at 9am, list the DNS records in our Cloudflare zones that point at Fly.io — CNAME records to fly.dev hostnames and A records to Fly IPs — and check each one against our active Fly apps to find dangling records pointing at deleted apps, the classic subdomain-takeover vector. Post the findings to #security in Slack and create a Jira ticket for each dangling record. Read-only.
Making Fly.io easier to operate
Fifteen workflows, five domains, one pattern: describe the operation in plain English, let the Workflow Agent assemble it from real Fly.io blocks and the stack around them, review the graph, and put approval gates on anything with blast radius. The recurring 80% of Fly.io operations — the crash triage, the restarts, the secret rotations, the overnight staging shutdowns — runs itself, and your team's time goes to shipping.
This isn't Fly.io-only, either. The same workflow engine spans 30+ integrations across infrastructure, PaaS, observability, comms, databases, CI/CD, and IaC — so the workflow that restarts a Fly Machine lives next to the one that resizes an RDS instance or rolls back a Vercel deploy. For deeper dives, see the internal developer platform guide and the companion posts on automating Kubernetes operations, automating AWS cloud operations, automating Vercel operations, and automating Railway operations.
FAQ: automating Fly.io operations
How do you automate Fly.io operations?
Codify each recurring task — incident response, provisioning, developer requests, CI/CD, and security — as a deterministic, approval-gated workflow. With Kestrel, you describe the automation in plain English, the Workflow Agent assembles it from real Fly.io blocks plus the stack around them (GitHub, Cloudflare, Slack, PagerDuty), and it runs the exact same steps every time, pausing at approval gates before anything touches production.
What is the best automation tool for Fly.io?
It depends on scope. flyctl and GitHub Actions automate deploys; Fly's auto-stop handles basic scale-to-zero. For automating operations across the whole lifecycle in one governed system, Kestrel Workflows builds deterministic workflows from plain English, triggered by Machine crashes, stops, and app-down states — detected by polling the Fly Machines API, so there's no webhook setup — with native blocks for Fly.io, GitHub, Cloudflare, Slack, PagerDuty, and Jira.
Is it safe to automate machine restarts and secret rotations?
Safer than doing them by hand at 2am, if the automation is deterministic and gated. Kestrel workflows run fixed step sequences — no runtime improvisation — scoped to specific Fly apps, and approval gates pause execution before any restart, secret change, or production rollout until a human signs off in Slack or the dashboard. Read-only workflows (crash triage, posture audits, DNS checks) can run ungated.
How do you make Fly.io easier to operate?
Reduce how much of Fly.io your team touches by hand. Automate the recurring operations with workflows, give developers plain-English self-service for restarts, investigations, and maintenance drains, and keep humans in the loop only where judgment matters — approvals, not execution.
Getting started
Connect your Fly.io apps — an org-scoped token and a poll interval is all it takes — then pick the domain that hurts most. For most teams that's either the crash channel or the restart bottleneck. Paste one of the prompts above into the Workflow Agent, review the workflow it builds, tighten the scoping and approvals, activate it, and add the next one. The Workflows Quickstart walks through your first workflow end to end, and the Create Workflows guide and Integrations Setup cover connecting GitHub, Cloudflare, and the rest of your stack. Prefer the terminal or an AI coding agent? Workflows can also be created and managed through the CLI, Python SDK, and MCP. Within a few weeks you'll have a governed automation layer over your Fly.io operations that no amount of flyctl access can match.
Automate your Fly.io operations with $1,000 in credits
Incident response, provisioning, developer requests, CI/CD, and security — describe each workflow in plain English and Kestrel builds it. New accounts get $1,000 in usage credits to get started.
Get Started