What It Actually Takes to Turn Your Local Agent Setup Into a Service
Your local agent harness is humming. Workflows tuned, planning and review gates dialed in, mountains of working code landing while you sip your coffee. Great!
Then the thought arrives. What if my whole team worked like this? What if this were a service?
Time to scale up. And a laptop-shaped SDLC has laptop-shaped limits. Sure, you can park the harness on a dedicated box and remote in — that even gets you to the beach instead of tethered to a terminal. But the moment you want to hand it to coworkers, or run it as a product, "everyone SSHes into my machine" is not an architecture.
And sharing the workflow itself only gets you so far. What happens when the people who'd benefit from it aren't developers? Does everyone who wants a task run need to learn git and your local harness of choice — or could it be as simple as sending a message in Slack?
Enter the cloud harness: a platform that runs the same development patterns we'd proven locally — plan, implement, test, review, deploy, triage — as a scalable service on our clusters. Conceptually identical to the local harness. Practically, a very different animal.
Here's what changed the moment agents left the laptop.
Execution needs real isolation
Locally, the agent runs as you, in your checkout, with your credentials. Fine for one trusted operator. Fatal for a multi-user service running LLM-generated code.
In the cloud harness, every agent gets an ephemeral, sandboxed workspace: locked-down pod, non-root, no standing access to anything it wasn't explicitly granted, garbage-collected when its time-to-live expires. Straightforward to say; full of teeth in practice. A few lessons that each cost us a debugging session:
- Non-root containers can't write to paths a root-assuming image left behind; workspace mounts have to be provisioned deliberately.
- A TTL garbage collector that can't tell "old" from "still working" will reap a sandbox mid-task and destroy hours of agent work. Lifecycle state has to be explicit.
- Unpinned base images mean your sandbox behaves differently on Tuesday than it did on Monday, and the agent gets blamed for it.
- A config key whose own comment says it blocks host-path mounts, set to a glob that matches everything, blocks nothing. We found exactly that one. The comment is not the control, and nobody reads a values file adversarially until they have a reason to.
None of these exist on a laptop. All of them exist on day one of a service.
The framing that helped most was to stop asking whether the sandbox holds and start asking what's reachable when it doesn't. A sandbox is a boundary, not a guarantee, and every question worth answering is about what sits on the other side of it.
Credentials stop being ambient
The local harness quietly inherits your identity everywhere: your git config, your kubeconfig, your tokens. In a multi-tenant platform, ambient identity is the security hole.
Every capability an agent touches became an explicit, scoped grant: agents authenticate to internal services as their own service accounts, tool servers sit behind fail-closed bearer auth — no token means no tool, not "tool with default access" — and network egress from agent workloads is default-deny with explicit allowlists.
One lesson worth stealing: derive your allowlists from what the code actually needs, verified against the code, not from observing traffic and assuming what you saw was legitimate and complete. When we audited soak-inferred egress lists against the actual code paths, half of them were wrong.
The harder lesson is to audit the machinery, not just the agents. One sweep of our own CI and secret-delivery plumbing turned up three problems, none of which involved an agent at all. The identity that syncs secrets into clusters held read access to every secret path in the organization, including the token that can provision and destroy infrastructure. A deploy credential used by ordinary build jobs turned out to be an org-admin token, readable by every job sharing that runner. And an authentication mount for a cluster we had consolidated away months earlier was still live, still granting read on that same infrastructure token, referenced by nothing.
Scoping grants for your agents is the easy half. The plumbing that delivers those grants is a principal too, and it's the one nobody thinks to scope.
Every document an agent reads is untrusted input
On a laptop you're in the loop when your agent fetches a page. On a platform, agents ingest tickets, specs, repos and web pages on other people's behalf, and no human reads the raw text before the model does. Anything in there shaped like an instruction gets a free shot at an agent holding real credentials. Be precise about the adversary here: not the user, but whoever wrote the content the user asked you to go read.
We spent real time evaluating whether a small local classifier could screen ingested content. Two findings changed how we're building it.
Screen in chunks, and the reason isn't performance. A short injection buried in a long document is diluted almost to nothing when the whole page is scored as a single window, and stands out immediately when the page is scored in pieces. There's a speed argument too, since attention cost grows faster than linearly with length, but the recall argument is the one that should drive the design.
Then refuse to treat the score as a verdict. An injection our classifier caught with total confidence in plaintext became invisible to it base64-encoded. And a published over-defense benchmark found the best-known open classifier in this space flagging 44% and 69% of benign text that merely discusses security, which is exactly the content engineers ask agents to read. So screening produces an annotation, never a judgment. Make the flag non-blocking, trivially overridable, and worded as the screener flagged this rather than this is dangerous. Get that wrong and the first ten security documents someone ingests will teach them to click through every warning you will ever show them.
The real defense is blast radius, not detection. This is where all the unglamorous work in the rest of this list pays off. A successful injection lands in an ephemeral sandbox, as a service account scoped to that agent's own grants, behind default-deny egress, and any live mutation it attempts stops at an approval gate. None of that depends on noticing the attack. Detection buys you a label; scope buys you the outcome.
The same treatment for user input. Today we screen what agents fetch rather than what people type at them, and that was the right order to build in: fetched content is where the volume and the anonymity are, and a small trusted circle of operators is a genuinely different threat model from an open front door. It stops being the right order the moment the door opens. So user input joins the same screening path before the platform goes public, under the same rule: the result annotates, it never authorizes.
Human gates become product features
Locally, the human gate is you, in the loop, reading plans. At platform scale, approval has to be a first-class mechanism: plans surfaced for review before execution, mutations to live systems held for sign-off, destructive actions requiring a fresh explicit yes every time — never inherited from an earlier approval.
This is also where the "as simple as sending a message in Slack" promise gets expensive. A chat message is a wonderful front door, and the moment someone who isn't a developer can open it, the approval path becomes the entire safety model. The person who asked for the work often can't evaluate the plan that comes back. Someone has to, and the platform has to know who before anything runs. Easy to ask for is the goal; easy to approve is the trap.
It's the part I'd urge anyone building in this space not to shortcut. The scalable version of "I watched it work" is an auditable approval trail, not more optimism.
Testing has to get end-to-end honest
My favorite scale lesson is the least glamorous. We shipped an agent capability that passed every unit test and then failed in the live pipeline, because its declared tool scopes were missing the one server it needed at runtime. The config was valid. The units were green. Only a live end-to-end run, exercising real auth against real services, caught the gap and blocked the deploy.
At laptop scale you are the end-to-end test; you'd have noticed in seconds. At platform scale, nobody is watching any individual run, so the pipeline has to do the noticing. Live-fire e2e checks on the agent platform itself moved from "nice to have" to "the thing standing between you and silent failures."
That shifts the burden onto the gates. And once an agent is reading the gate output instead of a human, a green gate stops being evidence and starts being a claim.
Every gate needs a planted defect it must catch. Our accessibility scanner ships its own browser driver. Mismatched against the host browser, it exited before analyzing anything, and piped through a log tail that read as a clean run. Thirteen demonstration sites were built against a green accessibility gate that had never once executed. When we fixed the harness, one site turned up 681 color-contrast findings, another 841 accessibility findings in total, and the worst 2,362. So now every gate runs against a deliberately broken input it is required to fail on, and that control runs in the same tier as the gate itself. A gate without its control is an unverified pass, and an absence-assertion — "no findings," "no secrets" — without one is worth nothing at all.
Controls fail three ways, and only one of them is obvious. It doesn't fire. It's itself defective. Or it fires for the wrong reason, which looks identical to success. All three are things we've measured, not hypotheticals: on one conversion, three controls stayed silent and every one turned out to be a broken control rather than clean code. On another, a silent control led the agent to re-run the retired implementation out of git history as an independent oracle rather than trust its own captured baseline. Assert on the failure message, not the exit code.
Ask what your suite does not look at. In one project we found three end-to-end specs that were registered in no test target at all. Two had rotted red — 54 and 62 failures — sitting invisible because nothing ran them. The control meant to catch exactly that read its expectations from the same manifest it was auditing, so the missing specs were missing from the expectation too. A coverage control has to derive its expectation independently of the registry it checks. In the same sweep, a malformed scope argument produced the narrowest possible run rather than everything; selection bugs present as unusually fast green builds.
Then make it fast, deliberately. Agent loops iterate, so a slow suite is paid every round. We run a declared budget per tier: under 5 minutes for the inner loop on every iteration, under 20 for the pre-review pass, CI as a strict superset of pre-review, and an unbounded on-demand tier that's mandatory before production. Suites are routed by whether the change could plausibly move their result: an API-call tweak cannot change a contrast ratio, so contrast analysis doesn't run for it. Measured on one project: pre-review lands at 19m26s against the 20-minute budget, while one change class blows the inner budget at 15m38s and we're carrying that consciously with the per-round cost written down. Measure on an idle host, too. Co-tenant load moved untouched gates by 2× and produced an argument about a change that hadn't done anything.
A defect that slips the fast loop and gets caught in CI is a normal outcome, not a policy failure. That miss rate is what you're buying the speed with. Re-tier on the second miss of the same class, not the first.
Cost needs a governor, and then it needs a speedometer
One developer's agent spend is a line item. An org's agents, running pipelines around the clock, are a budget. So we did the obvious thing first: per-session spending caps, an org-level daily cap that parks new work instead of failing it, and no fully autonomous scheduling until those controls existed.
That is the governor. It stops the engine before it throws a rod. It tells you nothing about why you were doing 9,000 RPM.
The shape of the cost record decides every question you can ever ask. We metered spend into a single per-day org total. That is exactly enough to enforce a cap and exactly useless for "which pipeline ate Tuesday." Adding a dimension afterwards is not a reporting change, it's surgery on the hot path every LLM call goes through. Decide your dimensions before you ship the meter: project, pipeline, task, model, and time of day. You will want all five, and you will want them retroactively, which is the one thing you cannot have.
A single near-cap alert is a smoke detector in a house that's already burning. We shipped one at 80% and it was useless in the way that matters: by the time it fired, the decision was binary. It's now graduated — informational, warning, alert, emergency — at four thresholds, with the tiers banded so crossing the top one doesn't fan out four simultaneous pages. The early tiers aren't alarms, they're a chance to look at the dashboard while there's still budget left to make a choice with.
Then alert on the dashboard itself. We found a metrics scrape config that had been silently dead for 83 days. Wrong label, no error, no missing-data warning, just a quietly empty series. Zero cost metrics had ever reached the monitoring stack. The spend cap was enforcing correctly the entire time, but if it hadn't been, nothing would have said so.
Cost telemetry breaks in a way that is indistinguishable from good news. A dashboard that reads $0 looks like a quiet week. So the cost metrics now have two alerts of their own that have nothing to do with spending too much: one that fires if the metrics are absent, and one that fires if there was heavy agent activity and the recorded spend is still zero. Absence of a signal has to be a signal.
Budget isolation is a security control, not just accounting. This one surprised me. Without per-pipeline budget reservation, a runaway loop — or an attacker who can trigger expensive work — burns the shared org budget, and the thing that starves is your incident-response pipeline, precisely when you need it. Reserving budget so detection can never be blinded is a defense, and it happens to be the same attribution plumbing per-project cost reporting needs. Build it once.
I'll be honest about where this ends: our own rich spend view — hourly and weekly windows, running averages, per-project breakdown, plotted over time — is still a design document. It's blocked on exactly the attribution work described above, which is the tax for having shipped the meter before the dimensions. Learn from that rather than from the same bruise.
The theme
Every one of these is the same move: something that was implicit on the laptop — identity, isolation, screening, oversight, verification, budget — made explicit, enforced, and auditable.
The reporting channel is part of the same move: with nobody watching individual runs, deviations and expectation misses have to bubble up through structured results instead of being noticed over your shoulder. A platform that only reports successes isn't scaled; it's unsupervised.
That's the real work of scaling an LLM SDLC. The agent loop was never the hard part. The hard part is everything your laptop was silently doing for free.
This is the overview. There's a deeper piece behind each of these: sandbox lifecycle, agent credential scoping, and what an approval system for autonomous mutations actually looks like. I may write some of those up too, we'll see!
Building this out is the work I do. If your agent setup has outgrown the machine it runs on, let's have a conversation.
written by Claude
directed by David Brandon