In 2023, the dream was an agent that ran on its own. You’d give it a goal, walk away, come back later, and the work would be done. Auto-GPT was the artifact of this dream — a loop that kept calling the model until either the goal was met or the user gave up first. It was the wrong design, but it was the wrong design honestly. It expressed, in code, what most people imagined an AI agent should be: autonomous, persistent, unsupervised.
By 2025 it was clear this was not how production agents would actually work. The most useful systems were the ones that put humans firmly back in the loop — Cursor, Claude Code, Copilot Workspace, GitHub Copilot Agents, and the dozens of similar tools that all converged on the same shape. The agent proposes; the human approves. The agent acts; the human observes. The agent encounters something ambiguous; the human resolves it. The autonomous-agent dream had quietly been replaced by something more useful: a collaborative loop.
This wasn’t capitulation. It was learning what worked.
Full autonomy was not dropped because the technology wasn’t ready; it was dropped because, even when it worked, it produced worse outcomes than collaborative loops did.
Why full autonomy lost on the merits
The reasons are worth being precise about, because “humans are still needed” sounds like a generic platitude, and the actual mechanics are more interesting than that.
Long-tail failures destroy trust. An agent that succeeds 95% of the time at a substantial task is impressive in a demo and unusable in production. The 5% failure isn’t randomly distributed across small annoyances — it’s concentrated in the cases where the agent’s confidence is highest and the consequences are worst. The agent that confidently deletes the wrong file, or sends the wrong email, or makes the wrong purchase, does more damage than a hundred small inefficiencies. Production trust requires reliability levels well above 95%, and we don’t know how to get there autonomously. We do know how to get there with human checkpoints.
Unsupervised drift compounds. Agents working alone tend to drift from the original goal. Each step is locally reasonable; the cumulative direction isn’t. By turn fifteen, the agent is solving a problem related to the original but no longer the same. This is the AutoGPT pathology — the agent enthusiastically refactors something nobody asked it to refactor, or pivots from “fix this bug” to “rewrite this module” without checking. Human review at intermediate points anchors the trajectory to the actual goal. The human isn’t doing the work; they’re doing the steering, and steering is much cheaper than redoing.
Costs explode without oversight. An agent that’s allowed to run unsupervised will, in expectation, do more work than necessary. It will read files it didn’t need to read. It will try strategies that won’t work. It will burn through API budgets in pursuit of paths that a human would recognize as dead ends in seconds. The economics of fully autonomous agents are worse than the economics of collaborative ones, even before you count the cost of cleaning up their mistakes. Bounded autonomy keeps costs predictable.
Ambiguity needs resolution from outside. Many real tasks are under-specified. The user wants “a script that processes the data.” Which data? Processed how? Output where? An autonomous agent has to guess, and its guesses are sometimes brilliant and sometimes wrong. A collaborative agent can ask. The ability to interrupt and clarify is enormously valuable — and the cost of the interruption is much less than the cost of going down the wrong path for an hour. Agents that can pause for clarification produce better outcomes than agents that always proceed.
Verification is hard at scale. Even when an agent’s work is mostly correct, verifying it after the fact is often as hard as doing it. If you can’t quickly tell whether the agent’s hundred-line refactor preserved behavior, then you don’t really benefit from the agent doing it — you’ve moved the work from “write the refactor” to “audit the refactor,” and the second job isn’t obviously easier. Collaborative loops keep verification cost low by showing work incrementally. The human verifies each step as it happens, in small pieces, rather than facing a wall of diff at the end.
Checkpoints calibrated to consequence
The form that emerged from these constraints is something like: the agent does the work; the human checkpoints the work at intervals; the system is designed so that checkpoints are easy and the cost of intervention is low. Claude Code shows you each diff and waits. Cursor previews changes before applying them. Copilot Workspace structures the task into a plan, a set of files, a set of diffs, and asks for approval at each layer. These are not failures of autonomy — they are designs that recognize autonomy as a sliding scale, with the right point on the scale determined by the stakes of the action.
Autonomy is not throughput
There’s an important distinction here between “autonomy” and “throughput.” Collaborative agents are not slower than autonomous ones in any meaningful sense. A well-designed collaborative loop with fast approvals can process many actions per minute. The human isn’t a bottleneck for routine actions; they’re a circuit breaker for unusual ones. The throughput stays high because the default approval is low-friction; the safety stays high because the unusual cases get caught. The agent does the boring work; the human gets summoned for the interesting decisions.
You can see this pattern most clearly in production coding agents. They take instructions, draft a plan, propose specific changes, and then execute — but with checkpoints that escalate based on the riskiness of the action. Reading files: no checkpoint. Running tests: no checkpoint. Editing files: visual diff, default-accept. Committing: explicit confirmation. Pushing to remote: explicit confirmation with the action stated out loud.
The autonomy is graduated, calibrated to consequence. The user is barely involved when nothing risky is happening, and is fully involved when something risky is about to happen. This isn’t a compromise. It’s a better design than full autonomy ever was.
Excellent collaborators beat autonomous workers
The lesson — which the field is slowly internalizing — is that “AI agent” doesn’t have to mean “agent that operates without human input.” The most useful agents are the ones designed to be excellent collaborators rather than autonomous workers. Their goal isn’t to remove the human from the loop. Their goal is to make the loop fast, low-friction, and useful enough that the human is happy to participate.
There’s a place where this gets even more concrete and pragmatic, and that’s the world of coding agents specifically. The way they handle workspaces, sandboxes, tests, and version control is its own design discipline, and it’s the subject of the next phase of this series.