The human-in-the-loop design most transformation programmes get backwards
A colleague asked me a version of this question that I think most enterprise AI rollouts quietly get wrong: if the agent can see all my tasks and reminders, why can't it just run my backlog for me? I've watched this fail in practice for two structural reasons, and neither one is "the model isn't good enough yet."
The first reason is that an agent cannot reliably tell you when a task is actually complete. This sounds like a solvable engineering problem until you look at how these systems actually fail. Large language model agents are built to always return something. They produce an answer that looks finished when they run out of the context needed to finish a task properly. A study analysing over sixteen hundred multi-agent traces found roughly three quarters of failures were silent, visible only if a human went and looked manually, not surfaced by anything in the system itself. Wrong looks exactly like done, from the outside, and a backlog that lets the agent self-certify completion will accumulate exactly that kind of quiet rot: items marked closed that were never actually resolved, discovered only when someone downstream notices the thing they were promised didn't happen.
The second reason is subtler and, in my experience, the one that actually kills adoption of these tools day to day. Capture only relieves anyone's mind if it's frictionless and if they trust it will resurface at the right time. The moment a person has to start second-guessing whether the system caught something, or whether it filed it correctly, they've added a decision back into a step that was supposed to remove decisions. This is old wisdom, not new. Getting-things-done doctrine has said for two decades that the instant you start making judgement calls at the point of capture, you've broken the thing that made capture valuable in the first place. An agent that tries to be clever about parsing intent at the moment of capture, rather than just writing it down instantly and dumbly, reintroduces exactly the friction the whole system exists to remove.
The fix that actually holds up under both failure modes is a clean separation of roles, and it's less exotic than it sounds. A deterministic tool of record, something like a ticketing system or a project board, owns task state and owns what counts as complete. State only changes there, and only through an explicit, auditable action, a human click or a defined workflow transition, never a probabilistic judgement call. The agent's job is narrower than "manage my tasks." It captures, it surfaces, and it can act, meaning it can genuinely write into that system of record: draft a ticket, add a comment, propose a status change. It proposes, never committing completion on its own authority. The system of record, and a human, confirm.
The correction I had to make in my own head, going into this, was assuming the agent layer had to be read-only to be safe. It doesn't, and treating it that way undersells what's actually useful about it. A capable agent can genuinely write into your system of record, under your own permissions, the same way a colleague with appropriate access could. The safety comes from denying it the authority to declare something finished. Those are different guardrails, and conflating them either makes the system too restrictive to be useful, agent as read-only dashboard, or too permissive to be trustworthy, agent silently closing things it shouldn't.
The uncomfortable part of this design, for anyone selling the vision of an agent that just runs your life, is that it explicitly refuses to let the agent be the source of truth about anything that matters. It's the actual design principle that makes the rest of the system trustworthy enough to use daily. An agent that proposes into a system a human commits to is something I'll rely on constantly, because I know exactly where the seam is between what it decided and what I decided. An agent that quietly owns the truth about what's done is something I'll stop trusting the first time it's confidently wrong, and once that trust breaks, people go back to running everything from memory, which is the exact failure state the whole tool was supposed to fix.