Current chapter · Six-part field guide
Part 5 of 6 Building an AI-first development system
If the agent cannot prove it, the work is not finished
A generated diff is a proposal. Completion requires risk-aware evidence that connects functional requirements to tests, browser behavior, and known limitations.
A diff is not evidence, and a green build is not a product verdict.
Generated code is a claim
When an agent produces a change, it is making a claim: this implementation satisfies the requirement without damaging the surrounding system. The code is not the proof of that claim. It is the object being evaluated.
Compilation proves that the language and framework accepted one version of the program. A unit test proves the behavior that test was written to observe. A screenshot proves one rendered state at one moment. Each form of evidence is useful, but none can stand in for the complete product contract.
I want verification planned alongside implementation. The system should know which behavior is risky, which existing guarantees might move, and which evidence will be required before work starts. Adding proof at the end often means discovering that the implementation was built in a shape that is difficult to verify.
A sanitized failure below the desktop breakpoint
In one sanitized interface change, an agent fixed a panel-closing workflow and added a focused test for the component event. The application built successfully, and the desktop path behaved correctly. At a narrow viewport, the visual panel disappeared but its overlay remained active and intercepted taps on the page beneath it.
The implementation had satisfied the component-level claim while violating the actual user requirement: after the action, the user must be able to continue working. No browser check had exercised the interaction at the breakpoint where the layout mode changed.
The artifact I use to close this gap is a verification packet. It links requirement identifiers to checks, routes, states, viewports, roles, evidence, and known limitations. For the panel example, the packet would require the component transition, the narrow-screen interaction, focus behavior, and confirmation that the underlying page becomes usable. The evidence stays attached to the claim it supports.
What I automate
I automate the creation of a risk-aware verification plan from the requirements and the proposed change. The system can select focused tests, type checks, builds, static validation, browser paths, viewports, themes, and permission states. It can inspect console errors, failed requests, unexpected redirects, and visual evidence.
Verification should form a loop rather than a report generated after implementation. When a browser path exposes a failure, the exact state and evidence return to the agent. The implementation changes, the focused proof reruns, and the broader regression checks follow when the local claim is supported.
I also automate honesty about coverage. The final summary should say what was tested, what passed, what changed during verification, and what could not be proved. A vague statement that testing succeeded is less useful than a small, precise evidence map.
What remains a human decision
Humans decide what sufficient proof means for the risk involved. A visual spacing change does not need the same evidence as a permissions boundary, destructive action, data migration, or tenant-sensitive route. The verification system can recommend depth, but the team owns the risk.
A person also evaluates qualities that are difficult to reduce to binary checks. Does the interface preserve orientation? Is a failure state honest? Does the interaction feel predictable? Has the implementation technically followed the requirement while making the workflow worse?
The human role is not to repeat every automated step manually. It is to inspect the meaningful evidence, challenge the missing areas, and decide whether the remaining uncertainty is acceptable for release.
Proof has a cost, so spend it according to risk
Exhaustive verification is impossible. Every combination of data, role, browser, viewport, network condition, and product history would consume more time than the feature itself. The trade-off is to build a verification ladder that starts with the most direct evidence and expands according to blast radius.
That means some changes take longer after the code exists. I consider that a feature of the system. A fast implementation followed by a hidden regression is not faster delivery; it is deferred work with worse timing.
My definition of done is therefore simple: the requirement is named, the implementation exists, the relevant claims have evidence, and the remaining limitations are explicit. If the agent cannot help produce that proof, the work is still a proposal.