Why AI-Generated Code Is Easy but Engineering Trust Is Hard

Later, your team discovers the agent quietly misinterpreted an ambiguous requirement near the beginning of the implementation. Every decision that followed was internally consistent. The tests validated the implementation. The reviewer approved the diff. The code did exactly what the agent believed the requirement asked for.

The AI made a mistake, but the truly frightening part is that nothing in the engineering process ever proved the agent’s interpretation was correct before the code reached production. If passing tests, code review, and a successful build still cannot prove that AI-generated code is trustworthy, what can?

At Salesforce, we encountered this challenge while using AI coding agents to develop an AI-powered mobile application designer. The agents could generate code remarkably well. Our challenge was determining why we should trust that AI-generated code.

Where Was the Mistake Supposed to Be Caught?

A coding agent can often find a path from a requirement to a functioning implementation. The problem is that many paths may produce the same visible result. An agent might introduce a utility even though one already exists, follow a framework convention while violating a repository-specific convention, or interpret an ambiguous requirement without revealing that it made a decision. The code can compile and still increase technical debt.

A final reviewer must then reconstruct everything that happened before the diff appeared. Which assumptions did the agent make? Did it search for existing components? Did the tests validate the intended behavior or merely the behavior the agent implemented? As agents produce larger changes, code generation scales faster than human review capacity. If trust cannot reliably be reconstructed at the end, it has to begin earlier.

Make Uncertainty Visible Before Writing Code

Instead of asking an agent to immediately implement a request, the team first required a specification that captured the required behavior, measures of success, assumptions, unresolved questions, and failure conditions. This changed the role of the specification. It was no longer background documentation, rather, it became the contract for everything that followed. The plan had to show how the contract fit the repository. Tests had to encode its success criteria. Reviewers could evaluate the code against the same definition of success.

This approach became an agentic Spec-Driven Development, or SDD, workflow. The name mattered less than the shift in thinking: success was now defined by the specification instead of the implementation.

Four phases, each with a gate that must pass.

Making the contract explicit also exposed uncertainty, which raised another question: should every unknown stop the workflow and interrupt an engineer?

Lookup Is Not Judgment

These were legitimate questions, but most were not decisions. Their answers already existed in the codebase, documentation, tests, or version history. When every gate sent those questions to an engineer, the human became a search service for the agents.

Agents should resolve lookups with evidence; humans should make decisions that require judgment. As AI coding agents take on larger software engineering tasks, this distinction becomes increasingly important.

Agents resolve lookups; humans decide what needs judgment.

That reduced interruptions, but created the next question. If an agent answers its own lookup questions, how do you know it found something real instead of producing a plausible explanation?

Ground Every Decision in Repository Reality

Language models can produce polished plans that reference imagined files, nonexistent APIs, or architectural patterns the repository does not use. Plans were therefore required to prove they had examined the development context. A plan could not simply say “Add a utility to identify a data-bound value.” It first had to determine whether that capability already existed and show how the repository used it.

The plan identified which components would be extended, which utilities would be reused, which state-management path owned the change, which tests would be updated, and what evidence supported each step. The governing principle was reuse before create. An independent Skeptic Agent then reviewed the plan for imagined dependencies, hand-waved integration steps, unnecessary abstractions, and missing test coverage. Only a plan grounded in repository evidence could proceed. Even then, a grounded plan established only that the proposed route was real; it did not prove the implementation would remain faithful to the contract.

Did Every Requirement Become Executable Proof?

Once the specification and plan were accepted, the success criteria became tests. The workflow confirmed that those tests failed for the expected reason, wrote the minimum implementation needed to satisfy them, and then ran the tests, regression suite, and production build. This created a direct relationship: the specification defined success, the tests encoded it, and the code earned it. For AI-generated code, passing tests alone is no longer sufficient evidence of trust.

A green test suite could still leave a dangerous gap, confirming only that the written tests passed rather than that every requirement had become a test. A Compliance Matrix was therefore used to map each success criterion to supporting implementation and test evidence, asking whether every requirement could be traced to code and executable proof rather than simply whether the tests passed.

Even complete contract compliance left one final uncertainty: what if the same development process had failed to notice a weakness?

Challenge the Evidence

The implementation then entered a multi-agent review process called Conclave.

Independent critics find flaws; the judge only tightens.

Reviewers first worked independently to avoid anchoring, each applying a different lens including security, correctness, user experience, accessibility, and repository history. A designated Advocate presented the strongest evidence supporting the implementation. The reviewers then examined one another’s findings, and each claim had to be supported against the code and classified as standing, revised, or conceded. An independent Judge Agent evaluated the remaining issues.

One asymmetric rule protected the process: the Judge could downgrade a passing result, but it could not upgrade a failed gate. Persuasive language could not compensate for missing evidence. Trust could become stricter as the change advanced, but it could not become weaker.

Would that process hold up on a real feature?

Every gate tightens trust—spec to trusted change.

What Happened in Production?

The workflow was tested on a feature in the AI-powered mobile application designer. Users needed a control that appeared when a property was data-bound so they could remove the binding or connect it to a different data field. The visible feature was small, but implementing it required repository-specific knowledge. The workflow needed to determine how binding expressions were detected and serialized, which existing picker handled data selection, how it should be initialized, how property changes integrated with undo and redo, and which accessibility conventions applied.

Agents discovered those answers through the repository and supplied evidence for each one. Only one unresolved question reached a person: when a user removed the binding, should the system clear the entire value or preserve any static portion? No repository search could determine the desired product behavior. The team chose to clear the entire value for a predictable experience. That decision contradicted an assumption elsewhere in the draft specification, so the workflow updated the success criteria, tests, and failure behavior before implementation continued.

The run produced six focused commits, more than 4,500 passing tests, a compiling production build, compliance evidence covering every success criterion, an adversarial review with no required actions, and one human interruption for one product decision. A second work item required no human intervention because the repository contained enough evidence to resolve every question. The objective was to ensure human involvement was determined by the nature of the uncertainty, not to remove humans from the process.

What Should You Change Tomorrow?

The next time an AI coding agent opens a large pull request, do not begin by asking whether the code looks correct. AI generates code. Engineering generates trust.

The post Why AI-Generated Code Is Easy but Engineering Trust Is Hard appeared first on Salesforce Engineering Blog.