Part 4 of 4 · Simplicity Is Security

Simplicity Is Security: Agentic Coding

Keeping agent code simple and secure.

By M. Robert Martin·17 min read

Imagine we ask a coding agent to implement a TLS client. It writes the code, builds it, runs tests, fixes failures, and reports that the client connects successfully. The change is almost definitely large enough that reading every line would take more time than generating the code did.

The implementation contains a configuration option that sets Go’s tls.Config.InsecureSkipVerify without requiring custom verification. The client works. It connects to servers and exchanges encrypted data. But the configuration option doesn’t establish that the server is the one we intended to reach.¹

We saw this problem in Part II. The difference now is that in Part II, it was a human engineer who made the decision to offer the bypass, and now it was an agent who made that decision. An engineer may introduce it deliberately, carelessly, or by mistake. The agent makes the decision as one detail among thousands that we didn’t individually choose.

This is the fourth and final essay in Simplicity Is Security. The first three examined unintended behavior, unnecessary capabilities, and the limits of understanding. When agents write the code, all three problems remain. And we have inserted a new boundary between ourselves and the program.

A new boundary

A programmer’s instructions are not the same as a program’s instructions.

The agent generates code through a probabilistic process. The resulting program is a concrete implementation of what we told the agent we wanted, that will run according to the language and runtime’s rules, with whatever inputs, scheduling, and environment it encounters. It’s honestly amazing to watch.

We supplied the requirements, but the agent likely chose most of the implementation details: the dependencies, abstractions, error handling, defaults, and even the permissions requested by its deployment configuration. Its immediate feedback may be that the code builds and the tests pass. Neither simplicity nor security are built in.

We added a second boundary between us and the finished code.

The original is what the program can do when deployed. It may read credentials, modify accounts, execute subprocesses, change configuration, or communicate over the network. When we wrote the code, we made these decisions as part of the implementation.

The new boundary is what choices the agent makes while writing the code. The agent is non-deterministic. We can give it instruction, and assume they were followed. We might discover that’s not entirely true. The agent is probabalistic. It will write code that is informed by all of the code it was trained on. The result isn’t necessarily simple and it isn’t necessarily secure, no matter how we write our prompts.

This new boundary is easy to miss, especially when the agent is giving us the program results we expected.

Working code is the starting point

In Part I, we distinguished software doing what it is specified to do from software doing nothing it is not specified to do. The first is the easier part to demonstrate. We can write functional tests, exercise success and failure paths, and check that the implementation produces the required results.

Even then, passing tests is evidence about the cases exercised, not a general proof of correctness. And it says still less about the enormous number of things we never asked the program to do.

Saltzer and Schroeder described this negative quality of security in The Protection of Information in Computer Systems. Preventing all unauthorized use is difficult to establish because we would have to anticipate every possible threat. They also noted that unwanted access paths may go unnoticed in normal use—precisely because normal use doesn’t attempt them.²

A TLS client that successfully retrieves a page hasn’t demonstrated that it rejects an impostor server. A passing test that checks a transfer completed hasn’t demonstrated that an unrelated caller cannot alter another account’s balance. The behavior we specified occupies a small part of what the program might be able to do.

Also in Part I, I recounted how OpenBSD’s Theo de Raadt told me that secure programming requires that the program does what it’s specified to do, and does not do what it is not specified to do. How do we know it doesn’t? At that time it was by auditing the code. Since then, their engineers still audit code, searching not only for known vulnerabilities but for ordinary defects that might later prove exploitable. But they also introduced privilege separation and restrictions such as pledge and unveil, which limit what a process can do even when mistakes escape inspection.³⁴

The lesson isn’t that auditing has become useless. It’s that inspection and containment answer different questions. An audit asks what defects we can find. An enforced boundary limits what an undiscovered defect can accomplish. Most of us aren’t programming for OpenBSD, but we still have the same concerns when we need secure code. We audit code and we restrict what it’s capable of doing, even when bugs get through.

With agent-generated code, we need to carry these practices forward. First make the implementation manageable. Then investigate its actual behavior and authority. Finally, enforce the limits the problem requires.

Simplicity at agent scale

I see a pull request with tens of thousands of lines of human-written code maybe twice a year. An agent can produce changes of that scale on almost any feature ticket. The cost of generating code has fallen dramatically. The effort required to understand that code has not.

Simplicity matters even more when agents write software. An agent will produce a working implementation based on the material it trained on, which does not necessarily emphasize simplicity. And we can’t just tell it to “make the code simpler”.

Why not? Because what the agent thinks is simpler and what we think is simpler are not the same thing. The agent wants code that looks simpler - shorter functions, reduced duplication, fewer abstractions, more helper functions. We want code that is easier to reason about. We want to reduce the obligations placed on the next person who must understand, change, test, or secure the program. We want to use fewer brain cells understanding more of the system to solve what should be a local issue.

Understand how the agent understands the task

Before asking the agent to simplify its implementation, we should establish what problem it believes it solved. We supplied the instructions, but that doesn’t establish that the agent understood our requirements — or that our requirements were complete or correct.

1Examine the feature you implemented and describe its required behavior, using the original specification, existing contracts, and relevant documentation. Distinguish explicit requirements from assumptions and implementation choices. Identify the tests or other evidence supporting each requirement, including failure cases, boundary conditions, and invariants. Report tests that passed, failed, were skipped or disabled, or haven't been run. Identify requirements with insufficient verification, assumptions needing confirmation, and behavior not supported by the specification. Do not modify the code.

This isn’t merely checking whether the agent completed its assignment. It can reveal functionality we didn’t request, assumptions the agent made to fill gaps, and tests that may have been ignored because they failed.

Our prompts are imperfect. When we under-specify, the agent must make decisions we didn’t explicitly authorize. When we over-specify, we may impose assumptions or constraints that limit the agent too much. Either can introduce complexity that the implementation doesn’t need.

Before simplifying the code, we need to surface these decisions.

Protect the behavior before changing the code

For the changes we intend to make, we need tests that protect contracts, not internal structure.

Start at the boundaries we must preserve. Where does the software receive inputs and produce results? What public APIs, protocols, file formats, and database schemas must remain compatible? What state invariants must always hold? These are the contracts our tests need to enforce.

Test those contracts at boundaries that will survive large scale refactoring:

• Interface tests verify inputs, outputs, errors, and compatibility through public APIs.
• Integration tests verify interactions with databases, filesystems, external services, and other components.
• Behavioral tests verify complete operations, including failure paths, rejected inputs, and expected side effects.
• Invariant tests verify properties that must remain true across operations, including concurrency, retries, and partial failures where relevant.

Private functions are likely to disappear. Required behaviors are not. We need to understand what requirements are stable and test those. We want the ability to change anything behind those requirements.

Before refactoring, ask the agent:

1Identify the boundaries and contracts that must survive a substantial reorganization of this code. For each contract, describe the required behavior, including inputs, outputs, failures, invariants, and external effects. Map these requirements to existing tests, noting which have passed, failed, been skipped, or not been run. Identify missing tests and propose specific test cases with expected results. Favor tests at stable boundaries that will remain valid even if the implementation is replaced. Distinguish specified requirements from incidental behavior and assumptions. Present a test plan that makes it easy for us to agree on what must remain unchanged, what may change, and how we will verify correctness. Flag any ambiguities or disagreements that require my decision. Do not modify code or tests until we've agreed on the contracts and test plan.

This prompt requires an explicit test plan with agreement between the developer and the agent. We’re not asking the agent to whether testing is adequate. We’re requiring agreement on what constitutes correct behavior before authorizing changes that may replace much of the implementation.

Characterization tests may help establish what existing code does, but we must not mistake accidental behavior for a requirement. The agent may not know which behaviors are intentional and which are accidental, and may have even preserved or created the accidental behaviors.

Run the tests and establish a passing baseline before changing the implementation. Then run them after each meaningful stage of the refactoring.

The better our tests protect the contracts, the more freedom we have to simplify the code behind them.

Find the complexity

An agent asked to simplify code may focus on what looks simpler: shorter functions, fewer lines, less duplication, or familiar abstractions. These changes can help, but they can also move complexity around without reducing it.

Complicated control flow, shared mutable state, unnecessary indirection, concurrency, and side effects create different kinds of complexity. Each presents different challenges.

What we want is code that’s easy to reason about and minimizes the cognitive load on future developers. They should have fewer things to understand, fewer assumptions to remember, and fewer interactions to trace before they can establish correctness. Before changing anything, the agent should identify where that cognitive load comes from and how to reduce it.

1Examine this change and adjacent code for complexity that makes its correctness difficult to establish. Consider control flow, mutable state and ownership, dependencies between modules, unnecessary abstractions, concurrency, and side effects. Identify the three most consequential problems. For each, show the relevant code, explain what a developer must understand to establish correctness, and suggest a bounded improvement. Do not modify the code.

Notice that we’re not asking for a complexity score or a list of every function that could be shorter. We want evidence of specific reasoning problems.

The result should help us choose where to begin, and that doesn’t mean everywhere. We choose the problem that offers the greatest improvement, make one bounded change, verify that we’ve preserved the required behavior, and repeat where further improvement is justified.

Make changes that matter

The complexity analysis gives us a set of problems worth considering, not a list of changes we must make. We should select the problem that offers the greatest opportunity for improvement and ask the agent to develop a proposal.

1Take this finding from the complexity analysis and propose the change that would do the most to reduce complexity and improve our ability to reason about the code. Consider changes to the architecture, responsibilities, and dependencies where appropriate. Explain what would become easier to understand, what behavior and invariants must be preserved, and how we can verify correctness. Identify risks, alternatives, and unresolved assumptions. Do not modify the code yet.

The smallest change is often the right choice when fixing a defect or adding a feature. But our objective here is different. What matters is how much complexity the change removes, not how much code it changes. What change reduces the cognitive load for future developers?

Once we’ve agreed on a proposal, we can authorize the work. A substantial change may be implemented in stages, with tests and other verification after each stage. We must preserve the required behavior and establish that the resulting design actually reduces complexity.

Then we can return to the analysis and consider the next opportunity. We don’t need to address every finding. We stop when further changes no longer meaningfully improve our ability to reason about the software.

We’ve made the implementation easier to understand. Now we can use that understanding to investigate what the software can do, even if it’s not intended to.

Ask what the code can do, not whether it’s secure

We’ve established what the software is supposed to do, agreed on tests that protect that behavior, and simplified the implementation. But knowing the software does what it’s supposed to do does not establish that it does nothing else.

We can ask the agent to audit its code for security vulnerabilities. That will produce useful findings, but it asks for a broad security judgment about an enormous space of possible failures that the agent was instrumental in creating. We already know the limits of that approach.

As we discussed earlier in this series, the behavior we don’t want occupies a huge negative space. We can make that problem more manageable by finding where side effects and security decisions are scattered throughout the implementation, and looking for ways to consolidate them.

1Examine this program for side effects, including shared-state modifications, filesystem access, network communication, database operations, environment changes, process execution, and privileged API calls. Trace direct and indirect operations, including the authorization checks and trust decisions governing them. Show source locations, call paths, and affected resources where identifiable. Group related operations by responsibility, resource, and security boundary. Identify where related side effects or security decisions are distributed across independent paths. Rank the most consequential opportunities for consolidation by how much they could reduce the number of places developers must understand, verify, and control. Present the ranked opportunities first, with the detailed inventory as supporting evidence. Report unresolved paths, assumptions, and analysis limitations. Do not assume that unreported operations are impossible. Do not propose implementation changes or modify the code.

We want the agent to find patterns, not hand us thousands of individual operations to inspect.

Suppose several modules modify the same account records, each with its own authorization checks. Rather than verifying every path independently, we might consolidate those modifications behind a common boundary where authorization is enforced consistently. That could substantially reduce the number of places where security depends on getting a decision right. But only if the old paths are eliminated or prevented from bypassing the new boundary.

Once we’ve selected an opportunity, we can ask the agent to design the improvement. As with our earlier simplification work, we shouldn’t limit the scope of the change simply to make the refactoring smaller. We want to eliminate the complexity, not move it around.

1For the consolidation opportunity we've selected, propose the changes that would most reduce complexity and improve our ability to reason about and control these side effects. Consider substantial architectural changes where appropriate. Explain which operations, responsibilities, authorization checks, and trust decisions would be brought together, which independent paths would be eliminated, and how the resulting boundaries would be enforced. Identify what a developer would no longer need to understand or verify independently, along with any new dependencies, privileges, or risks the proposed design would introduce. Identify which established contracts and tests protect the proposed changes, and any additional tests needed to verify failure behavior, external effects, or prohibited operations. Present the proposed design, its expected benefits, and its tradeoffs for our approval. Do not modify the code or tests yet.

When we’ve agreed on the design and confirmed that our tests protect the required behavior, the agent can implement it. We use those tests throughout the refactoring, then inspect the result to establish that the independent paths really have been eliminated and the new boundaries work as intended.

The objective isn’t fewer side effects. It’s fewer independent places where side effects and security decisions must be understood, verified, and controlled.

Still, consolidating side effects makes them easier to understand and control, but it doesn’t necessarily make them safer. A shared interface may still permit operations that none of its callers should be allowed to perform. So next we ask the agent to compare the capabilities of those interfaces with what the software actually needs.

1Examine the consolidated side-effect boundaries and compare what they permit with the established requirements and contracts. For each boundary, identify which callers can reach it, what operations they can request, what resources or state they can affect, and what authorization and trust checks are actually enforced. Identify capabilities that exceed the requirements, overly broad interfaces, missing restrictions, and paths that bypass the intended boundaries. Explain what unintended or unauthorized effects the current implementation would permit, with supporting code evidence. Propose changes that eliminate unnecessary capabilities, restrict permitted operations, and prevent callers from bypassing the intended rules. Identify tests that demonstrate required operations still succeed, prohibited operations are rejected, and failures preserve the required invariants. Distinguish verified findings from assumptions. Do not modify the code or tests yet.

With this information, we can direct the agent implement restrictions and verify them against our established contracts — and write tests demonstrating that these side effects cannot do what the program does not need to do.

We’ve accomplished more than making the code easier to inspect. We’ve reduced the number of operations that can be performed incorrectly, and the opportunities for unintended behavior to escape the boundaries we’ve established.

Less to trust

We began this series with two requirements for secure software: it must do what it’s specified to do, and it must not do what it isn’t specified to do. The first is relatively straightforward to test. The second is much harder to establish.

We’ve explored several ways to make that problem more manageable, and almost all of them can be summarized with this:

• Reduce complexity so we can reason about more of the program.
• Enforce boundaries that limit the consequences of defects.

Agentic coding doesn’t change those principles. It changes how we apply them.

An agent can produce an enormous amount of working code, faster than we can understand it. We need to also use the agent to help us establish the requirements, protect them with tests, replace complicated implementations, consolidate side effects, and design interfaces that reject operations the software shouldn’t perform.

We shouldn’t measure the value of an agent only by how much code it can produce. We should also consider how much complexity it can eliminate, how many independent opportunities for mistakes it can remove, and how much easier it can make the resulting software to understand and maintain.

This does not guarantee secure software. There will always be requirements we didn’t anticipate, defects we didn’t find, and behavior we don’t completely understand. But there can be fewer places where those mistakes matter, and fewer assumptions we must get right. And more importantly, there will be more code we can understand and reason about.

We are responsible for the code our agents produce. Working code is only step one. We should require the agent to help us make that code understandable, testable, and less capable of doing things we never intended.

We can use agents to write simple, secure, and demonstrably correct code.