The Road to Next — your interactive course for Next.js with React

Testing AI-Generated Code: Lint the Pattern, Test the Decision

Robin Wieruch •

Coding agents love to write tests. Ask one for a new handler, a form or a combobox, and it comes back with a test file next to it, unasked. In a large codebase where agents write most of the code from documented patterns, that sounds like the testing problem solves itself.

It does not, because those tests are not streamlined. Every agent-written test brings its own mock setup, copied from whatever test file it saw last and changed a little. Many of them assert things a linter should catch: that the session was checked, that the error was mapped, that a translation key is the expected string. And most of them test files that do nothing but follow the pattern, so they restate the implementation and never find a bug.

As a freelance team lead software engineer, I helped my team streamline this. The goal was never a fully tested codebase. The goal was coverage that follows the structure of the code: linters hold every file to its pattern, one shared harness per kind of file makes tests cheap, and a test sits on top only where a file goes beyond its pattern. This post is about unit and integration tests. End-to-end tests are a topic of their own and not covered here.

Here is what that changes for a single test file:

What one test file spends its lines on
Illustrative. The same handler, tested before and after streamlining.
Agent-written, unstreamlined70 lines
One test per file, every file fakes the world on its own.
Streamlined16 lines
The lint holds the pattern, the harness holds the setup.
setup moved into the shared harnesspattern checks moved into the lint
Own mock setup, copied from the last test file
Asserts the pattern, session checked, error mapped
Restates the code, markup, translation keys
Asserts a decision, what the file decides

The streamlined test is a fraction of the size, and every line it has left asserts a decision the file makes. The rest did not disappear. It moved to places where it covers every file instead of one.

Most Files Follow a Pattern

An agentic codebase runs on patterns. Every kind of file has a pattern document that says what it looks like, and both humans and agents follow it. A write handler, for example, has the same shape in every file:

text
├─ await requireSession()
│
└─ try {
     ├─ const client = createServiceClient(Projects)
     └─ await client.archiveProject({ id })
   } catch (error) {
     return toFailure(error)
   }
   return toSuccess("project.archived")

I call that shape the skeleton. Most handlers are nothing but the skeleton: check the session, call one backend method, map the error, return a result. The same is true for forms, comboboxes, table columns and endpoints: each kind has its skeleton, and most files of that kind are nothing more.

A test for a pure skeleton file restates the implementation line by line. It breaks when the code changes, and it never catches a bug, because there is no decision in the file that could be wrong. But the skeleton itself can break, and with agents it breaks in a particular way. An agent rarely forgets the whole pattern. It drops one line of it: the session check in a handler that “is only called from a signed-in page”, or a backend call that slips out of the try during a refactor. A test only notices that in the one file it tests.

So instead of one test per file, every kind of file went through the same three steps:

Three steps per kind of file, in this order
Each step narrows what is left for the next one. Pure skeleton files never get a test of their own.
  1. Lint the skeleton
    A script enforces the invariants the pattern document states.
    Covers: every file of the kind, including the ones agents have not written yet
  2. Build one shared harness
    Fake what the kind of file reaches outside its own code, once.
    Covers: the setup every test would otherwise copy, so a test file holds only its scenario
  3. Write the rule, then test the riskiest
    The pattern document says which files earn a test and what it asserts.
    Covers: the files that decide something the skeleton does not
every file, held to the skeleton by the lintthe few that decide something, with a behavior test

Write handlers went first, and they are the running example. Forms, comboboxes and the other kinds followed the same way, and they come at the end.

Step 1: Lint the Skeleton

The first step covers every file of a kind, including the ones no agent has written yet, without a single test. A small script parses every exported handler with the TypeScript compiler API and checks the invariants the pattern document states:

What the lint checks on every write handler
Four sentences of the pattern document, checked on the body of every exported handler.
export const archiveProject = async (id) => {
await requireSession();
try {
const client = createServiceClient(Projects);
await client.archiveProject({ id });
} catch (error) {
return toFailure(error);
}
return toSuccess("project.archived", { input });
};
  1. Session first
    The first statement is await requireSession(). An exported handler is a public endpoint, whoever calls it.
  2. Service call in a try
    Every createServiceClient() sits inside a try that has a catch, so a backend failure reaches it.
  3. The catch maps the error
    Every catch returns the result of toFailure(), or rethrows into an outer try.
  4. Input through the helper
    The submitted input only travels back through toSuccess() or toFailure(), which strip secret fields.

None of these rules is clever. Each one is a sentence from the pattern document turned into a check. The pattern document is what the agents already follow when they write a file, and what our review agents already match a change against. The lint makes the same patterns hold for every file, instead of hoping that every agent read them and every reviewer remembers them.

Read More
Agentic Code Review: Pattern Matching for AI
text
// Command Line
npm run lint:handlers

  projects/archive-project.ts:7
    `archiveProject` [session first] does not start with `await requireSession()`
  invoices/list-invoices.ts:12
    `listInvoices` [call in try] calls `createServiceClient(...)` outside a `try` block

For an agent, a lint like this is the best kind of feedback there is. The message names the file, the line and the rule, so the agent fixes it in the same session without a human pointing at it. A comment in a code review arrives a day later. A lint error arrives before the agent says it is done.

Getting a lint like this into a codebase that agents have been writing for months took four steps, and the same four repeated for every kind of file:

Rolling out a pattern lint
The same four steps for every kind of file. Rule names and hit counts are illustrative.
  1. Survey the candidate rules
    An agent runs each rule over every file and reports the hits. The hits decide which rules make it.
    keepSession first, 3 hits
    keepService call in a try, 1 hit
    keepNo stand-in data next to an error, 1 hit
    dropNever wrap a mapped error again, 0 hits, the type checker already enforces it
  2. Fix every violation first
    In their own commit, before the lint lands, so the lint starts green.
    archive-project.tslist-invoices.tsget-owner.ts
  3. Land the lint, with almost no exemptions
    A real violation gets fixed, never exempted. The single exemption carries its reason next to it.
    exempt: 1, with reason
  4. Delete the tests the lint made redundant
    Assertions the lint now covers for every file go, and so do test files with nothing else to say.
    expect(requireSession).toHaveBeenCalled()expect(result.status).toBe("ERROR")

The survey is the step I would not skip. An agent ran every candidate rule over every file before a line of the lint existed, and the hits decided which rules made it: one rule was dropped because the type checker already made its violation impossible, and the others found real bugs, like a read handler that returned an empty list next to an error, so the UI could not tell “failed” from “empty”. The last step is where the streamlining starts: every test assertion the lint now covers for all files goes away.

Step 2: Build One Shared Harness

The second step makes the remaining tests cheap and uniform. A handler reaches outside its own code all the time: the backend client, the session, translations, redirects, cache invalidation, cookies. If every test file fakes those on its own, every test file starts with forty lines of mocks, and every one of them fakes the backend slightly differently. Agents are particularly good at copying forty lines of mocks, which makes the problem worse with every file they write.

So there is one harness per kind of file. The test runner picks it by file name: every *-handler.test.ts gets a setup file that fakes everything a handler touches, and the test file holds only its scenario.

The interesting part is the backend. The easy fake is a mock function per client method that returns whatever the test wants. Our harness is stricter: it keeps the real generated client and replaces only the network with an in-memory transport. Every request is serialized and deserialized again, exactly like on its way to the real backend.

Where the fake backend sits
Only the network is replaced. Every request still goes through the same serialization as in production.
The shared harness
Test file
serves methods, runs the handler
Handler
unchanged production code
Service client
the real generated client
In-memory transport
serializes every request like the network would
Fake methods
fake.serve(Service, { ... })
unserved method answers unimplementeda crashing fake fails the testsame limits as the production server
Every request is recorded after serialization, so fake.calls() shows exactly what would have reached the backend. Redirects, invalidated caches and cookies land in effects.
A mock per test file
Test file
stubs every method
Handler
unchanged production code
Mocked client
returns what the test says, nothing is serialized
Nothing is serialized, so a request the backend would reject still passes, and every test file fakes the backend a little differently.

The fake backend has three rules, and each one exists because a looser fake would let a bug through:

  • A method the test did not serve answers “unimplemented”. The handler under test sees a real backend error, not undefined.
  • A fake handler that crashes fails the test. Otherwise a bug in the fake would pass for a backend failure, and the test would assert the wrong thing with a straight face.
  • Requests are decoded with the same limits as the production server. Not with the client library’s defaults.

That last one was not on anyone’s list. When we moved the existing tests onto the real transport, a deeply nested request that production accepts every day was rejected by the fake. The client library and the server disagreed about how deep a message may nest. The old mocks had never serialized anything, so they had hidden that mismatch for as long as the tests existed.

Step 3: Write the Rule, Then Test the Riskiest

Only then came the tests themselves. Before writing any, the pattern document got a section that says which files of this kind earn a test and what that test asserts. This section is what keeps agents from going back to one test per file. Without it, “add tests for the handlers” produces a test for every skeleton handler, all of them green, none of them useful.

Skeleton
No test of its own. The lint and the helpers' tests cover it.
  • checks the session, calls one method, returns its result
  • create vs. update in one handler
  • redirects after success
  • a read that only guards the write
Earns a test
It decides something the skeleton does not.
  • a branch over a kind, a scope or a state
  • a write built from a read, and a failed read that must stop it
  • a partial update: which fields reach the write
  • more than one write: their order, a partial failure
  • a security decision, like deciding a value on the server
  • an error it remaps or swallows, a payload it reshapes
What the test asserts: what leaves the handler
  • the requests sent, and none after a failure
  • the returned result
  • side effects: redirects, invalidated caches, cookies
Not asserted, checked elsewhere:
  • that the session was checked
  • a translation key for its own sake

A handler earns a test when it decides something the skeleton does not: a branch over a kind or a state, a write built from a read, more than one write, a security decision, or an error it remaps. Everything else is skeleton, and the lint already covers it.

The rule also says what a test asserts: what leaves the handler. That is the requests it sent (and the ones it must not send after a failure), the result it returns, and its side effects. It does not assert that the session was checked, because the lint does that. Here is a test for a handler that reads a project and writes its members back:

projects/add-members-handler.test.ts
beforeEach(() => {
  fake.serve(Projects, {
    getProject: () => ({ id: "p1", members: ["ada"] }),
    updateProject: () => ({}),
  });
});

it("adds the new members in front of the existing ones", async () => {
  const result = await addMembers("p1", ["grace", "linus"]);

  expect(fake.calls(Projects, "updateProject")).toMatchObject([
    { project: { id: "p1", members: ["grace", "linus", "ada"] } },
  ]);
  expect(result).toMatchObject({ status: "SUCCESS" });
});

it("writes nothing when the project cannot be read", async () => {
  fake.serve(Projects, {
    getProject: () => {
      throw new ServiceError("unavailable");
    },
  });

  const result = await addMembers("p1", ["grace"]);

  expect(result).toMatchObject({ status: "ERROR" });
  expect(fake.calls(Projects, "updateProject")).toEqual([]);
  expect(effects.invalidated).toEqual([]);
});

The beforeEach serves the happy path for every method the handler calls. Each test overrides only the one method it is about, runs the handler, and checks what went out. There is no mock setup in the file at all, because the harness owns it. That is also why agents write these tests well: the file is all scenario, and the rule tells them which scenarios matter.

These tests paid for themselves right away. Two of them turned up permission bugs. One handler sent a grant with an empty scope when the role did not allow the requested one. Another took a value from the form that the server should have decided on its own. Neither the lint nor the type checker could have seen either of them, because both were wrong decisions, not wrong shapes.

The Same Steps for Forms and Comboboxes

Handlers are the obvious candidates, because they talk to a backend. But the method works for every kind of file with a pattern, and in a frontend codebase most of them are UI.

A form has a skeleton as strict as a handler’s: it is built on the shared form component, it validates with a schema, it submits through a handler, and every field has a label and a slot for its error. The lint checks exactly that. Its harness renders the form with a fake handler and the real schema. A form earns a test only when it decides something: a field that appears depending on another one, or input that is reshaped before it is submitted.

A combobox that loads its options is the same story. It is built on the shared async combobox and gets its options from a read handler, never by fetching on its own. Its harness renders it on the same fake backend the handlers use, so a combobox test serves the read handler’s backend method like any handler test would:

projects/team-member-combobox.test.tsx
it("only offers the members of the selected team", async () => {
  fake.serve(Members, {
    listMembers: ({ teamId }) => ({
      members: teamId === "t1" ? [{ id: "ada", name: "Ada" }] : [],
    }),
  });

  const view = renderWithHarness(<TeamMemberCombobox teamId="t1" />);
  await view.open();

  expect(view.options()).toEqual(["Ada"]);
  expect(fake.calls(Members, "listMembers")).toMatchObject([
    { teamId: "t1" },
  ]);
});

A combobox that only lists what a read handler returns gets no test at all. This one filters by another field, so it earns one. The same reasoning works for table columns, endpoints and every other kind with a pattern document:

The same three steps, per kind of file
Each kind has its own pattern document. The lint and the harness follow from it, and so does the rule for when a file earns a test.
Write handler
Lintsession first, service call in a try, the catch maps the error
Harnessfake backend, session, redirects, cache invalidation
Earns a test whenit branches, writes from a read, or writes more than once
Read handler
Lintsession first, failure returns an error and no stand-in data
Harnessthe same fake backend
Earns a test whenit joins several reads, and one failure must not break the list
Form
Lintbuilt on the shared form, submits through a handler, every field has a label and an error slot
Harnessrender harness with a fake handler and the real schema
Earns a test whenfields depend on each other, or the input is reshaped before submit
Combobox
Lintbuilt on the shared async combobox, options come from a read handler, no fetching of its own
Harnessrender harness on the fake backend
Earns a test whenit filters by another field, groups options, or maps a selection
Table columns
Lintevery sortable column has a matching sort key in its query
Harnessnone needed
Earns a test whena cell decides what to show, beyond formatting a value
Endpoint
Lintevery endpoint is a one-line call into a shared implementation
Harnessrequest builder for the shared implementations
Earns a test whenonly the shared implementations: refused without auth, refused on a bad body

Not every kind needs all three steps. Table columns need no harness, because their lint already checks the one thing that tends to break. Endpoints are one-line calls into a few shared implementations, so the shared implementations got the tests and the thin files did not. And plain utility functions have no skeleton at all: they are tested directly, as they always were.

Streamlined, Not Fully Tested

After the first round per kind, tests stopped being a campaign. Whoever touches a file that matches its kind’s rule adds its test, and since the rule lives in the pattern document, agents pick it up the same way they pick up the skeleton. That turns the three steps into a loop the agents run on their own:

The loop an agent runs for every file it touches
Steps 3 and 4 send the agent back to step 2 until both pass.
  1. The pattern documentinstruction
    The skeleton, its variants, and which files earn a test.
  2. The agent writes or changes a file
    It follows the skeleton, and adds a test when the file decides something.
  3. The lint runsfeedback
    File, line and rule in the message, so the agent fixes it in the same session.
  4. The tests run on the shared harnessfeedback
    A wrong decision fails a behavior test before anyone reviews the change.
  5. A human reviews last
    Is this the right change, was it asked for? The pattern matching is already done.

The pattern document is the agent’s instruction. The lint and the tests are the agent’s feedback. A human reviews last and asks the questions an agent answers confidently and badly: is this the right change, and was it asked for? The pattern matching is already done.


Agents will write tests whether you ask them to or not. The question is whether those tests line up with how the codebase is built. A lint covers every file of a kind in one go, a shared harness turns each test into a few lines of scenario, and a written rule puts a test on top only where a file leaves its pattern. The result is not a fully tested codebase, and it does not need to be. It is a codebase where every file is held to its pattern, and every test that exists has a reason to.

Never Miss an Article

Join 50,000+ developers getting weekly insights on full-stack engineering and AI.

AI Agentic UI Architecture React Next.js TypeScript Node.js Full-Stack Monorepos Product Engineering
Subscribe on Substack

High signal, low noise. Unsubscribe at any time.