Why prose API docs break AI agents

A few weeks ago I was building an integration on top of an email API for axrank.ai, and I did the lazy thing. Instead of reading the reference myself, I pasted the docs page into an agent and told it to write the request. The docs looked great. Clean typography, friendly headings, a little "Add your first subscriber" walkthrough. The agent read it, wrote confident code, and the API threw a 422 back in my face.

I've now watched this happen enough times to stop blaming the model. The docs page was written for a human who skims. The agent wasn't skimming. It was parsing sentences and guessing at structure, and guessing is exactly where it goes wrong.

The setup: one API, two ways to describe it

Here's a tiny endpoint. It adds a subscriber to a mailing list. Three fields matter: email and list_id are required, first_name is optional. That's the whole thing.

The human docs page describes it like this:

To add a new subscriber, send a POST request to /v1/subscribers. Include the subscriber's email address and, optionally, their name. You'll also want to specify which list they should be added to.

Read that as a person and it's fine. You'd figure it out. Read it as an agent that has to produce an exact JSON body, and every noun in that sentence is a landmine.

The agent reads the prose page and gets it wrong

I gave an agent that paragraph and asked for the request body. Here's what it wrote:

POST /v1/subscribers
{
  "user_email": "sam@example.com",
  "name": "Sam Rivera",
  "plan": "pro"
}

Every line is a reasonable guess and every line is wrong.

The prose said "email address," so the agent named the field user_email. Plausible. Also not the field. The real key is email. It said "their name," so the agent used name. The API wants first_name. It saw "specify which list," decided that sounded like configuration, and invented a plan field to go with it because subscriber APIs usually have plans, right? That field doesn't exist. And the one field that's actually required, list_id, never made it into the body at all, because the sentence buried it in "you'll also want to," which reads as optional.

Wrong param, hallucinated field, missing required value. One paragraph, three different failure modes. The API returns:

{
  "error": "unprocessable_entity",
  "message": "Unknown field: plan. Missing required field: list_id."
}

Why prose breaks agents, and it isn't the model's fault

The docs page never actually stated the structure. It encoded the structure inside English and trusted a human to decode it. "Their name" carries no field name. "Optionally" carries no required: false flag. "Email address" doesn't tell you whether the key is email, user_email, emailAddress, or contact_email.

So the agent does the only thing it can. It fills the gaps with the most statistically likely shape of an API it has seen a million times. That's the hallucinated plan field. It's not the model going off the rails. It's the model completing a pattern because you handed it a pattern instead of a contract.

This is why I get twitchy when people say the fix for agent-unfriendly docs is "write clearer prose." Clearer prose helps the human. The agent still has to reconstruct a typed data structure from paragraphs, and reconstruction is guessing with extra steps.

We spent fifteen years obsessing over developer experience, the DX of getting a human to a working integration fast. Agent experience is the same idea for the reader that's now doing most of the integrating. AX isn't about being polite to robots. It's whether the surface you ship, docs, schema, errors, gives an agent an exact contract or forces it to guess. The prose page above has terrible AX for one boring reason: it never states the structure.

Same API, typed schema, first-try success

Now I gave a fresh agent the exact same endpoint, but described as an OpenAPI schema instead of a paragraph:

paths:
  /v1/subscribers:
    post:
      requestBody:
        content:
          application/json:
            schema:
              type: object
              required: [email, list_id]
              properties:
                email:
                  type: string
                  format: email
                first_name:
                  type: string
                list_id:
                  type: string
                  description: ID of the list to subscribe the contact to

No new words of explanation. Just structure. Here's what the agent produced:

POST /v1/subscribers
{
  "email": "sam@example.com",
  "first_name": "Sam",
  "list_id": "list_28xk"
}

200 OK, first try. The field names are exact because they're literally in the schema. Nothing got hallucinated because a closed property list means there's nowhere to invent a plan. And list_id shows up because required: [email, list_id] is a fact, not a hint you have to infer from the phrase "you'll also want to."

The model didn't get smarter between those two runs. It got a contract instead of a story.

This is a tooling problem, so treat it like one

You already have the source of truth. It's your OpenAPI spec, your GraphQL schema, your type definitions. The failure is that the thing you point agents at, the pretty docs page, threw all of that away and kept only the prose.

Tools like Fern exist to stop that from happening. You give it one typed spec and it generates the docs site, the SDKs, and an MCP server from the same definition, so an agent can pull the schema and the exact operations directly instead of scraping sentences off a marketing page. The human still gets a nice page. The agent gets the contract. Nobody's reconstructing types from adjectives.

Agent experience is the whole reason I built axrank.ai. It grades exactly this: it points real frontier agents at a company's developer surface and scores how well they can actually use it, then sorts the failures into a taxonomy. The single most common one is the endpoint above. Good-looking docs, no machine-readable structure, agents guessing field names and hallucinating the rest. You can have a 9 out of 10 DX and a 3 out of 10 AX at the same time, and until an agent is the one reading your docs, you'd never notice.

What I'd do about it on Monday

Ship an OpenAPI spec if you don't have one, and make it the source your docs generate from, not a file that rots in a repo while the docs drift. Expose something an agent can read without scraping: an llms.txt, an MCP endpoint, a raw schema URL. Keep the prose. Humans still need the walkthrough and the "why." Just stop making the paragraph the only description of your API, because the agent can't read between the lines. It doesn't have any lines. It has a schema, or it has a guess.

If this is the kind of thing you think about, subscribe at quintonwall.com and come find me on YouTube at @seeqcode. I build a lot of this in public.