# Measurements [Why this site exists](why-this-site-exists.md) makes a claim about cost. This page is the evidence, measured on live pages of both sites so anyone can repeat it and disagree with the result. ## What one page costs an agent A **token** is how a language model measures text — roughly three or four characters — and it is what the model pays for, in money and in the limited space it has to think in. A model with a 200,000-token working memory can hold **two** of our documentation pages as the live site serves them, or about **seventy** of the same pages as this site serves them. | Page | Today, on `docs.seon.io` | On the new docs, as Markdown | Cheaper by | Content kept | |---|---:|---:|---:|---:| | Workflow API | 96,495 | 3,917 | **25×** | 109% | | Orchestration integration | 60,281 | 4,551 | **13×** | 139% | | eKYC | 118,868 | 1,820 | **65×** | 124% | | Proof of Address | 116,261 | 1,211 | **96×** | 111% | **Content kept** is the guard against the obvious objection — that a page could be made cheap simply by deleting things. Every page here carries *more* than the original, so the savings come from dropping markup, not substance. {% callout type="note" title="How this was measured" %} Both sites were fetched with `curl`, following redirects, taking exactly what the server returns. Text was tokenised with `cl100k_base`. The two right-hand columns use different baselines on purpose, because they answer different questions. **Cheaper by** compares the whole response, since that is what an agent actually downloads and pays for. **Content kept** compares only the live page's article — its `
` element, without the navigation and sidebar — against our Markdown, since that is what tests whether the substance survived. Counting the live site's navigation as content would have flattered these numbers rather than tested them. {% /callout %} ## What each site offers an agent Every row below was probed against both live sites. | Capability | `docs.seon.io` | This site | What it means for an agent | |---|---|---|---| | `/llms.txt` index | {% yes label="real, text/plain" /%} | {% yes /%} | How it discovers what exists at all. Without one, it crawls or guesses URLs. | | Per-section `llms.txt` | {% no /%} | {% yes /%} | Lets it pull just the Fraud API without ingesting the entire corpus first. | | `/llms-full.txt` for bulk ingestion | {% no label="404" /%} | {% yes label="219 KB" /%} | One request instead of hundreds when vectorising or working offline. | | Page as raw Markdown, `.md` | {% no label="404" /%} | {% yes /%} | The whole ballgame: 3,917 tokens instead of 96,495 for the same page. | | Page as raw Markdown, `/index.md` | {% no label="404" /%} | {% yes /%} | Agents guess differently. Serving both means neither has to know our conventions. | | Content negotiation on `Accept: text/markdown` | {% no label="returns HTML" /%} | {% no label="returns HTML" /%} | Would let an agent holding only the canonical link get Markdown with no path convention at all. | | `/robots.txt` | {% no label="404" /%} | {% yes /%} | The one standard channel for telling a crawler what not to ingest. | | Deprecated pages kept out of agent indexes | {% no /%} | {% yes /%} | Stops a coding assistant confidently recommending an integration path we no longer support. | | Copy-as-Markdown control in the page | {% no /%} | {% yes /%} | Lets a person hand the page to their own agent in one click, correctly. | | `` in the head | {% no /%} | {% yes /%} | An agent that landed on the HTML learns a cheaper representation exists, without guessing a path. | | Markdown served as `text/markdown` | {% no /%} | {% no label="octet-stream" /%} | Content-type-sensitive agents may skip a file they cannot identify. Ours needs a host-side fix. | {% callout type="tip" title="Credit where it is due" %} `docs.seon.io` already publishes a genuine `/llms.txt`, correctly served as `text/plain`, listing the whole knowledge base. Any claim that the current docs ignore agents entirely is wrong. What is missing is everything downstream of discovery: having found a page, an agent still has no way to fetch it as anything but a 96,000-token HTML document. {% /callout %} ## Gaps on the current documentation 1. **No Markdown representation.** `llms.txt` points an agent at pages it can only then read as full HTML. Discovery without cheap retrieval. 2. **No `llms-full.txt`.** Bulk ingestion means crawling every page at full HTML cost. 3. **`/robots.txt` returns 404.** Crawlers get no directives at all, so there is no way to steer them away from superseded pages. 4. **No way to mark a page superseded.** Legacy integration paths read to an agent exactly like current ones, so a coding assistant can confidently recommend an approach we no longer support. 5. **Hydration overhead.** Up to half of each page is a duplicate copy of its own content in JSON. ## Gaps on this site Stated with the same directness, because a measurements page that only flatters its author is worthless. Two remain, and both live in the web server rather than in this repository — the deploy is an `rsync` of static files, so nothing here can set a header. 1. **Markdown is served as `application/octet-stream`.** It should be `text/markdown; charset=utf-8`. Browsers download the file instead of showing it, and content-type-sensitive agents may skip it. The fix is one line in the nginx config on the host: ```nginx types { text/markdown md; } location ~ \.md$ { add_header Vary "Accept"; charset utf-8; } ``` 2. **`Accept: text/markdown` is not honoured.** The canonical URL returns HTML on both sites. The `.md` paths work; header negotiation does not, and it also needs host access. A sketch is in [Reading these docs as an agent](ai-consumability.md). 3. **Rendered HTML is still HTML.** Serving this site's own page costs 54,076 tokens against 3,917 for its Markdown. Markdown is not an optimisation of HTML; it is a different representation, and agents should be sent to it. Every page now advertises its Markdown twin in the head, which is the part we can do without the server: ```html ``` Fixed since the first version of this page: the broken canonical on `document-verification.md`, and the absence of `` in the head — every page now points search engines at its `docs.seon.io` original, so this parallel site cannot compete with the real documentation in search. ## Reproducing this ```bash # What the canonical URL actually returns curl -sSL https://docs.seon.io/api-reference/workflow-api | wc -c curl -sSL https://hackathon.seondf.net/api-reference/workflow-api.md | wc -c # What each site offers an agent curl -sSI https://docs.seon.io/llms.txt | head -1 curl -sSI https://docs.seon.io/llms-full.txt | head -1 curl -sSI https://hackathon.seondf.net/llms-full.txt | head -1 ``` Token counts used `cl100k_base` via `tiktoken`. Figures were taken on 2 September 2026; the live site changes, so re-run rather than trusting them.