--- title: "Future-Proofing Your Site for AI Agents with `.md` Routes and `/llms.txt`" description: "SanityPress serves clean markdown from any URL via .md routes and a live /llms.txt index: two layers of AEO built for the emerging world of AI agent crawlers." --- # Future-Proofing Your Site for AI Agents with `.md` Routes and `/llms.txt` ![first-person persective of an orange crab claw holding an iPhone in Safari view with the text "SanityPress" queried. he's inside a cafe with employees who are also crabs brewing coffee (slight focus blur).](https://cdn.sanity.io/images/cyu7k2r0/production/feb93d16750af6a1ca6750bed6c25683c3077040-1448x1086.png) Search engines have always had _crawlers_—automated programs that read your pages, parse your content, and use it to answer user queries. For two decades, SEO (Search Engine Optimization) was how you made sure those crawlers understood your site correctly. Now there’s a new class of crawler: AI agents. Tools like ChatGPT, Claude, Perplexity, and Google’s AI Overviews don’t just index your content. They read it, synthesize it, and surface it in conversational answers. **AEO (Answer Engine Optimization)** is the emerging practice of structuring your content so these AI-powered tools can accurately find and represent it. Where SEO optimized for ranking, AEO optimizes for being understood. The core challenge: HTML is noisy. Your pages are full of navigation menus, scripts, style attributes, and layout scaffolding that means nothing to an AI trying to understand what your page is about. What agents actually need is clean, structured text. Every `page` and `blog.post` in SanityPress now has a `markdown` field. Fill it in, and that document gets a clean `.md` route built for AI agents. Leave it empty, and no route is generated. It’s a small field with a big job: handing the machines that read your site a version of your content with none of the layout noise. ## Why a markdown field? AI agents like ChatGPT, Claude, and Perplexity increasingly read sites directly. HTML is noisy for them: navigation menus, scripts, and styling that mean nothing to a model trying to understand your page. What they actually want is clean, structured text. For two decades we shaped that text for search engines. Now a new kind of reader matters just as much, and it rewards content it can parse without wading through markup. Rather than guess at that text by scraping your rendered HTML, SanityPress now lets you author it. The `markdown` field sits right next to your content in the Studio, on both pages and blog posts, so the machine-readable version is something you own. ## How `.md` routes work Append `.md` to any page or post URL. If that document's `markdown` field is populated, the route serves it verbatim as `text/markdown`. If the field is empty, there is no `.md` route and the request returns a 404. - `/` serves the HTML Homepage page, for humans - `/index.md` serves your curated markdown, for agents - `/about` serves the HTML About page, for humans - `/about.md` serves markdown, for agents - `/blog/my-post` serves the post, for humans - `/blog/my-post.md` serves markdown, for agents Responses are cached at the edge, so agents get fast, stable markdown without re-rendering your pages on every visit. That keeps things cheap to serve and quick for crawlers that fetch many URLs at once. ![Editing a page document's markdown content (used for .md routes)](https://cdn.sanity.io/images/cyu7k2r0/production/bf16fd63ae59b62cd14bea203fefc73502df69cd-3248x2122.png) ## What goes in the field Whatever serves the reader best. For most pages that is a faithful markdown copy: headings, prose, lists, and links. For others it might be a tighter summary. The field is plain markdown, so you are free to shape it however a model would read it most easily. ## You decide what agents see Because the markdown is authored rather than scraped, you control it completely. Trim a long page to its essentials, restructure it for a model, or summarize it for quick consumption, all without touching the version humans see. It’s the difference between hoping a converter did a good job and knowing exactly what the model receives. The same field powers `/llms.txt`, the root-level index that tells agents what content exists and where to find it. Only documents with a populated markdown field appear there, so your index always points to routes that actually exist. ### Writing all that markdown by hand? You have help. Sanity's own AI Assist plugin can take a swing at it, but honestly it’s been flaky for me. It couldn’t even produce the right image URLs, it kept whining that it could not see the `asset._ref` values 😅. So my preference: just let Claude generate the markdown for you. A future post walks through that workflow. > **Ready to make your site legible to AI agents?** > [Get started with SanityPress](/docs/getting-started) 🚀 ![an orange crab in the style of disco ball with reflecting mirror squares. has the orange Claude logo on it's large claw. 3:2 aspect ratio](https://cdn.sanity.io/images/cyu7k2r0/production/7410ff1dc6e058b6757a6ff29d616c37eab2829b-1536x1024.png)