Skip to content

feat(md-exports): Return helpful markdown 404 for agent requests#17868

Open
sergical wants to merge 1 commit into
masterfrom
sdybskiy/helpful-md-404
Open

feat(md-exports): Return helpful markdown 404 for agent requests#17868
sergical wants to merge 1 commit into
masterfrom
sdybskiy/helpful-md-404

Conversation

@sergical
Copy link
Copy Markdown
Member

DESCRIBE YOUR PR

When AI agents request a non-existent .md page (e.g., /platforms/javascript/ai-monitoring.md), they currently get a 0-byte 404 response — a dead end with no guidance on where to go next.

This adds a dynamic catch-all route at app/md-exports/[...path]/route.ts that returns a proper markdown 404 with:

  • YAML frontmatter (title, url)
  • Sibling page suggestions from the closest ancestor in doctree.json
  • Links to llms.txt, index.md, and platforms.md

Example response for /platforms/javascript/ai-monitoring.md:

---
title: "Page Not Found"
url: "https://docs.sentry.io/platforms/javascript/ai-monitoring"
---

# Page Not Found

The page `/platforms/javascript/ai-monitoring` does not exist.

## Pages in Browser JavaScript

- [Installation Methods](https://docs.sentry.io/platforms/javascript/install.md)
- [Capturing Errors](https://docs.sentry.io/platforms/javascript/usage.md)
- [Browser AI Monitoring](https://docs.sentry.io/platforms/javascript/ai-agent-monitoring-browser.md)
...

## Find what you need

- [Site index](https://docs.sentry.io/llms.txt)
- [Documentation root](https://docs.sentry.io/index.md)
- [Platforms](https://docs.sentry.io/platforms.md)

Key design decisions:

  • Static .md files in public/md-exports/ still take priority — the catch-all only fires when no pre-generated file exists
  • Supports both docs.sentry.io and develop.sentry.dev via isDeveloperDocs
  • doctree.json included in outputFileTracingIncludes for Vercel deployment
  • Graceful fallback if doctree loading fails (generic 404 with navigation links)

Context: ~7,500 markdown rewrites/month from agent traffic (99.97% via User-Agent detection). Agent 404s on .md requests were completely invisible — no metrics, no spans, no logs — because the static file miss bypasses all instrumentation.

IS YOUR CHANGE URGENT?

  • Urgent deadline (GA date, etc.)
  • Other deadline
  • None: Not urgent, can wait up to 1 week+

SLA

  • Teamwork makes the dream work, so please add a reviewer to your PRs.
  • Please give the docs team up to 1 week to review your PR unless you've added an urgent due date to it.
    Thanks in advance for your help!

PRE-MERGE CHECKLIST

  • Checked Vercel preview for correctness, including links
  • PR was reviewed and approved by any necessary SMEs (subject matter experts)
  • PR was reviewed and approved by a member of the Sentry docs team

When AI agents request a non-existent .md page (e.g.,
/platforms/javascript/ai-monitoring.md), they currently get a 0-byte
404 response with no guidance. This adds a dynamic catch-all route that
returns a proper markdown 404 with YAML frontmatter, sibling page
suggestions from doctree.json, and links to llms.txt/index.md.

Static .md files in public/md-exports/ still take priority — the
catch-all only fires when no pre-generated file exists.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@vercel
Copy link
Copy Markdown

vercel Bot commented May 24, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
develop-docs Ready Ready Preview, Comment May 24, 2026 12:55pm
sentry-docs Ready Ready Preview, Comment May 24, 2026 12:55pm

Request Review

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants