Sitemap

No sitemap found

ax-audit looked for a sitemap in two places and didn't find one:

• The Sitemap: directive in /robots.txt
• The default fallback path /sitemap.xml

Publish a minimal sitemap and reference it from robots.txt:

<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
  <url>
    <loc>https://your-site.com/</loc>
    <lastmod>2026-04-30</lastmod>
  </url>
  <url>
    <loc>https://your-site.com/about</loc>
    <lastmod>2026-04-15</lastmod>
  </url>
</urlset>

# robots.txt
User-agent: *
Allow: /

Sitemap: https://your-site.com/sitemap.xml

Sitemap is not valid XML

The response at the sitemap URL doesn't start with <?xml, <urlset>, or <sitemapindex>. Most often this means a 200 HTML error page is being served instead of the sitemap.

• Test directly: curl -i https://your-site.com/sitemap.xml
• Make sure the route doesn't fall through to your SPA index.html.
• Validate the output at xml-sitemaps.com/validate.

Unexpected Content-Type

Serve sitemaps with Content-Type: application/xml (or text/xml). A wrong content type — like text/html — causes some crawlers to discard the file before parsing.

# Nginx
location = /sitemap.xml {
  default_type application/xml;
}

# Express / Next.js custom server
res.setHeader('Content-Type', 'application/xml; charset=utf-8');

Empty <urlset>

Your sitemap is well-formed XML but contains no <url> entries. Add at least one URL per page you want crawlers to discover — at minimum the homepage.

Empty <sitemapindex>

A sitemap-index file must reference at least one child sitemap:

<?xml version="1.0" encoding="UTF-8"?>
<sitemapindex xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
  <sitemap>
    <loc>https://your-site.com/sitemap-pages.xml</loc>
    <lastmod>2026-04-30</lastmod>
  </sitemap>
  <sitemap>
    <loc>https://your-site.com/sitemap-blog.xml</loc>
    <lastmod>2026-04-29</lastmod>
  </sitemap>
</sitemapindex>

Child sitemaps unreachable

Your sitemap-index references child sitemaps that returned non-2xx responses. Crawlers may give up after the first failure and never reach the rest of your URLs.

• Verify each <loc> URL inside the index returns 200 OK.
• Use absolute URLs only — relative <loc> values are invalid per the sitemap spec.
• If a child sitemap is gated by auth or geo-blocked, move it outside the public sitemap.

Low <lastmod> coverage

Less than half of your <url> entries declare a <lastmod>. AI agents and search crawlers use lastmod to decide what to re-fetch — without it, every URL looks equally fresh (or equally stale).

<url>
  <loc>https://your-site.com/blog/post-1</loc>
  <lastmod>2026-04-30</lastmod>
</url>

Use ISO 8601 dates (YYYY-MM-DD or full timestamps). Most sitemap generators auto-populate this from the page's git history or filesystem mtime.

Stale <lastmod>

The newest <lastmod> in your sitemap is more than a year old. Crawlers interpret this as "nothing has changed on the site," which slows or stops re-indexing.

• Regenerate the sitemap on every deploy.
• Touch the lastmod when content actually changes — don't blanket-update every URL on every build (crawlers learn to ignore that).

Above the 50,000 URL limit

The sitemaps.org spec caps a single sitemap at 50,000 URLs. Split into a sitemap-index pointing to multiple smaller sitemaps:

/sitemap.xml              -- index
/sitemap-pages.xml        -- 1 to 5,000 marketing pages
/sitemap-blog-2025.xml    -- 1 to 30,000 blog posts from 2025
/sitemap-blog-2026.xml    -- ongoing blog posts

Above the 50 MB size limit

Same fix as above — split the sitemap. The 50 MB limit applies to the uncompressed file size. You can also serve sitemaps gzipped (sitemap.xml.gz) to reduce transfer size, but the uncompressed limit still applies.