Crawl Efficiency

Informational in 3.x — runs and reports but does not affect your AX score yet (it will gain weight in v4.0). Bot traffic is projected to exceed human traffic by 2029; how cheaply agents can crawl your pages increasingly matters. This check measures compression, conditional-request support, and response size.

Response is not compressed

Your server returned the page without a Content-Encoding. Compression cuts text payloads by roughly 70–80%, reducing transfer cost for every agent and bot. Brotli (br) typically beats gzip by another 10–20% on text.

# Nginx (gzip + Brotli module)
gzip on;
gzip_types text/html text/css application/javascript application/json image/svg+xml;
brotli on;
brotli_types text/html text/css application/javascript application/json image/svg+xml;

# Caddy — compression is automatic
encode zstd br gzip

On managed platforms (Vercel, Netlify, Cloudflare) compression is on by default — if this finding appears there, check that a proxy in front isn't stripping the header.

No ETag or Last-Modified

Without a cache validator, a crawler must download the full body on every visit — it has no way to ask "has this changed?". Emit an ETag (content hash) or a Last-Modified timestamp so revalidation becomes possible.

# Nginx — ETag is on by default for static files
etag on;

# Express — enabled by default; for dynamic responses set it explicitly
res.set('ETag', strongHashOfBody);

Conditional request not honored

Your server sends a validator but does not act on it. ax-audit re-requested the page with If-None-Match / If-Modified-Since and received the full body again instead of a 304 Not Modified. A 304 is a near-empty response — it saves the entire payload when content is unchanged.

curl -sI https://your-site.com | grep -i etag
# Then revalidate with the returned value:
curl -sI -H 'If-None-Match: "<etag>"' https://your-site.com
# Expected: HTTP/2 304

Static-file servers do this automatically. For dynamic responses, compare the incoming If-None-Match against your current ETag and return 304 when they match.

Oversized page

Large HTML documents inflate both crawl cost and the token budget agents spend reading them. Common culprits: inlined base64 images, large embedded JSON state, and un-split single-page content.

Move inlined images to real image URLs with alt text.
Trim or lazy-load large embedded data blobs.
Offer a Markdown representation to agents — see the Content Negotiation guide; it cuts tokens ~80%.

Homepage request failed

ax-audit could not retrieve the homepage, so efficiency could not be assessed. Resolve general reachability — see the other failing checks in your report — and re-run.