Collection diagnostics · FIELD GUIDE
Explain different crawler responses using Vary
Compare language and representation selection before labeling different responses at one URL as an indexing or analytics defect.
What the evidence can establish
Content negotiation allows a server to choose a representation using request headers. MDN describes Accept-Language as a language preference and Vary as the response header that names request fields used in selection. A URL-only comparison can therefore overlook meaningful request context. Record the relevant request headers and response language or content fingerprint before interpreting differences as a crawler-specific failure.
Hypothetical example: a browser requesting one language receives a localized page while a diagnostic client with a different preference receives another version. The first question is whether representation selection is intentional and consistent. If a shared cache serves the wrong version under repeated controlled conditions, investigate how its cache selection matches the documented Vary behavior. Do not jump from a language mismatch to a claim that an AI service was misled.
The saved evidence used here
This guide interprets dated official documentation. It does not report traffic from a customer or a measured AI answer. Each material claim is mapped to a source below.
- c1 · Explanation and verification steps
- Accept-Language communicates language preferences. Source 1
- c2 · Explanation and verification steps
- Vary lists request headers used in representation selection so caches can account for them. Source 1
The diagram explains the investigation or a clearly labeled hypothetical example. No measured counts or platform outcomes are added.
AI answer visibility: Not measured. No original platform answers or verified citation records were collected for this guide. A request count is not an audience, ranking or citation count.
What to do next
- Capture the exact public URL and relevant request headers for each comparison. Keep cookies and authorization information out of the exported evidence.
- Inspect Vary and record the delivered language or a response fingerprint. Change one selection input at a time so the comparison can identify which input matters.
- Repeat equivalent requests through the actual delivery path. Compare the observed variants with the intended site behavior, including any explicit language selector.
- If the variants are unintended, correct the narrow negotiation or cache configuration and rerun the same matrix. Keep language selection separate from canonical and indexing checks.
How to check the result
The request-to-representation matrix is reproducible and matches the intended behavior. The report includes the selection context, rather than treating every response at one URL as interchangeable.
Limits of this conclusion
Vary is a declaration used by caches, not proof that every cache honored it. User-agent names and language preferences do not establish the identity or purpose of an AI platform request.
This is a saved, dated edition. Source documentation can change. This edition retains the original retrieval dates and claim mapping; follow the sources before applying version-specific guidance.
Sources and publication checks
Automated checks validate source availability, supported fields, duplicate wording, numeric figures and page configuration. This guide was prepared with AI assistance and checked against saved source passages in the same authoring session. The batch importer checks source fingerprints, quotation locations, wording overlap and required fields. No paid generation API or separate model-review pass was used for this batch. These checks can miss errors and are not expert or human review.
Obtained: 2026-11-02T19:15:16.193992+00:00
Location: The Accept-Language header / b24; The Vary response header / b31
Saved source fingerprint
80ffa5045dd6dcc8a06c739d1763f157d8cc6bd01e552e8d47a099cc646d7ce5