What this guide will help you do
Turn an unreadable or low-quality sitemap into a dependable discovery list of canonical pages worth indexing.
Key takeaways
- A sitemap should list preferred canonical URLs, not every address the server can produce.
- Valid XML is only the first check; status codes, redirects, canonicals, robots directives, and content quality still matter.
- A submitted sitemap helps discovery but does not guarantee that every listed page will be indexed.
Start with the response, not the XML
Open the sitemap URL directly and check what the server actually returns. It should be publicly accessible, respond successfully, and serve XML rather than an HTML error, login screen, CDN challenge, or branded not-found page. A browser showing angle brackets is not enough evidence.
Follow redirects and update Search Console and robots.txt to the final sitemap URL. Keep the location stable. If the sitemap is generated dynamically, make sure a database timeout or empty cache cannot quietly turn it into a 500 response during a crawl.
Validate the document structure
A URL sitemap uses a urlset root and places each complete absolute URL inside a loc element. A sitemap index uses sitemapindex and points to child sitemap files. Escape reserved XML characters correctly and declare UTF-8 when that is the encoding being served.
Keep child sitemaps on an allowed site location and avoid circular indexes. Large sites should split files into logical groups, which also makes Search Console reporting easier to interpret. Compression is fine when the server and filename consistently represent the gzip content.
Remove URLs that contradict the sitemap
Each listed page should normally return 200, be crawlable, avoid a noindex directive, and identify itself as canonical. Remove redirected URLs, error pages, tracking parameters, search results, duplicate filters, and staging hosts. Point the sitemap directly at the final preferred address.
The sitemap is not a repair layer for poor site architecture. Important pages still need crawlable internal links. If a URL appears only in the sitemap and nowhere in the visitor journey, decide whether it is genuinely important or effectively orphaned.
Use lastmod honestly
A lastmod value should reflect a meaningful change to the page, such as revised main content, availability, or structured information. Replacing every date with today on every deployment makes the field less useful and creates noisy crawl signals. Connect it to content changes, not server build time.
Use a valid date format and derive it from trustworthy content data. If the CMS cannot produce an accurate date, omitting lastmod is better than publishing a fabricated one. Change frequency and priority values do not force a search engine to crawl or rank a URL.
Submit, monitor, and diagnose samples
Submit the final sitemap in Search Console and review whether it can be read, how many URLs are discovered, and whether indexing exclusions cluster by template. Inspect representative pages from each group instead of repeatedly resubmitting an unchanged file. Record the submission date and any reported parsing error.
When listed pages are crawled but not indexed, move beyond XML validation. Compare their usefulness, duplication, canonical selection, internal links, and rendered content. A clean sitemap improves discovery and diagnosis; it cannot make an unhelpful page index-worthy. Fix the page or consolidate it instead of blaming the file.
A practical next-step checklist
- Fetch the live sitemap and confirm a successful XML response.
- Validate whether it is a URL set or sitemap index.
- Remove redirects, errors, duplicates, non-canonical URLs, and noindex pages.
- Check that every priority URL has useful internal links.
- Submit the final URL and inspect exclusion patterns by page type.
When you make a technical change, test a small group of representative URLs first. Then run the free SEO audit to catch broken links, metadata gaps, crawl signals, and page-level regressions before applying the change across the whole site.
Frequently asked questions
Does submitting a sitemap guarantee indexing?
No. A sitemap supports discovery and supplies metadata. Search engines still decide whether a page is eligible and useful enough to index.
Should redirected URLs remain in an XML sitemap?
No. Replace them with the final canonical 200-status URLs so the discovery signal is direct and consistent.
Can a sitemap contain more than one website?
Sitemap location and ownership rules apply. In ordinary setups, keep URLs within the property and location the sitemap is intended to cover, or follow the documented cross-site submission requirements.
Primary references
Use these first-party references to confirm details that may change after this guide is published: