Generate and inspect an XML sitemap

An XML sitemap is a file that lists URLs you want search engines to discover and crawl. For a small site, the useful part is not generating the XML; it is deciding which URLs are canonical, public, and worth including, then keeping the list in sync as the site changes. Submission can help discovery, but it does not guarantee that a page will be indexed or rank.

Decide what belongs in the sitemap

Start with a list of pages that return a successful response and are intended for public search. Usually that means canonical pages with useful content. Do not automatically include every URL your application can render.

Review each candidate:

• Is it accessible to visitors without an account?
• Does it return the final page instead of redirecting somewhere else?
• Is it the preferred canonical URL, rather than a duplicate parameter or alternate path?
• Is it allowed to be crawled and intended to appear in search?
• Does the page provide enough value to justify discovery?

Common exclusions include login pages, internal search results, filter combinations that create duplicates, staging URLs, redirects, deleted content, and pages marked noindex. A sitemap should not contradict page-level signals. If a page is intentionally hidden from search, do not list it as a preferred URL.

Google’s official overview explains sitemap formats and their role: Build and submit a sitemap. Check current requirements there before relying on size limits or submission behavior.

Create a minimal example

A basic sitemap uses the sitemap XML namespace and a <url> entry for each preferred address:

<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
  <url>
    <loc>https://example.com/guides/image-formats</loc>
  </url>
  <url>
    <loc>https://example.com/about</loc>
  </url>
</urlset>

Use absolute URLs with the correct protocol and hostname. XML requires special characters to be escaped appropriately; do not paste raw ampersands in query strings without encoding them. Keep the sitemap at a stable URL, commonly /sitemap.xml, and serve it with a suitable XML content type.

The optional lastmod value should represent a real, meaningful page modification date. Do not update every date every day just to make the sitemap look fresh. Search engines may use the value as a hint, but inaccurate metadata undermines trust in the file.

Generate from the site’s source of truth

For a site with a content management system, build the sitemap from the published, canonical page records rather than maintaining a hand-copied list that drifts. For a static site, generate it from route data and published content. Include redirects only through their final destination, and make sure draft pages remain out.

A useful generation pipeline is:

1. Gather public routes from your CMS or route manifest.
2. Filter out drafts, duplicate variants, private routes, redirects, and noindex pages.
3. Resolve the canonical hostname and protocol.
4. Escape XML content and serialize valid XML.
5. Validate the output and compare its URLs with the route list.
6. Deploy the sitemap at a stable location and reference it from robots.txt if appropriate.

For a small manually maintained site, keep a simple checklist beside the sitemap and review it whenever a page is published, removed, or moved. A Toolsnix XML Sitemap Generator can help format a candidate list, but it cannot determine which URLs your site should expose or confirm that they return the right status.

Inspect the generated file

Open the sitemap URL directly. Check that it is valid XML, contains no template placeholders, and lists only URLs on the intended host. Click or request a sample of entries and confirm the response code, canonical tag, robots directives, and page purpose.

Look for mistakes such as:

• http URLs when the canonical page uses https.
• A nonpreferred hostname, such as an apex domain when the site canonically uses www.
• URLs with tracking parameters or session identifiers.
• Old paths that redirect after a site migration.
• Private or staging routes accidentally exposed in the list.
• A lastmod date unrelated to a real update.
• Malformed XML from an unescaped ampersand or special character.

A sitemap can itself be publicly fetched. Never put sensitive paths, tokens, customer IDs, or private content in it. Excluding a URL from a sitemap is not an access-control mechanism; protect private pages with proper authentication and authorization.

Submit and monitor without overreading the result

You can submit the sitemap in a search engine’s webmaster tools and list its location in robots.txt. Submission tells the service where the file is. It does not order the service to crawl every page, and it cannot make low-value, duplicate, blocked, or technically inaccessible pages indexable.

After submission, monitor processing errors and sample affected URLs. If a page is missing from search, investigate its response status, canonical, robots rules, internal links, and content quality. Do not keep adding every possible URL to the sitemap as a substitute for fixing those issues.

When a route changes, update the sitemap to the new canonical path and retain an appropriate redirect from the old path. When content is deleted without a replacement, remove it from the sitemap; decide separately how the old URL should respond.

Coordinate sitemap, robots, and canonical signals

A sitemap is only one source of crawl information. Check it alongside the site’s robots.txt, page-level robots directives, canonical tags, redirects, and internal links. If the sitemap lists a URL that robots rules block, the signals conflict. If an alternate URL redirects to a canonical page, list the final preferred URL rather than the redirecting one.

Do not use robots.txt as a way to hide confidential pages. It is a public crawl instruction, not authentication; a blocked URL may still be known from links elsewhere. Protect private pages with access controls and avoid publishing sensitive paths in a sitemap.

If your site uses multiple languages, follow the current search-engine guidance for alternate language relationships and include only valid canonical pages. Language annotations can be complex and need to match the actual page set. Confirm every referenced language URL exists, is indexable as intended, and links back consistently. Flag uncertain locale behavior for review rather than generating tags from a guessed rule.

Keep publication state in the data model

The safest sitemap generator derives entries from the same source that decides whether content is public. A CMS can filter by published state; a static site can use the same frontmatter or route manifest used to generate pages. This avoids listing a draft that has not shipped or forgetting a page that was just published.

For a very small site, a reviewed static list can be enough. Add a check to the release process that requests each listed URL and checks for an unexpected redirect or error. A script can catch technical mismatches, but a person should still decide whether a page belongs in search.

Treat sitemap dates as editorial data

Use lastmod only when you can provide a meaningful modification date. If a title typo is fixed, the page may have changed less materially than when its instructions or facts are revised. Your content system should define what counts as a substantive update and maintain the date accordingly.

Avoid changing every lastmod value simply because the sitemap regenerated. Search engines may ignore unreliable dates, and the date no longer helps a reader or crawler distinguish real changes. Keep publication dates, modification dates, and build timestamps separate in the data model.

Diagnose common sitemap errors

A sitemap URL returning an error can come from deployment routing, file permissions, content negotiation, or a malformed generated file. Request the exact public URL without being logged in. Confirm the response is successful and the body is XML rather than an HTML error page with an .xml suffix.

If a search console reports a parsing error, inspect the line it identifies and nearby entries. XML is stricter than ordinary HTML: every element must close correctly, attributes must be quoted, and reserved characters in text must be escaped. Test the file with an XML parser before deployment. Do not attempt to repair malformed XML by hand after each content edit if the generator can be fixed.

A URL discovery error may not mean the sitemap syntax is wrong. The URL could redirect, be blocked, return a not-found response, or point to a page with a canonical elsewhere. Check individual entries and fix the source route data. A valid XML file can still contain poor URLs.

Avoid oversized or fragmented inventories

Most small sites can use one sitemap file. Larger sites may need a sitemap index that references multiple sitemap files, each with a subset of URLs. The sitemap protocol and search-engine guidance set limits; verify current limits before designing a large export rather than relying on an old number.

Split files by a useful ownership or content boundary when it helps diagnostics, not just to create more files. Give each file a stable public location and ensure the index lists the exact paths. When one section has an error, a structured split can make it easier to identify whether the issue is blog content, product routes, or another source.

Do not generate a sitemap for every query parameter combination. That can create an unbounded number of low-value or duplicate URLs. Define canonical parameters, filter rules, and excluded routes in the generator and review samples after changes to search or filtering features.

Keep sitemap changes reviewable

When a site publishes or retires content, review the sitemap diff alongside the content change. A new public article should usually add a canonical URL; a draft should not. A renamed path should remove the old URL and add the new destination after the redirect and canonical metadata are in place.

If a page remains public but is intentionally not for search, record that decision in the content system and make the sitemap generator honor it. Do not rely on a manual cleanup months later. A sitemap is most useful when inclusion policy is clear enough for the build process to apply consistently.

FAQs

Does every small website need an XML sitemap?

No. Search engines can discover pages through links, and a very small, well-linked site may not need one. A sitemap is useful when it improves discovery or gives you a clear inventory of preferred public URLs.

Does a sitemap guarantee indexing?

No. It communicates URLs to search engines, but indexing decisions are made separately. A listed page can still be excluded for technical or quality reasons.

Should I include pages marked noindex?

Generally do not list a URL as a preferred indexable page when you intentionally mark it noindex. Keep your sitemap and page directives consistent.

How often should a sitemap be updated?

Whenever the set of published canonical pages changes. Automated generation from a reliable content source can keep it current without artificially changing modification dates.

Next step

Generate the sitemap from published routes, inspect representative entries, and compare it with your canonical and indexing rules. Submit it only after the file accurately describes the pages you want discovered.