ABC Tools

XML Sitemaps, Explained

After reading this you will know what an XML sitemap does, what belongs in one, how lastmod earns its keep while changefreq and priority do not, and how to split a site that outgrows a single file.

What an XML sitemap is

An XML sitemap is a text file that lists the URLs on your site you want a search engine to know about. Each URL sits inside a <url> element, and the whole list lives inside a <urlset> root. That is the entire job: a machine-readable index of your canonical pages.

Here is a minimal example with two pages:

<url><loc>https://example.com/</loc></url> and <url><loc>https://example.com/about</loc></url>, wrapped in one <urlset> with the namespace http://www.sitemaps.org/schemas/sitemap/0.9.

A sitemap is a hint, not a command. It tells a crawler that a URL exists and, optionally, when it last changed. It does not force indexing, and it does not raise rankings. A page that a crawler would never find through links can still get discovered if it appears in a sitemap, so the file mostly helps large sites, new sites, and sites with pages that few internal links point to.

When to use one, and when it does little

A sitemap earns its place when discovery is the bottleneck. Consider four cases where it helps:

  • A new site with few external links pointing in.
  • A large site (say 10,000 or more URLs) where crawling every page by following links takes many days.
  • Pages buried deep in the structure, reachable only after five or six clicks.
  • Content that changes often, where an accurate lastmod tells the crawler what to revisit.

A sitemap does little for a tidy 30-page brochure site where every page links to every other page. The crawler will find all 30 pages on its first visit regardless. Submitting a sitemap there costs you nothing, but expect no change in coverage.

A sitemap is not a way to fix indexing problems. If a page is blocked by robots.txt, carries a noindex tag, or returns a 404, listing it in a sitemap will not get it indexed. The sitemap only surfaces the URL; the page still has to be crawlable and worth keeping.

What each field means

Beyond the required <loc>, the sitemap protocol allows three optional child elements. Their value in practice differs sharply.

loc
The full, canonical URL of the page, including the scheme (https://). This is the only required element.
lastmod
The date the page last changed, in W3C format such as 2024-06-15 or a full timestamp. Google uses this when it is accurate and consistent.
changefreq
A hint about how often the page changes: daily, weekly, and so on. Google ignores it.
priority
A number from 0.0 to 1.0 suggesting relative importance. Google ignores it too.

The lesson is blunt: spend your effort on lastmod and leave the other two out or leave them at defaults. A changefreq of daily on a page that has not changed in two years teaches the crawler nothing, and a priority of 1.0 on every URL carries no information because it does not distinguish anything.

The lastmod rule, with the intuition

The reason lastmod works and the other two do not comes down to verifiability. A crawler can fetch the page, hash its content, and compare that against what it fetched last time. If your stated lastmod keeps matching the truth, the crawler learns to trust it and can skip pages it knows are unchanged. If your lastmod lies (for example, if it updates to today on every page every night), the crawler learns to ignore it.

Think of it as a trust score. Let m be the fraction of your lastmod claims that match reality:

m = \frac{\text{URLs where lastmod matches actual change}}{\text{total URLs with a lastmod}}

Here m ranges from 0 to 1. When m \approx 1, the crawler can rely on the dates and recrawl efficiently. When m drops toward 0.5 or below, the signal is noise and gets discarded. There is no partial credit worth chasing: aim for m = 1 or omit the field.

If you cannot generate accurate lastmod values, leave the field blank. An absent lastmod is honest. A wrong one is worse than none because it trains the crawler to distrust your whole file.

A worked example

Take a small site with four pages and put them through the generator using its default settings.

Four URLs into a valid sitemap

Paste these four lines, one per line:

  1. https://example.com/
  2. https://example.com/about
  3. https://example.com/products
  4. https://example.com/contact

The generator wraps each line in a <url> block. With lastmod set to 2024-06-15 and changefreq and priority left off, the first entry becomes:

<url><loc>https://example.com/</loc><lastmod>2024-06-15</lastmod></url>

The finished file holds 4 <url> elements inside one <urlset>. It is well under the 50,000 URL and 50 MB limits, so no splitting is needed. You download it as sitemap.xml, upload it to your site root, and submit the URL https://example.com/sitemap.xml in Search Console.

The math that matters here is size, not content. Four URLs is far below the ceiling, so the file stays a single sitemap. The next section shows where that ceiling starts to bind.

Splitting large sites and the size limits

One sitemap file may contain at most 50,000 URLs and weigh at most 50 MB uncompressed. Cross either limit and you split the list into several files, then list those files in a sitemap index, which is itself an XML file whose entries point to sitemaps rather than pages.

The number of sitemap files you need is the ceiling of your URL count divided by 50,000:

n = \left\lceil \frac{U}{50000} \right\rceil

Here U is your total URL count and n is the number of sitemap files. A site with 120,000 URLs needs n = \lceil 120000 / 50000 \rceil = \lceil 2.4 \rceil = 3 files. A site with 50,001 URLs already needs 2.

Files needed jumps from 1 to 2 the moment you pass 50,000 URLs, then rises one step per additional 50,000.

The index itself can list up to 50,000 sitemaps, so at 50,000 files of 50,000 URLs each you could in theory cover 2.5 billion URLs. You will hit other problems long before that. Gzip compression does not change the 50,000 URL cap and does not change the 50 MB uncompressed measurement; it only reduces transfer size.

With 50,000 URLs per file, a site of 120,000 URLs needs 3 sitemap files (two full files of 50,000 and one partial file of 20,000), plus one sitemap index that lists all three.

Common mistakes

Most broken sitemaps fail for a handful of reasons. Check for these before you submit:

  • Listing non-canonical URLs. If /products?sort=price and /products are the same page, list only the canonical one. Duplicates waste the crawl budget.
  • Including blocked or noindexed URLs. A URL that robots.txt disallows should not appear. The two signals contradict each other and the page will not be indexed anyway.
  • Listing redirects. A URL that returns a 301 should be replaced by its target. Listing the redirect makes the crawler follow an extra hop for nothing.
  • Relative URLs. Every <loc> needs the full absolute URL with scheme. /about is invalid; https://example.com/about is correct.
  • Stale or faked lastmod. Covered above. Accurate or absent, never invented.

Related tools

Before you submit a sitemap, confirm that every URL in it is actually reachable. Use the robots.txt Tester to check that none of your listed URLs are blocked by a disallow rule, and the Redirect Chain Checker to catch any URL that redirects, so you can list the final target instead of the hop.

Frequently asked questions

Does a sitemap improve my rankings?

No. A sitemap affects discovery and crawling, not ranking. It can get a page found and indexed faster, but once a page is indexed the sitemap plays no part in where it ranks.

Where do I put the sitemap file?

Upload it anywhere on your site and reference it. The common location is the site root, for example https://example.com/sitemap.xml. Add a line to robots.txt such as Sitemap: https://example.com/sitemap.xml, and submit the same URL in Search Console.

How many URLs can one sitemap hold?

At most 50,000 URLs and 50 MB uncompressed. Past either limit, split into multiple files and list them in a sitemap index. A site of 200,000 URLs needs \lceil 200000 / 50000 \rceil = 4 files.

Should I set changefreq and priority?

You can, but Google ignores both. Setting them causes no harm and gains nothing. Put your effort into an accurate lastmod instead.

How often should I resubmit?

You do not need to resubmit manually after the first submission. When the file at your sitemap URL changes, crawlers pick up the new version on their next fetch. Resubmit only if you move the file or want to prompt a re-read after a large change.

Try it yourself in the tool

Related tools

Redirect Chain Checker

Follow every hop: 301s, 302s, loops and chain length.

robots.txt Tester

Test which URLs a robots.txt blocks, rule by rule.

Keyword Density Checker

Word and phrase frequencies of any text, with stopword filter.

Meta Tag & Open Graph Generator

Generate meta tags and preview the Google, Facebook and X cards.

SERP Snippet Preview

Pixel-accurate preview of your Google result on desktop and mobile.

UTM Campaign URL Builder

Build tagged campaign URLs with correct encoding.