All tools

XML Sitemap Generator

Enter a URL and this crawls your site's internal pages (up to 500, same site only, robots.txt-aware) to build a clean XML sitemap you can submit to search engines. It leaves out the pages that do not belong in a sitemap and says why. Any URL can be crawled, not just a tracked Dodolytics website. Free tool, nothing you crawl is stored.

Addresses read so far

The newest is at the top. The full result shows when the crawl is done.

Result	Address

How to use

1
Enter your site's URL in the field above. A bare domain or a full URL both work.
2
Click Generate sitemap. This crawls the site's own internal pages: same site only, respecting robots.txt, up to 500 addresses. You see each address while the crawl runs.
3
Look at the "Left out" list to see which addresses are not in the file and why. Then download the result as sitemap.xml and upload it to your site's root.

Tip: you can generate a sitemap for any publicly reachable site, not just a tracked Dodolytics website.

What is an XML sitemap?

An XML sitemap is a file, usually sitemap.xml, listing the URLs on a website that you want search engines to know about. It helps them discover and crawl your pages, especially ones that are hard to find through normal link-following: brand new pages, pages with few internal links pointing at them, or a site with a large or complex structure.

How do I use it?

Enter your site's URL above. This tool crawls its internal pages: same site only, respecting robots.txt, up to 500 addresses in one run. It keeps only the pages that belong in a sitemap: a page that answers with an error, a page marked noindex, a copy of a different page, and a redirect are left out, and the result says why for each one. Download the result as sitemap.xml, upload it to your site's root, and submit its URL to search engines (for example via Google Search Console or Bing Webmaster Tools, or by referencing it in your robots.txt). A page gets a <lastmod> date only when its server sends the date of the last change. The tool never makes up a date, and it leaves out <changefreq> and <priority>, which a crawl cannot know and which Google does not read.

Why is it useful?

Helps search engines discover and index your pages faster, especially newer or deeper pages that few other pages link to yet.
Gives search engines a clear, explicit list of the real pages that make up your site, rather than relying entirely on them following links to find everything.
A missing or broken sitemap can mean some of your content never gets indexed at all, especially on larger sites.
Free, no sign-up required, and nothing about a crawl is stored: generate a sitemap for any site, not just one you already track.

FAQ

Is this tool free?

Yes, no login required either.

Do you store the sites I crawl?

No. Nothing about a crawl (the URL, the pages found, or that it happened at all) is saved anywhere. Results are shown once and gone.

Does this respect robots.txt?

Yes. The crawl reads only the pages that robots.txt permits (Allow and Disallow rules, with * and $), and it stays on the site of the URL that you enter. It never follows links out to other sites.

Why is a page left out of the sitemap?

A sitemap must list only the pages that you want in search results. So the tool leaves out a page that answers with an error, a page marked noindex, a copy that names a different page as its canonical address, a redirect, a file that is not a web page, and a page that robots.txt blocks. The "Left out" list gives the reason for each address.

Does the sitemap include lastmod, changefreq, or priority?

It includes lastmod for each page whose server sends the date of the last change (the Last-Modified header). A page with no such date gets no lastmod: the tool does not guess. It leaves out changefreq and priority, which a crawl cannot know and which Google does not read.

How many pages can it crawl?

Up to 500 addresses in one run, with a time limit of about 40 seconds. If a site has more, the result is marked partial, and the sitemap has the pages found so far.

My site has a sitemap already. Why use this?

To check it. The tool reads the sitemap that your site has and tells you how many of the pages that it found are not in that file.

Does a download crawl the site again?

No. Your browser makes the file from the result that is on the page. Only a browser with scripts off posts the form again, and the server then crawls a second time, because nothing is stored.

Related tools