XML Sitemaps: What Are They & Why You Need One
If you know anything about search engine optimization, you know that Search engines like Google, Yahoo and MSN use website crawlers (also called robots or spiders) to identify new and updated pages to include in their index. What you may not know, however, is that there are other ways that a search engine can get information about pages that it might include in search result pages.
Google recently published a whitepaper, Sitemaps: Above and Beyond the Crawl of Duty, that examines the effectiveness of XML sitemaps.
XML sitemaps help web site owners let search engines know which pages are on their site, and which pages should be indexed. The paper tells us that approximately 35 million websites publish XML sitemaps, as of October 2008, providing data for several billion URLs.
If you frequently make changes to your site by adding new content or landing pages, it’s important to have an XML sitemap that helps search engines index your website.