Generating a sitemap is really old days school for SEO and one of the important basics lessons in On-SEO optimization. I spent the last two days how providing you with the right tutorial to create an amazing sitemap for your site but found many users have a lot of wrong information about sitemap creation. So, let’s free our minds and go deep about the core of sitemap guidelines, generating, submitting, and fixing the most common errors.
What is XML Sitemap?
A sitemap is a structured list of links to a website accessible to search engine crawlers bots or users.
XML Sitemap allows webmasters to notify search engine crawlers about URLs on their websites that are available for crawling and indexing by including additional information about each URL like the last modified updated time, how often it changes, and how important it is with other URLs in the site. This makes search engines understand the complete structure of your site (Posts, Pages, Archives, and Categories) which lead to crawling your site more intelligently.
Types of Sitemaps
An XML Sitemap is a structured format that is not helpful for users or visitors and it’s useless for them because it’s mainly designed for search engines to tell them about the importance of URLs and the relation between them.
HTML sitemaps are designed for users to give them the most important content on your site and you don’t have to include all your website pages, unlike XML Sitemap. As it’s in HTML format, you can add visual elements like CSS to give it a more beautiful look because it’s for users not for bots.
Hint: Google Webmaster Tools Doesn’t support HTML sitemap format, only XML Format.
Do You Need XML Sitemap?
When you submit new website to Google Webmaster Tools, you get instant message from Google called “Improve the search presence of example.com” has five suggestion to maximize your site performance on Google Search. The fifth suggestion is “Submit a sitemap file” with a small description “This helps Google better understand how to crawl your site.” What do you need More?!!
If there is no other reasons more than “Better Understand for Crawler my site“, then its enough for me. Also here is some other good reasons for having sitemap.
- Your site is very big (e-commerce sites)
- Giving the search engines priority information
- Push the non-discoverable pages to search engine
- Correctly indexing the structure of your website
- Find and Index your new published content faster
- You have pages that not linked to each other well
- You have a lot of archive pages (because your site is too old)
- Your site is new and need to discovered by search engine and index fast
- Your site has multiple types of content ( Images, Video, News and Mobile)
- Websites built with Flash, only the homepage probably find and subsequent pages are unlikely to be found without an XML sitemap
Hint: Sitemap submission doesn’t guarantee you that all pages listed in your sitemap can be crawled or indexed because it depend on your content quality or maybe you have duplicate content issue.
Important XML Sitemap for SEO?
XML Sitemap and robots.txt are the guide for bots to discover your site correctly for better and fast crawling and indexing. As both are submitting to Google Webmaster Tools, this will help you to quickly see where search engines are having issues like duplicate content or Not found. It ensures that the spiders can easily reach all parts of your website and indexes your pages quickly, it’s actually a path record of your site and it’s a very important part of SEO.
Hint: It is very important to link your sitemap from your robots.txt file, This will help search engines to find your sitemap more easier to index it as fast as possible.
What is the Best WordPress XML Sitemap Plugins?
There are many users who think that any plugin could do the job but they are wrong because if you add new content constantly the plugin will generate a high load on your server if it’s not coded well or if your site is on the shared host has fewer resources than enough. So, you have to choose the sitemap generator plugin carefully to stay away from any headaches.
#1 Google XML Sitemaps
Google XML Sitemaps (by Arne Brachhold) is the best wordpress XML sitemap generator ever because its supported and active since over nine years and rated as the best WordPress plugin,
Once you installed, you are done and you don’t need to keep checking its function or fix issue like other plugins do. It has more than one million active installs.
Installation of Google XML Sitemaps
- Go to your wordpress Dashboard >> Plugins >> Add New >> Type “Google XML Sitemaps” in Search box, then hit install OR Upload the plugin folder to the /wp-content/plugins/ directory
- Activate the plugin through the Plugins menu in WordPress
- Customize settings located under Settings -> XML-Sitemap
Recommended Setting for Google XML Sitemaps
1- Basic Options:
Notify Google about updates of your Blog
Notify Bing (formerly MSN Live Search) about updates to your Blog
Add sitemap URL to the virtual robots.txt file. (if you don’t know how to add it manually)
Try to automatically compress the sitemap if the requesting client supports it.
Include sitemap in HTML format
2- Post Priority: check “Do not use automatic priority calculation” box
All posts will have the same priority which is defined in “Priorities”
3- Sitemap Content: (check boxes for the following options)
Include static pages
Include the last modification time.
Don’t check other options to prevent duplicate content index
4- Excluded items: choose which area in your site needs to be excluded from sitemaps like categories or posts
5- Change frequencies: Set Homepage, Posts, Pages, and Categories as “Always”, Others as “Weekly”
6- Priorities: Set these options as the following image
Don’t forget to click the “Update Options” Button
#2 Yoast XML Sitemap
WordPress SEO plugin (by Joost de Valk) is well known All in One SEO plugin and the developer included sitemap generator to add powerful usage for users. Once you check the box at SEO -> XML Sitemaps setting, it automatically creates XML sitemaps and notifies Google and other search engines of the sitemaps updates.
#3 XML Sitemap
XML Sitemap (by Denis de Bernardy, Mike Koepke) is your way if you are looking for very simple xml sitemap generator without headache of many options, just activate it and leave the job for it. The pings occur automatically on an hourly basis, if the sitemap file is updated.
#4 Better WordPress Google XML Sitemaps
Better WordPress Google XML Sitemaps (by Khang Minh) is The first WordPress XML Sitemap plugin that comes with comprehensive support for Sitemapindex, Multi-site and Google News sitemap. With this plugin you will no longer have to worry about the 50000 URL limit or the time it takes for a sitemap to be generated. This plugin is fast and consumes much less resource.
#5 XML Sitemap & Google News feeds
XML Sitemap & Google News feeds (by RavanH) is dynamically creates feeds that comply with the XML Sitemap and the Google News Sitemap protocol. Multisite and Polylang compatible and there are no files created.
What is XML Sitemap Priorities?
The priority of URLs is relative to other URLs on your site. Valid values range from 0.0 to 1.0. This value does not affect how your pages are compared to pages on other sites, it only lets the search engines know which pages you deem most important for the crawlers. So, you have to give the homepage the higher priority because it’s the most visited page on your site and other comes lower like posts, pages, etc.
Also, please note that assigning a high priority to all of the URLs on your site is not likely to help you. Since the priority is relative, it is only used to select between URLs on your site.
Hint: If you would like to generate your sitemap.xml with any free generator tool , i don’t recommended it because you have to re-generate and re-submit it every time you publish new content.
What is the Difference Between Sitemap.xml and Sitemap.XML.gz?
Sitemap.xml.gz is just a compressed version of your main XML sitemap using GZIP, it is very important and helpful for huge sites that have more than 50000 pages because their main sitemap.xml file will be large. Most search engines should accept GZip compressed XML sitemaps with no problems.
Hint: You don’t have to worry about build Sitemap.xml.gz because most of all WordPress sitemap XML generator plugins has this option without any additional help from you.
How to Submit XML Sitemap to Search Engines?
All above plugin has an option “Notify Search Engine about updates of your Blog“, so no worries about this point. Regarding to Google, you have to submit your sitemap.xml file only once through Webmaster Tools page
- Open Google Webmaster Tool Home page, choose the site need to submit sitemap for it.
- Expand the Crawl heading on the left dashboard, and select “Sitemaps” tool.
- Click on “ADD/TEST SITEMAP” Red Button.
- A small popup window appear, then type your sitemap file path
- Click on “Submit Sitemap” Blue Button.
Hint: You don’t have to re-submit your sitemap.xml file each time you publish new content, your plugin will do it automatically.
Common XML Sitemap Errors
Sometimes you may see errors in the Sitemap page in Google Webmaster Tool, that’s maybe because of server misconfiguration, server connections issues, plugins conflict, etc. Yoast Knowledge Base already listed most of Common Errors with XML Sitemap and you can find the solution of your issue in this great articles.
Sitemap XML Plugins Best Practices
- Offer HTML sitemap for users as well as offer XML sitemap for Google.
- Use light and well coded plugin to prevent any high consume resources from your site.
- A great and fast way to notice your sitemap.xml by search engines is linked from your robots.txt file.
- Keep your eyes open for any errors in Google Webmaster Tools in sitemap page to fix it as soon as possible.