In this article we would discuss why site maps are important and how to create site maps for your website. We will discuss both automated and manual methods for creating a sitemap.

Before going in to the details let us see what site maps are. A site map is a hierarchical visual model of the pages of a Web site. Site maps help users navigate through a Web site that has more than one page by showing the user a diagram of the entire site’s contents.

Similar to a book’s table of contents, the site map makes it easier for a user to find information on a site without having to navigate through the site’s many pages. Also, in SEO, a site map can make it easier for a search engine spider to find all a site’s pages. A site map (or sitemap) is a list of pages of a web site accessible to crawlers or users.

It can be either a document in any form used as a planning tool for web design, or a web page that lists the pages on a web site, typically organized in hierarchical fashion. This helps visitors and search engine bots find pages on the site.

While some developers argue that site index is a more appropriately used term to relay page function, web visitors are used to seeing each term and generally associate both as one and the same. However, a site index is often used to mean an A-Z index that provides access to particular content, while a site map provides a general top-down view of the overall site contents. These site maps are often used to show an index for users that belong to the YLC.

Sitemaps are an easy way for webmasters to inform search engines about pages on their sites that are available for crawling. In its simplest form, a Sitemap is an XML file that lists URLs for a site along with additional metadata about each URL (when it was last updated, how often it usually changes, and how important it is, relative to other URLs in the site) so that search engines can more intelligently crawl the site.

Web crawlers usually discover pages from links within the site and from other sites. Sitemaps supplement this data to allow crawlers that support Sitemaps to pick up all URLs in the Sitemap and learn about those URLs using the associated metadata. Using the Sitemap protocol does not guarantee that web pages are included in search engines, but provides hints for web crawlers to do a better job of crawling your site.

If your website is having indexation problems, has a poor internal link structure, or falls in one of the three situations described by Google, then using a Sitemap would probably be a good idea.

There are two primary kinds of site maps, the HTML ones and the XML ones.  An HTML sitemap is a page within your site that links to all the other important pages on your site. You can see an example of this on my Archives page. I call it “Archives”, but it is nothing more than an HTML sitemap.

An XML sitemap does pretty much the same thing, but instead of using HTML (which a browser can interpret like any other page) it uses XML, which is a markup language to encode documents. This means that the XML sitemap will only be useful/make sense to search bots, while the HTML sitemap will also be visible and useful to human visitors.

XML is a coding language used, (amongst many other things) as the standard for web crawlers to find and parse sitemaps. There is an example of an XML sitemap below (missing link to site). The instructions to the sitemap are given to the crawler bot by a Robots Text file, an example of this is also given below. Site maps can improve search engine optimization of a site by making sure that all the pages can be found.

This is especially important if a site uses a dynamic access to content such as Adobe Flash or JavaScript menus that do not include HTML links.

They also act as a navigation aid by providing an overview of a site’s content at a single glance.

Google introduced Google Sitemaps so web developers can publish lists of links from across their sites. The basic premise is that some sites have a large number of dynamic pages that are only available through the use of forms and user entries.

The Sitemap files can then be used to indicate to a web crawler how such pages can be found. Bing, Google, Yahoo and Ask now jointly support the Sitemaps protocol.

Since Bing, Yahoo, Ask, and Google use the same protocol, having a Sitemap lets the four biggest search engines have the updated page information. Sitemaps do not guarantee all links will be crawled, and being crawled does not guarantee indexing. However, a Sitemap is still the best insurance for getting a search engine to learn about your entire site.

XML Sitemaps have replaced the older method of “submitting to search engines” by filling out a form on the search engine’s submission page. Now web developers submit a Sitemap directly, or wait for search engines to find it.

XML (extensible Markup Language) is much more precise than HTML coding. Errors are not tolerated, and so syntax must be exact. There are automated XML site map generators available (both as software and web applications) for more complex sites.

The Sitemaps protocol allows a webmaster to inform search engines about URLs on a website that are available for crawling. A Sitemap is an XML file that lists the URLs for a site. It allows webmasters to include additional information about each URL: when it was last updated, how often it changes, and how important it is in relation to other URLs in the site. This allows search engines to crawl the site more intelligently. Sitemaps are a URL inclusion protocol and complement robots.txt, a URL exclusion protocol.

Sitemaps are particularly beneficial on websites where:

Some areas of the website are not available through the browsable interface, or Webmasters use rich Ajax, Silverlight, or Flash content that is not normally processed by search engines.

The webmaster can generate a Sitemap containing all accessible URLs on the site and submit it to search engines. Since Google, Bing, Yahoo, and Ask use the same protocol now, having a Sitemap would let the biggest search engines have the updated pages information.

Sitemaps supplement and do not replace the existing crawl-based mechanisms that search engines already use to discover URLs. Using this protocol does not guarantee that web pages will be included in search indexes, nor does it influence the way that pages are ranked in search results

Why we need Sitemaps?

If a website has hundreds of pages, it can get rather chaotic in a navigation bar. Apart from this, both human browsers and search engine spiders need an easy access to find all the pages within a site. Therefore a sitemap is very essential. It gives them a simple navigational tool. Lack of a sitemap may be the reason why your site hasn’t been completely indexed.

If you are using many pages, some search engines have a difficulty in indexing them. So a site map makes it easier for a search engine spider to find and index every page on your site. It will surely improve the search engine optimization of your site.

Using sitemaps has many benefits, not only easier navigation and better visibility by search engines. Sitemaps offer the opportunity to inform search engines immediately about any changes on your site. Of course, you cannot expect that search engines will rush right away to index your changed pages but certainly the changes will be indexed faster, compared to when you don’t have a sitemap.

Also, when you have a sitemap and submit it to the search engines, you rely less on external links that will bring search engines to your site. Sitemaps can even help with messy internal links – for instance if you by accident have broken internal links or orphaned pages that cannot be reached in other way though there is no doubt that it is much better to fix your errors than rely on a sitemap.

If your site is new, or if you have a significant number of new or recently updated pages, then using a sitemap can be vital to your success. Although you can still go without a sitemap, it is likely that soon sitemaps will become the standard way of submitting a site to search engines.

Though it is certain that spiders will continue to index the Web and sitemaps will not make the standard crawling procedures obsolete, it is logical to say that the importance of sitemaps will continue to increase. Sitemaps also help in classifying your site content, though search engines are by no means obliged to classify a page as belonging to a particular category or as matching a particular keyword only because you have told them so.

Having in mind that the sitemap programs of major search engines and especially Google are still in beta, using a sitemap might not generate huge advantages right away but as search engines improve their sitemap indexing algorithms, it is expected that more and more sites will be indexed fast via sitemaps.

An important point to remember always is that people come to your site looking for information. Net surfers are a ruthless lot. If they find information quickly, they stick around; else, they click the “Back” button on the browser, close the browser window or simply go to another web page.

So the most important reason a site map is indispensable is because it helps the visitors to understand the site structure and layout and thus, quickly gain access to what your site has to offer. A site map is a page wherein the structure of a web site is clearly shown to a visitor. They can then choose the page to which they want to go with just one click. In addition to the advantages a site map offers to “human” visitors, it is also important for “robotic” visitors from search engines.

As a web developer your aim should be to get all public pages on your web site in the search engine database. Since a site map carries links to all pages from one page, it is an ideal candidate to submit to a search engines. Submission of the site map to a search engine might help in getting all web pages indexed quickly by the search engine.

For example, if you have a page buried deep inside the directory structure on your web site, it might not be picked up by such search engines. In such cases, the importance of a site map is that it pushes the search engine to go directly to the page instead of hunting for links. Site maps are also important in the design phase of a website. A site map helps you in planning your site before you even start creating it.

Once you decide which pages you want, your job is actually made a lot simpler when you start designing the web pages. A site structure helps you in understanding the number of pages on the site and how they would be laid out.

How to create Site maps?

The Sitemap Protocol format consists of XML tags. The file itself must be UTF-8 encoded. Sitemaps can also be just a plain text list of URLs. They can also be compressed in .gz format. Once you have your website ready with your XML site map, you need to upload your website to your server and then notify Google, through Webmaster Tools, where your sitemap lives . Typically, there are two ways of generating a site map:

1. You can install a sitemap generator script on your website.

2. You can use an on-line sitemap generation tool to create one.

Though the first option is difficult it offers you more control over the final result. Google Sitemap Generator is a good tool that can be used for generating a site map. Since the tool is coded in Python script, you need to have Python 2.2 version on your web server to run it.

The second method of generating a site map while being easier doesn’t give you much flexibility. Google also suggests some 3rd party Sitemap tools but warns users that it has not tested or verified them. Having created a site map, it’s time to upload it to your website and let Google, Yahoo and MSN know about it.

Each time you run the Create Site Map command, it creates a totally new site map file. Thus, if you customize the site map file after running the command, you will need to customize it again after rerunning the command, unless you save a copy of the file to another location.

The Create Site Map command works by retrieving the titles of your HTML files from their <TITLE> tags. If you have an HTML file without a <TITLE> tag, the command will use the file name. To build a good site map, you should use unique descriptive strings in the <TITLE> tags of your files. Category names in the site map file come from the Web project name, which should also be descriptive.

Once a site map file is created, you can add it to your Web project using the Add Files command. You must check the file out of the database before running the Create Site Map command again.

It is possible to customize the site map by adjusting the HTML in the site map file. You can use a different background color or pattern, change indentation levels, delete or add items, and so on. One way you can customize the site map is to change the graphics files used for the folder and file icon.

To create a site map:

Open a Web project in Visual SourceSafe Explorer.

On the Web menu, click Create Site Map.

In the Create Site Map for <name> dialog box, use this box to specify the file to contain the site map. Alternatively, you can click Browse to locate a file to use.

Select the Check in to Visual SourceSafe check box.

Click OK. The specified file is created and checked in to the database.

If desired, you can navigate to the file and customize it.

Sitemap Generators

Sitemap generators are used to create sitemaps. There are many sitemap generators available, both free and paid, offline and online use, and automatic or manual. Using a site map generator makes creating a sitemap easier than manually created ones.

Online generators

These tools tend to be much simpler to use than the offline generators and should be ok for most websites. All you need to do is supply the url of your own page and your site will be scanned and a map produced.

You tend however to have very little choice on the look and feel, but you can always manually edit the map afterwards.

Offline generators

The first place to check is your web editing software. If your editing software doesn’t support sitemaps then you can purchase third party solutions or use free tools. Commercial sitemap generation software gives you far more control over the look and feel of the sitemap, but if your primary intention is for the search engines then that isn’t so important.

In addition most commercial web page editing tools like Dreamweaver and FrontPage will create a sitemap for you.

Google Sitemaps

Google have recently introduced Google Sitemaps which should not be confused with standard sitemaps. The Google sitemaps are intended to be submitted to Google to help the Google spider index your site:

1. Sign in to Google webmaster tools with your Google account 2
2. Add your site to your webmaster tools account.
3. Verify your site, either by creating a blank html file with a certain name which would be generated by Google, or by inserting a specific Meta tag in your site index.
4. Go to http://www.auditmypc.com/web-master-tools.asp (text based). You can also now create video sitemaps.
5. Click on the Webmaster Tool button.
6. Enter you site name and url.
7. Hit the play button in the bottom-left corner.
8. Click on the Sitemap tab.
9. Wait until all the files have finished loading, then select the one which location is the root of your website.
10.  Click on the Save Sitemap button, then on the pop-up menu that comes up, click on Google Sitemap.
11.  Save the file to you sites directory or just anywhere if you don’t have one.
12.  Upload the file to your web server at the highest publicly available directory normally “www” or “public_html”.
13.  Go back to Google’s webmaster tools, and click on the Sitemaps tab.
14.  Click “Add a Sitemap”.
15.  Select General Web Sitemap on the drop down list.
16.  Enter the URL of you sitemap. Eg. http://www.mysite.com/sitemap.xml
17.  Click on Add Web Sitemap.

Important points for a good site map

Here are some hallmarks of a good site map, which helps visitors at finding information quickly on your web site:

- The site map should be the simplest page on your web site.

- Do not give a fancy name to the site map link such as “Web Site Tree” – keep it as “Site map”, this way your visitors understand immediately what you mean.

- You should always avoid “dynamic” site maps. Those in which the visitors have to “work” their way to get hold of information. Remember, the reason visitors comes to a site map page is because they are lost. To make them work again for something that you can display as a simple static link will just kill the purpose of having a site map.

- If the site map is list of text links, be sure to use the TITLE attribute of the anchor tag and include keywords inside it.

- It is a good idea to put a sentence describing the page contents below the link for that page on a site map.

- A site map should not be the primary navigation on your web site it should complement it.

- A link to the site map page is very important and all pages should carry this link. The site map link can be included with other links in the main menu on your web site or placed at a section on the web page from which is it clearly visible.

- Other important aspects on a web site should complement site maps. For example, the link color for visited links should be different from that of non-visited links so that visitors understand which pages they have already seen and thus, save time.

xibl.com