ASP.NET Search Sitemaps – something for SEO

ASP.NET Futures introduced support for search engine sitemaps. This is content detection protocol introduced originally by Google and later accepted also by Microsoft and Yahoo! ASP.NET makes sitemaps generating easy for us and as a bonus their model is perfect for more complex web applications like e-commerce and e-services sites. Let’s see now what we can do.

Couple of word about search sitemaps

If you have small site with low number of links then managing sitemaps manually is not a problem. But if your site has many links and content that changes often you may want to automate the task of generating sitemap.

Sitemap is important thing for each site because it tells to search engine spiders where to find information from your site. If you give this information there is something you will get back – Google gives you very rich set of information about your site through their Webmasters service.

ASP.NET 2.0 introduced us the new concept – sitemaps. Besides new site navigation definition format we got also providers that were used by navigation controls. ASP.NET search sitemaps technology extends this model and makes our site navigation understandable for search engine spiders.

ASP.NET Search Sitemaps

Let’s take a closer look at search sitemaps now. Suppose we have a web application that uses search sitemaps. As a first thing we will look at sitemap file that defines static navigation of our application.

<?xml version="1.0" encoding="utf-8" ?>
<sitemap xmlns="http://schemas.microsoft.com/AspNet/SiteMap-File-1.0">
  <sitemapnode url="/" title="Home"  description="Home page">
    <sitemapnode url="about-us.aspx" title="About us" description="">
      <sitemapnode url="corporate.aspx" title="Corporate information"  description="" />
      <sitemapnode url="values.aspx" title="Corporate values" description="" />
    </sitemapnode>
  </sitemapnode>
</sitemap>

Now let’s see section from application’s web.config where dynamic part of site’s navigation is defined.

<searchSiteMap enabled="true">
  <providers>
    <add name="Navigation"type="Microsoft.Web.Preview.Search.AspNetSiteMapSearchSiteMapProvider, Microsoft.Web.Preview"/>
    <add name="Product"
           type="ASPNETFuturesEnabledWebApplication1.MySitemapProvider, ASPNETFuturesEnabledWebApplication1"
            targetUrl="Products.aspx"
            targetUrlseparator="?"
            pathInfoFormat="false"
            queryStringDataFields="ProductId"
            queryStringDataFormatString="ProductId={0}"
         />
  </providers>
</searchSiteMap>

As we can see there are two providers given that provide the sitemap system with dynamic data about navigation. First provider is the default one that generates search sitemap based on site’s static navigation.

The second provider was added by me. This provider returns information about my products section navigation. When asked, it will return all links that products section contains. The code is as follows.

public class ProductEntry
{
    public string ProductId;
    public string ProductName;
    public String SiteMapLastModified;
    public String SiteMapChangeFrequency;
    public String SiteMapPriority;
}
 
public class ProductSitemapProvider : DynamicDataSearchSiteMapProvider
{
    public override IEnumerable DataQuery()
    {
        List<ProductEntry> products = new List<ProductEntry>();
        ProductEntry entry;
 
        foreach (Product product in DAL.GetProducts())
        {
            entry = new ProductEntry();
            entry.ProductId = product.Id.ToString();
            entry.ProductName = product.Name;
            entry.SiteMapLastModified = product.Modified.ToString("yyyy-MM-ddThh:mm:ss.fffZ");
            products.Add(entry);
        }
        return products;
    }
}

The class called ProductSitemapEntry defines search sitemap entry that is used with products. Second class, ProductSitemapProvider, is the one used by search sitemaps engine to get information about products. You can handle ProductSitemapEntry as one of the DTO classes for Products.

Search sitemaps structure

Search sitemaps protocol allows also sitemaps hierarchy to be defined. It is specially useful for large sites because we can say on higher level sitemaps which subsitemaps are changed meanwhile. This way we can lower our spidering traffic also.

Search sitemaps HttpHandler that we can find from web.config generates hierarchical search sitemaps. Let’s see what happens when we make request to this handler.

<?xml version="1.0" encoding="utf-8"?>
<sitemapindex xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
  <sitemap>
    <loc>http://localhost:49781/SearchSiteMaps.axd?sitemap=Navigation</loc>
    <lastmod>2008-04-20T01:58:41.694Z</lastmod>
  </sitemap>
  <sitemap>
    <loc>http://localhost:49781/SearchSiteMaps.axd?sitemap=Product</loc>
    <lastmod>2008-04-20T01:58:41.694Z</lastmod>
  </sitemap>
</sitemapindex>

This is pure hierarchical search sitemap that refers to two subsitemaps. These subsitemaps are asked through same handler and name of search sitemap is given by query parameter. When you look at search sitemaps configuration section above you may notice that these query parameters have same values as search sitemaps providers names.

Let’s see now what happens when we ask first sitemap. Remember that the first one corresponds to site’s static navigation shown before. Just copy and paste URL from above XML to your browser.

<?xml version="1.0" encoding="utf-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
  <url>
    <loc>http://localhost:49781/</loc>
  </url>
  <url>
    <loc>http://localhost:49781/about-us.aspx</loc>
  </url>
  <url>
    <loc>http://localhost:49781/corporate.aspx</loc>
  </url>
  <url>
    <loc>http://localhost:49781/values.aspx</loc>
  </url>
</urlset>

Now let’s see what is returned by products search sitemap provider.

<?xml version="1.0" encoding="utf-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
  <url>
    <loc>http://localhost:49781/Products.aspx?ProductId=100</loc>
    <lastmod>2008-04-20T02:02:02.544Z</lastmod>
    <priority>0.4</priority>
  </url>
  <url>
    <loc>http://localhost:49781/Products.aspx?ProductId=102</loc>
    <lastmod>2008-04-20T02:02:02.544Z</lastmod>
    <priority>0.4</priority>
  </url>
</urlset>

As we can see there are two products (oh, lazy me!) introduced to search engines spiders.

Conclusion

As we can see the search sitemaps technology is very powerful feature that makes our sites SEO much simpler than it was before. We can add as many search sitemaps providers to our application’s configuration and let them generate search sitemaps. Besides static navigation we may have many different sections in our site. By example, we may have blog, product vatalog, support forum, FAQ pages and so on.

ASP.NET search sitemaps will provide us with common and flexible ways to make our applications support search sitemaps protocol.