The Importance of URL Structure

I have a rant that I have wanted to get off my chest for a while now,  it's like an itch that has been just out of reach, one that I have not been able to scratch until now...

One of the most often overlooked SEO issues that can have an explicit impact on your organic search engine rankings is URL structure. This is one of the most commonly overlooked aspects of SEO.   I have witnessed countless times the mess left by poorly optimised and down right messy URLs. It is a crime and if asked I am willing to point those accused out in police line-ups.  Not paying attention to your websites URL structure can make it difficult for the search engine spiders to index your site, let alone rank well! Too often we see website developers not planning carefully their site's architecture and finally ending-up looking for corrective measures after launch. 

It's a bit like constructing a huge tower on poor bases...



Here're some issues we consider important when constructing the URL structure of a website.

Few folders - When creating a directory structure, we have to be careful not to create too many sub-categories under too many categories.  At the most, we should keep the depth of the directory structure to two levels.  Most search engines don't go beyond one or two levels of directory structure, therefore keeping it to 2 levels is good from a SEO perspective but it is also good for the user experience.  This is because shorter URLs are better than longer ones for usability (easier to remember)

Descriptive keywords in URLs - Avoid using numbers (usually from dynamically generated URLs) as opposed to using descriptive and targeted keywords in the URL.  The URL structure should be as straightforward as possible and constructed logically as well as being comprehensible to users.  That is using clear words rather than long ID numbers.
 
Keyword rich URLs - It is also important to have keyword rich URLs. Highly relevant keywords should appear in the domain name or the page URL. This became clear in a recent study on how the top three search engines, Google, Yahoo, and MSN, rank websites. The statistics showed that top ten ranking websites on the major search engines have keywords either in the URL or in the domain.
 
At the same time we can use words in the URL to promote the theme or content of a website, as far as possible we must try to ensure that our URLs accurately reflect the page content.  According to Matt Cutts  (Head of Google's Webspam team) we should keep from three to five words in our URL, having more than that can look a little 'abnormal'.

Static URLs vs Dynamic URLs - A dynamic URL is a page address that results from the search of a database-driven web site.  The dynamic page is basically only a template in which to display the results of the database query using a script.  Whereas static URLs are page addresses, in which the contents of the web page stay the same unless the changes are hard-coded into the HTML.
 
Now, static URLs typically rank better in search engine results pages than dynamic pages and they are indexed more quickly.  This is because a search engine normally wants to only list pages that are unique. Some Search engines decide to combat this issue by cutting off the URLs after a specific number of variable strings (e.g.: ? & =).
 
Example: The 3 pages may look all the same for a Search Engine


This can happen if the search engine purges the information after the first offending character, the question mark (?), now all three pages can look the same:
 
Whenever possible, we must try to shorten URLs by trimming unnecessary parameters. To be brief, static URLs tend to be shorter and more successful than dynamic URLs.  Search engines have been pretty straightforward about that, they have a preference for static URLs...

Use of robot.txt - If we think we can have problematic URLs in our website then we can consider using a robots.txt file to block crawlers access to those particular URLs. Usually we prevent crawlers to have access to dynamic URLs, such as those that generate search results.  We can also use the robot.txt file very effectively by using 'regular expressions' thus allowing us to easily block large numbers of URLs.

Good Navigation - We have to make sure that our visitors can easily jump from one category to another, without having to click the 'Back' buttons multiple times. This is why it is important that we list all the top category links significantly on every page.  We can also use 'Breadcrumbs' to help in navigation, using breadcrumbs are good as they are navigational techniques displaying all visited pages leading from the homepage of a website to the currently viewed page.
 
Conclusion
Even if a website might be having great content, information must be logically organised for both search engines and visitors to access it.  It's somewhat pointless to put effort in optimising a Web page without giving search engines simple access to that page's content.  Badly constructed navigation schemes and URL structures can constantly act as a stop sign to search engine indexing.