ically generated links except in directories that are not intended to be crawled/indexed (e.g., those should have a /robots.txt exclusion)."
Google's Webmaster Guidelines:
"If you decide to use dynamic pages (i.e. the URL contains a "?" character), be aware that not every search engine spider crawls
dynamic pages as well as static pages. It helps to keep the parameters short and the number of them small."
Let's analyze what Google has stated above.
1. the URL contains a "?" character: this means the definition of dynamic pages are those containing "?" characters in URL.
2. keep the parameters short: this means the number of characters in each individual parameter should be short. There is no quantitative measurement given by Google but we can check some web forums to see examples. My Search engine friendly article (
webactionguide/action-guide/build-site/se-friendly.php) referenced black hat seo discussion thread on Cre8ASiteForums. Its URL is
cre8asiteforums.com/viewtopic.php?t=8386 This page was crawled by Google. The length of its query parameter is 4 characters. There are many other examples on the internet that have more characters and were crawled successfully. The maximum number of characters that can be accepted by Google is unknown.
3. keep the number of them small: this means we should keep the number of parameters in each URL as small as possible. The above Cre8ASiteForums example has one parameter.
At least now we can say Googlebot is able to crawl dynamic pages that have one query parameter and the number of characters in the parameter can be 4.
How to get your pages crawled if using query parameters are not avoidable?
Query parameters are often used for database calls to retrieve stored information by using primary keys in one or more tables. Database Management System (DBMS) makes some tedious work easy to manage. When query parameters must be used for your site, consider build a site map page and hard code a page's URL. For example, the previous URL can be hard coded as
examplesite.com/product12345-23-3.asp Hand code every dynamic page is time-consuming. If you use Apache web server, there is a Apache mod_rewrite module to help you (
httpd.apache.org/docs/mod/mod_rewrite.html) rewrite the requested URL to one with no "?" character embedded on the fly.
Another mod rewrite resource site is
modrewrite.com.
An interesting article on
weberblog.com weberblog.com talked about a practical example of how Google successfully indexed a dynamic page after applying mod_rewrite module. The page originally had 17 characters in the query parameter.
Before rewrite:
weberblog.com/article.php?sroty=20040419170030157 After rewrite:
weberblog.com/article.php/20040419170030157 So, if your site is experiencing the same problem, hurry up and implement mod_rewrite now.
About The Author
Jerry Yu is an experienced internet marketer and web developer. Visit his site
WebActionGuide.com for FREE "how-to" step-by-step action guide, tips, knowledge base articles, and more.
