Search Engine crawlers may well take a look at a variety of various factors when crawling a web site. Not just about every web page is indexed by search engines like google and yahoo. The distance of web pages within the root Listing of the website may be a factor http://community.sellfree.co.kr/blog/hotboss/search/