I think the spider started not deep reason the first possibility is not set on the navigation links, you can not enter the depths of the spider web site through the navigation links, it how to crawl the content page? Second is the spider encountered dead links, search engine spiders to crawl under a page but not crawling, dead link website has become arch-criminal, forced to leave the site spider love Shanghai. The third possibility is that long time not update the content of the site will lead to love Shanghai spider bored.
, the first to set the site navigation links
1, the navigation links to the nearest principle, we do not set in the navigation links to search engine spiders too deep, but in fact he is a gripping tool, what is the most easy to crawl, is the nearest thing. So when we use the navigation links through link layer navigation links into the column, and the next layer column links only import the content page, this is a layer of import method.
spider came to the site is a good thing, but the spider visit record I discovered a problem, often came to the site after the spider grabbed a 4,5 page on the left, this is how it goes? Believe that many webmaster feel from the events of June, love of spiders in Shanghai is very unstable, I the site is the same, from June after the spider every time to capture a few pages left, the website has not included this up, I finally carried out a major inspection on the site, and then on the website of the link is carried out, also say, after finishing the site, the spider to grab the page every time at 20, 30, here I share my check method and improvement methods.
2, the imported URL is not too complicated, I think you can set up a simple website address URL, with a PHP program, set up a simple directory, so when the spider crawling in the crawl is also relatively easy.
die block links on the web site is very large, little attention dead link is likely to have a fatal impact on the site, check the dead links can use the tools in Chinaz, and clean up the dead link of some relatively difficult. Now I’m cleaning method of Web site links to death is through the browser FTP search function, the first link in the query tool, die address copy, then copy through the browser FTP search function, find it in which file, finally deleted, many would say I > this long
site navigation links is the user’s guide line, is the search engine spider guides navigation route good can help search engine spiders from the front page of the site gradually throughout the site that requires the entire page, when we set up a web site navigation to set up a layer, below I talk about a few requirements set navigation links:
second, dead links must be cleared, leaving the scourge is