--------------------------------------------------------
IN THIS ISSUE:
--------------------------------------------------------
==> Sponsor - Arelis
4 Link Builder
==> Feature Article:
Googlebot, Wherefore Art Thou Googlebot?
Abandoned by Google
==> Subscribe/Unsubscribe information
---------------------------------------------------------
By Mike Banks Valentine
© August 30, 2004
As a search engine optimization specialist I often optimize
existing web pages for small business clients, upload them to
the site and see pages re-indexed by Google within a week. This
only happens with existing business sites that have been online
for a few years. Google seems to be updating their index as often
as every other week at this point and older established sites
that are already indexed seem to be re- crawled on that twice
a month schedule on a fairly routine basis.
Two clients that hired me for recent work saw their rankings
shoot to the top for a newly targeted search phrase in a weekend
when I did optimization on a Thursday and they were ranked instantly
by Saturday. Now keep in mind that this doesn't happen for everyone,
only those that have been online for some period and already have
significant content that simply needs tweaking and proper title
and metatag information added. They usually have relatively good
existing PageRank and do well for other RELEVANT search phrases
already. I offer that warning only to avoid instilling false hopes
in anyone hoping to achieve the same instant ranking boost overnight.
Those clients that do succeed in this way are often thrilled
with the results accomplished in such short order. I'd love to
be able to offer that type of ranking boosts to everyone, but
some are more equal than others when it comes to easy, inexpensive
SEO tune-ups that rev up your rankings overnight. Your mileage
may vary.
WHY DO NEW SITES SUFFER?
What is going on with newer sites that don't get crawled for
months? I've got a client, a newer attorney directory that offers
tons of great information in the form of articles on specific
areas of law, links to incredibly valuable and relevant legal
sites and over 600,000 attorneys listed by practice area and state.
Yet the site has not been re-crawled by Google for over 3 months!
Now this would not be such a big issue for many sites, but this
site is relatively new and we've optimized all the titles, tags
& page text, created a complete site map and placed links to all
these resources on the front page.
I know that the site is not being crawled because Google's cached
copy of the front page shows it before we did the work four months
ago, without the new links and without title tags. We've submitted
the site by hand, (manually) once a month for three months via
the Google Add URL page. http://www.google.com/addurl.html
When the hand submission failed to get it re-indexed for four
months, we submitted the sitemap page, which has not been crawled
at all. Google shows only ONE page on this site, when in fact
it has thousands of pages, a sitemap and dozens static pages!
--------------------------SPONSOR AD-------------------------------
Get more customers, more sales and build
a powerful business contact network with ARELIS 4 ARELIS is a
top rated software program that helps you to build a powerful
business network quickly and easily. Your business benefits in
many ways from that network:
you'll get highly targeted visitors to your web site
you'll get more customers and more sales
you'll benefit from new business contacts
your web site will get higher ranking on search engines
your web site will get higher link popularity
Increase the link popularity of your site and get highly targeted
free traffic to your web site http://shorl.com/gatredimadrotu
Sign up for the Axandra Search Engine Facts Newsletter http://shorl.com/gydriryluhibu
--------------------------SPONSOR AD-------------------------------
Part of the problem is that this site must be dynamic, since
a database of over 632,000 attorneys must be accessed, retrieved
and served for any of those law firms searched for to be returned
to the site visitor. Google warns owners of dynamic sites that
Googlebot may not crawl dynamically generated pages with "?" question
marks in the URL. This is to avoid crashing the server with too
many concurrent page requests from Google's spider. http://www.google.com/webmasters/2.html#A1
The solution to this dynamic URL problem has been discussed
widely in search engine forums and solutions have been bandied
about including software provided by SEO's, URL re-write techniques
for dynamic pages on APACHE servers http://www.alistapart.com/articles/urls/
and PHP pages http://www.stargeek.com/php-seo.php
to generate search engine friendly URL's. Others recommend simply
adding static HTML sitemap pages as alternatives for the search
engine spiders.
In this instance the client's developer simply said "I can't
do that (PHP solution) on this server". So we resorted to putting
up the static HTML sitemap pages with hard-coded URLS to the main
54 pages of the site at http://lawfirm411.com/Law-Firm-411-sitemap.html
This should get at least those fifty pages crawled by Googlebot,
but Googles' spider appears not to be crawling this site at all.
How do we know this? See for yourself by using the following
query in the search box at Google: allinurl:www.lawfirm411.com
where the result page shows ONE page in the results. If you try
that query on your own site (replace your own domain name for
lawfirm411.com), you'll see the results lists ALL your pages.
The site home page was crawled by Google four months ago, when
they took their "Cached Snapshot" of the page. You can see this
by visiting Google cached page here
where the date of this snapshot is "Apr 20, 2004 07:42:19 GMT"
and they haven't been back since. The page in that snapshot has
none of the newly added links, an outdated title tag, and old
content.
This problem is not unique to this site. One client we worked
with two years ago had a dynamically generated, framed site! Those
two site structures have always given search engines trouble.
Their site was not crawled at all and only the front page showed
up. Our solution was to create a second domain (owned by the client),
which had static HTML pages that precisely mirrored the content
of the client's framed, dynamically generated site. Guess what
happened after Googlebot crawled the static site? Google indexed
the framed site in full and then banned the static site from the
index!
Not an approach we advocate, but the one that worked for this
client.
We're still searching for ways to get Googlebot back to LawFirm411.com
before creating that new static site, but decided to share this
odd experience with the SEO community before going to any extremes.
Google provides over 70% of most search engine referred traffic
to ALL of our clients and we realized we can't site idly by and
see a major client languish because Googlebot didn't like what
it found at the client site on the first visit four months ago.
This issue dogs newer sites in other places as well. The Open
Directory Project has also become notoriously slow in adding new
sites to the directory and in this case, has not picked up this
site even after 6 regular monthly submissions. The web playing
field may have begun tilting toward older, established sites and
away from new ones.
-------------------------------------------------
Mike Banks Valentine is the SEO for http://www.lawfirm411.com
Contact him at http://www.seoptimism.com/SEO_Contact.htm
Improve Your Small Business Online at our Ecommerce Tutorial http://website101.com/Free-Tutorials/index.html
-------------------------------------------------
Copyright © 2004 Mike Valentine
This article may be reproduced if used in it's entirety and the
resource box directly above is included, with URL's set as hyperlinks
and copyright notice included in all uses. Not to be Sold. See
complete usage information at http://website101.com/arch/
-------------------------------------------------------
