AnalyticsMy Decade Of Writing About Search Engines

My Decade Of Writing About Search Engines

Ten years ago today, I first starting writing publicly about search engines. If we had blogs back then, I suppose I would have been a search blogger. But we didn’t. We hand-coded our HTML, walked through the snow for eight miles to FTP files to our web servers, and we liked it 🙂 My involvement […]

Author
Date published
April 17, 2006 Categories

Ten years ago today, I first starting writing publicly about search engines.
If we had blogs back then, I suppose I would have been a search blogger. But we
didn’t. We hand-coded our HTML, walked through the snow for eight miles to FTP
files to our web servers, and we liked it 🙂

My involvement with search engines goes back to my first year as a student
at the University Of California, Irvine in 1983. No, I wasn’t part of the
university’s highly regarded information and computer science department.
Instead, I was an English major — and a pretty bored one for my first two
months, when I had to commute until getting on-campus housing.

I spent some time exploring the library, having been a big library user since
I was a child. The library had a magical electronic card catalog called
Melvyl (named
for Melvil Dewey, who created the Dewey Decimal System). For fun, I’d do Melvyl
searches for broad topics such as history, art or love, to see how many matches
would come back. I could routinely crash the search routine by doing this.

The system would diligently try, telling me it would take 50, 60, 80 search
cycles, and then the countdown would begin. Some searches would eventually get
through all cycles and give me a matching results count. Often, the system would
just give up as the countdown approached the teens.

My 1996 Study

Search engines remained fascinating to me when I reencountered them in 1995.
I’d left working as a newspaper reporter to go into web development, since I
didn’t want to miss out what was obviously going to be the future of publishing.
As the general manager of Maximized Online, my job was to help get people in the
Orange County, California area online. We’d build web sites, get them publicized
to search engines and other publicity venues, plus host them.

One of our clients was upset at the end of 1995 that his OC jobs site wasn’t
ranking tops for a search on “orange county” in WebCrawler. We didn’t have a
good answer to give him. We’d done the submission, made use of the meta tags the
search engines said to use, but why exactly a site would rank well wasn’t well
known. So I decided to look into it.

I spent January through April 1996 making changes to the InfoPages directory
that my company maintained, a search engine just for Orange County web
resources, to see if it could rank better in a search for “orange county.” I
tried putting those words in the body text, the title tag, in the meta tags and
also checked to see if spamming helped, if repeating the word over and over
would have an impact.

I published the results online, and 10 years on, a lot of the advice remains
exactly the same. Don’t depend on ALT text. Don’t fixate on only one or two
terms, because there are many ways people will seek you — a long tail before we
had talk of long
tails. Build links, because links can send you traffic. And don’t fixate on
getting traffic just from search engines. The conclusions from that study are below for those
really interested; others can jump past for the rest of this article.

There don’t seem to be any magic methods that will make a page appear at
the top of every search engines’ listings. There’s too much fluctuation on the
web for any page to claim a foothold, and all the engines handle relevancy
slightly differently. However, there are some general tips that do help a page
appear more relevant.

  • Have text on your home page: Search engine catalogs contain the
    text read from the various home pages the engines visit. If a page lacks
    descriptive text, then there is little chance that page will come up in the
    results of a search engine query. It’s not enough for that text to be in
    graphics. It must be HTML text. Some search engines will catalog ALT text
    and text in comment and meta tags. To be safe, a straight HTML description
    is recommended.
  • Pick your keywords: Focus on the two or three keywords that you
    think are most crucial to your site, then ensure those words are both in
    your title and mentioned early on your web page. Generally, most people will
    already have those words present on their pages but may not also have them
    in page titles. Keep in mind that the keywords you consider crucial may not
    be exactly what users enter. Our study focused on making the InfoPages
    directory appear high on lists if keywords “Orange County” were entered. The
    lack of success with some search engines does not mean that the site isn’t
    being found. Many people find the site by entering more words, such as
    “Orange County California” or “Orange County Web.” The addition of just one
    extra word can suddenly make a site appear more relevant, and it can be
    impossible to anticipate what that word will be. The best bet is to focus on
    your chosen keywords but to also have a complete description.
  • Have links to inside pages: If there are no links to inside pages
    from the home page, it seems that some search engines will not fully catalog
    a site. Unfortunately, the most descriptive, relevant pages that are often
    inside pages rather than the home page. You can also try sending search
    engines directly to your lower levels, if they don’t ordinarily go there.
  • Forget Spamming: For one thing, spamming doesn’t seem to work
    with every search engine. Ethically, the content of most web pages ought to
    be enough for search engines to determine relevancy without webmasters
    having to resort to repeating keywords for no reason other than to try and
    “beat” other web pages. The stakes will simply keep rising, and users will
    also begin to hate sites that undertake these measures. Efforts would be
    better spent on networking and alternative forms of publicity described
    below.
  • Network: If your site fails to make the top ten lists, then get
    together with those that do. Perhaps some might be considered “competitors,”
    but others might be happy to link to your site in return for a link back.
    After all, your site may appear first when slightly different keywords are
    used. Links are what the web was built on, and they remain one of the best
    ways for people to find your site.
  • Relax: Search engines are a primary way people look for
    web sites, but they are not the only way. People also find sites
    through word-of-mouth, traditional advertising, the traditional media,
    newsgroup postings, web directories and links from other sites. Many times,
    these alternative forms are far more effective draws than are search
    engines. The audience you want may be visiting to a site that you can
    partner with, or reading a magazine that you’ve never informed of your site.
    Do the simple things to best make your site relevant to search engines, then
    concentrate on the other areas.

A Webmaster’s Guide To Search Engines

Along with the study, I also published a collection of documents called “A
Webmaster’s Guide To Search Engines.” My goal was to help site owners better
understand the essentials of being found plus identify which search engines really
mattered. Knowing who mattered was crucial when you’d have some search engines
like Galaxy forcing your through a three part, multiple question submission
process to be included in their directory. Was spending all that time worthwhile? (For
Galaxy, the answer was no!).

The guide provided links to the FAQs of each search engines, along with my own
observations about whether how each search engines said it worked actually lived
up to reality. There was a guide to which search engines I considered to be
“major” or most important to site owners and searchers alike. I had a
“Strategic Alliances & Victories” chart to show which search engines had deals
with the Netscape or Internet Explorer browsers and which had gained positive
reviews in magazines.

The information I published quickly generated a lot of positive feedback,
both from site owners and searchers such as librarians. At the same time, the
web development company I worked for closed, so that the parent firm could
concentrate on web software development. I hung out my internet consultant
shingle and kept maintaining the Webmaster’s Guide on a part time basis, sending
out a newsletter update (The Search Engine Report)
twice that
year, along with making further site updates.

In 1997, I moved to the UK from California, so my wife could be closer to her
family. I also began spending more and more time on the site, as well as writing
freelance articles on search for various publications. In the middle of the
year, I rebranded the site as Search Engine Watch, which generated more
attention. By the end of the year, Mecklermedia purchased the site from me, and
I continued on as editor of it.

The Search Revolution

Ten years on, I remain as fascinated with search engines as ever. I’ve been
fortunate to help chronicle the birth of an entirely new advertising medium.
Equally important has been the birth of an entirely new way for people to seek
out information.

I knew search engines were important when I decided to write about them. The
journalist in me could see they were a good story, especially when you realized
that under the hood, they weren’t doing things like crawling as often as people
widely believed. But a
study by
Keen in 2001 especially resonated with me. Search engines (as a whole — we
weren’t Google obsessed yet then) were the single most likely way people would
seek information.

The study was small, but the findings were still stunning. In only about five
years, search engines had ousted things like friends, family, books, magazines,
libraries and other perfectly good resources for seeking answers.

Some of this was bad. I’d personally watched people when doing search
training spending ages trying to find a phone number, when a call to telephone
information would have found much faster. Old but still useful search strategies
were abandoned in favor of the magic search box.

Lots of this is good. Search engines remain amazing tools that get us the
right answers quickly in many circumstances.

Looking Ahead

Will I still be doing this in 20 years? Almost certainly not, at least not in
the daily grind format I’ve been doing. I’d like to keep writing about
search issues, but eventually I’ll move away from the regular day-to-day
coverage to perhaps focus on less frequent but deeper looks at particular search issues.

I’m also thinking a lot about doing a book these days. I’d always wanted to do a book on search, indeed the exact type of
history that
John Battelle did a
fantastic job
with in The Search.

Since that’s come out, I’ve thought more and more about
doing a more personal retelling of web search history — the evolution, developments and trends
I’ve seen from having been in the trenches of covering them over the years.

I’d
also like to do a separate one talking to various search marketers, spotlighting
them and focusing on how that medium has evolved over the years and where it
will be going. The most fascinating book idea remains the impact of search on our
everyday lives, how people make use of them, how habits have changed, our
laws are starting to account for the power of search and many related issues
like that.

Someday! What I can say is that for the near future, I expect to remain
working on the site and coverage as I have, bringing some of our standing
content back up to date, which I know has been neglected due to the need to
cover the news that continues to flow in. My original Webmaster’s Guide helped
many understand search engines, and I very much want to
ensure Search Engine Watch remains as a leading resource doing that in the years
to come.

Looking Back

I don’t have a succinct list of big picture items or “high order bits” to offer.
A lot of this has already been covered in things I’ve
written, so instead I’m going to spend some time recapping pieces I think are
most important below. These are either big trend pieces I’ve done or big shifts
in the search landscape I think worth noting.

I know — I KNOW — I’ve left some things out. My apologies, if so. It’s a
bit easier for me to cover all the things I’ve written that what me or Chris
Sherman both have done, and he’s clearly covered tons himself. Plus, skimming
through 10 years worth of writings means I’ll accidentally miss stuff. If you
want to go poking yourself, I’ll give more tips after the summary.

The biggest overall theme in doing the recap is how that big old wheel keeps
spinning around and around, with people often buying hype because they don’t
remember things have come before — or marketers making errors because they
don’t understand issues that were explored already in the past.

I’ve definitely felt myself getting more and more jaded. Part of that’s bad,
because there are cool, new things that I don’t want to be blinded to. But then
again, you go through the list below and tell me if you don’t emerge feeling a
big jaded about some ideas and concepts that are retreads.

1997

1998

1999

2000

2001

2002

2003

2004

2005

2006 (To Date)

As I said earlier, I know I’ve missed stuff. Unfortunately, there’s no easy
way for me to see everything written on Search Engine Watch over the years in
one single list. For those who wish to explore, the
Search Engine
Report archives are probably the best thing to review. Each month, there’s
an issue of the Search Engine Report that recaps virtually everything of
importance that was published on the site. You can also see the
SearchDay archives
and the SEW Blog
archives, though material from both of those places is integrated into
Search Engine Report mailings.

Thanks

Finally, some thanks….

Want to comment or discuss? There’s a
thread
going at our Search Engine Watch Forums.

Visit site for full experience
Exit mobile version