RustySearch: A Search Engine Relevancy Challenge

We all know that trying to define and measure search relevancy is quite a challenge. There are so many variables both human and mechanical) that can come into play. Nevertheless, it’s important to try. Today, news of a new site from Barry Schwartz (you might know him as RustyBrick) that he calls RustySearch. The site is a search engine relevancy challenge to measure relevancy from one of four randomly chosen engines. Details on how it all works here.

You might also want to take a look at this SEW Forums thread: Coke vs. Pepsi Challenge for Search Engines from about three weeks ago where Barry and others (including Danny) discuss the idea of such a test. This thread has even more links to material about relevancy.

For many non-search geek types (the people who are not likely to take this test) their happy with whatever they find quickly, often not realizing that something else could better serve their needs. We can call it the principle of least effort. However, as I’ve said many times before, a little bit of user education (I’m talking minutes not hours) on how to create a query and use some of the features many engines offer can help the searcher create precise queries that might produce more relevant results. Perhaps Udi Manber said it best at PC Forum a few weeks ago when he told the audience that search engines are not mind readers. I would also argue that user education also extends to vertical engines and specialized tools. Sure, verticals can produce very relevant results but unless the searcher knows that the engine exists, they can’t get these results. In this sense, user education is also how a vertical let’s the public know that they are open for business. It’s also important to consider the authority (where the info is coming from) and currency of the information being retrieved when looking at relevance. Critical info skills might be more important today than every before.

Related reading

Four tools your business needs for better metric reporting
steps to deliver better technical seo to your clients
search reports for ecommerce to pull now for Q4 plan
amazon google market share for ecommerce, data