Reader Q&A: August 2003

Q. How often do the search engines and/or directories refresh or update their content? I have just recently submitted and have attained number one position on most of the majors for every keyword I wanted – except the most important high traffic, competitive ones. My question is basically, when can I expect refreshes and movement in positioning?

A. Most search engines have results that come from crawling the web, and these results tend to be updated on a roughly monthly basis. However, some updates are constantly happening, so there’s always the chance for slight changes to happen. It may be that you’ll maintain your ranking at various search engines for months. However, it could also happen that updates and algorithm changes will cause you to slip. For this reason, it’s good to also consider having a paid listing budget and use that to cover spots you don’t do well in naturally.

=========================

Q. I am seeking a program that will allow me to track how each of my two dozen or so web sites rank with a multitude of search terms on the most important handful of search engines.

A. You want position checking software. There are a variety of packages, and you’ll find them listed on the Search Engine Optimization Toolbox page. Be aware that some search engines may not like the use of these tools. Google has actually blocked the ability of some people to search at Google for over using these tools, and articles about this are listed on the aforementioned toolbox page.

=========================

Q. I’m gathering information regarding lost opportunities due to poor site search engines. I wanted to know if you have any information/articles regarding companies that have quantified the opportunity loss due to poor, non-interactive site search.

A. I maintain a list of articles on site and enterprise search, and SearchTools.com provides even more extensive coverage of the topic. If such information is out there, these are sources that might list it.

=========================

Q. Why is it that Alexa shows 111 sites linking to my website and Google shows about 39, most of which are not high ranking for my keywords? In addition to our own site ranking very high for relevant keywords, we have links to us from other sites that rank high (first page) as well for the same keywords; and one of these is ranked #1. However this site does not show up in a link search, even though I’ve submitted it many times to Google for recrawling.

A. Google’s link lookup tool has been undependable for some time. I’ve been meaning to do an actual article about it, so this is a good prompting to address the issue. I asked Google about this at the end of June. The response was that they agree that you won’t see all the links pointing at you. Google says this has been the case for some time, historically because they didn’t have the computer capacity to run these type of intensive queries. Today, they say it doesn’t remain a priority to correct this, since it’s mainly search engine optimizers rather than ordinary searchers who do these type of queries. What does show up? Some have said pages with a PageRank value of 4 or higher. Google says there’s no particular value assigned but that the links you do see should be those with a high PR value. The good news is that while Google may not show all the links pointing back at you, it does say you should still be getting credit for the links.

=========================

Q. I was not showing up with search engines because I had a bad page title. Now I’ve changed it. Do I need to resubmit it to the search engines and if so how do I do that?

A. There’s no need to resubmit. All the major search engines are constantly crawling the web. If your page is already listed, then generally within a month, you should see the listing get updated. However, it doesn’t hurt — especially for AltaVista. You can find a list of Add URL pages on the Submitting To Crawlers page. Not all search engines offer these pages, however. Some also offer “paid inclusion” programs. With these, if you are willing to spend about $25 to $40, you can get your page updated every few days, for six months to a year. The nice thing is that if your title changes haven’t worked, you can make more changes and then quickly see if the changes have helped without waiting for the usual long refresh time. The page I mentioned also talks about the paid inclusion programs that are offered, as well.

=========================

Q. Google is extremely good in selecting results on language basis. It is specially interesting that it makes real good distinction between Serbian and Croatian sites which have almost the same language. And it doesn’t do that using HTML tags (AllTheWeb does that that way and results are very poor). My guess is that Google uses Open Directory data about regional sites and to find which sites belongs where. But maybe there is an easier method?

A. Crawler-based search engines generally have not depended on the HTML language meta tag to classify pages by language. This is because the tag isn’t always present or sometime is incorrect. Generally, they’ve instead relied on analyzing the page and looking for common words to a particular language. The charset meta tag has also been used. Google may simply have a better system for this with Serbian and Croatian pages. For more on the topic, see the Countries And Languages page.

=========================

Q. Could you post more info in your newsletter about how to contact Google when suspecting a site has a penalty? My site is not being indexed by Google and I have not used any spam methods at the site — and I have not linked (knowingly) to any link farms. However after 6 months, only the home page is indexed. I purchased this domain shortly after it expired and was wondering if the previous domain had a penalty that transferred to my site. If this is not the case then perhaps my affiliate links are causing problems?

Google’s webmaster information says the address to write is [email protected], but Google told me last week that this is a mistake — the better address to use is [email protected]. I’d explain the situation to them and hope they’ll look into the matter for you. By the way, for those who KNOW they’ve spammed Google, use the same address and a subject like such as “Reinclusion Request.”

=========================

Q. I have been contacted by phone today by The Original Browser Company claiming that they have designed software that will be used by millions that will allow graphic ads for half the click thru price of Overture and that will be incorporated in IE and Netscape. Is this a boiler room phone scam?

From reading the company’s web site, the only way anyone will see these ads is if they have special software installed. As this is a brand new company (apparently), the install base right now is probably close to zero. The company does make claims that the base will grow to 10 million by September 2003. Perhaps, or perhaps not. Certainly other companies have built traffic by bundling software into distribution channels. FYI, from reading the site and looking at some contact addresses, I also suspect this might be connected with plans NetWord had to revive some browser-based search-targeted advertising. See my past article about this. Overall, my advice would be to wait and see if they base really does grow and that traffic can be demonstrated to your absolute satisfaction. If so, then you might want to test this.

=========================

Q. Does a banner link count as a link in Google’s popularity rank? In other words, if some site purchases a lot banners, will it increase its Google PageRank score?

A. If the banner is a pure HTML link to you, then sure, you might get a little credit for it. However, since it is a banner, rather than a text link, you won’t get any context associated with that. Should the click be redirected, then maybe you’ll still get some link credit or maybe not. Google and other search engine can be iffy about this. Overall, by the banners because you think they’ll get you good traffic on their own, not because you think they’ll help with Google and other search engines. Also see the More About Link Analysis page for more on the issue.

=========================

Q. Can you tell me which search engines DO NOT save queries and do not make them available to others? Reason is that I sometimes want to search for ideas for inventions without possibly leaving clues behind for others.

A. Well, any search engine that you search at is going to make a record of your search. Several search engines do create lists of popular search topics that get released to the public, but your invention-oriented searches are likely to be so unique that they wouldn’t show up here. Some search engines do live displays, so it could be that your search might show up this way — but it’s really unlikely that someone in a position to steal your invention would happen to catch it. Aside from what I’ve outlined, most search engines don’t provide service where others can simply mine the search data. And — even for those who might do this — the sheer amount of data means your fairly obscure searches will get lost in the noise. In short, visit a place like Google should be pretty safe. However, when you click from a link at any search engine to visit a web page, your search terms may be transmitted in the referral data sent by your browser. That might be easily noticed by the people who operate the web site you come to — and if these are potential competitors, then you really might be broadcasting your intentions. Try a tool like Anonymizer.com to disguise your identity and remove your referral data.

=========================

Q. How can a web-page rank high on Google (or any other search engine) for a keyword that does not appear in the HTML code of the particular page?

A. If people are linking to you using that word in their links or near their links, that may help. See the Link Analysis And Link Building page for more about the concept.

=========================

Q. I just tried registering a new client with Lycos, and saw that they were working hand in glove with Inktomi and AllTheWeb. What’s going on? Why would Lycos give up its own stream of registration income?

A. Lycos sees part of its future as providing search engine marketing services such as paid inclusion to everyone, and that means its agnostic about the choice of providers it offers. If someone is important, then Lycos ultimately wants to include it in its offerings. In addition, the company does still have a partnership with Inktomi via the HotBot site, so it’s not giving up income by convincing you to sign-up for Inktomi as well as AllTheWeb.