SEM Support Tools for Enterprise-Level Projects

Most SEOs swear by their tools. Although human brainpower is the most important tool in any SEO engagement, that power can be further harnessed and utilized more efficiently when supported by advanced tools capable of providing insight into a Web site and its industry competitors.

In the introduction to the Avenue A | Razorfish SEM team, An Inside View of the Agency, I stressed that the Product Technology (PT) team was an invaluable resource in helping client Web sites rank well. The technologies created by this team go beyond merely combining many of the tools that can be found elsewhere for free in open source or for purchase.

This week, SiLC, the “Super-Intelligent Link Crawler,”™ introduced in SMTrends in early 2007, became fully operational. The first SiLC Case Study describes how the technology helped U.S. News & World Report realize additional organic search traffic.

The Importance of SiLC to the Future of SEO

SiLC is a technology that will continue to grow in value for our SEM team and its clients. Used in conjunction with SEOsource (a tool suite specifically developed to support our SEO team), the potential to improve efficiencies and raise the statistical probabilities of analyses and predictions derived from these tools is limitless.

In the article announcing the case study, VP of SEO and Web Analytics William Flaiz shared that SiLC “…serves many purposes beyond Web crawling. It is different than other Web crawlers because it can intelligently process why certain Web pages aren’t performing well and integrate with other SEO tools to improve efficiencies not only within a web site, but to other linked Web sites, portals and blogs.”

Mark McWilliams, the Chief Architect behind SiLC, offered some additional thoughts. “By crawling site content that search engines cannot, SiLC identifies on-page factors that prevent our clients’ Web sites from showing up in search engines. When combined with SiLC’s performance and error monitoring capabilities, this feature gives us deep insight into the structure of complex Web sites.” He went on to say, “SiLC’s extensible architecture lets us easily add features or perform custom analyses to meet specific client needs.” This is precisely the scalability that differentiates this kind of advanced technology from many other off-the-shelf tools.

As more and more SEMs grasp the importance of dedicating resources and personnel to creating supporting technologies, the industry as a whole will gain through analyses and case studies derived from these types of tools.

Other Tool Options to Consider

Creating an intelligent crawler is one thing, but what about other, more basic tools that can be used to help improve SEO efficiencies? One of the most popular free tools of all time was the now-defunct Overture Keyword Suggestion Tool. This provided armies of SEOs and SEO-wannabes with the ammunition to show activity within search engines related to a particular subject. Since as early as the 1990’s, various developers and smaller SEO shops have created tools to help support SEOs.

Some excellent collections of SEO tools from which to generate ideas can be found at,, SEOMoz (highly recommended: the premium tools including the “Juicy Link Finder”), and other sites such as, which offers collections of both free and paid tools.

When designing tools to support SEOs, it is important to try to get the people who actually use the tools involved. As mentioned in the original agency structure article, our team uses the “Agile” development process to help ensure that iterations of the tool always include feedback directly from the analysts using the tools on a regular basis.

SEOsource – The Ongoing Growth of an Enterprise SEO Toolset

As described in SMTrends in October 2006, SEOsource offers many functionalities, including the “recently released Site Overview Report feature, targeted at both analysts and the business development team, (which) allows a user to evaluate multiple metrics at once across a site. Based on the results, the report generates alerts for key problem areas that can prevent the site from ranking well within Organic search results…”

One of its most important features is that the tool is targeted toward both the analysts and the business development team. This special function allows a sales team member, with less technical knowledge, to provide a timely and important high-level overview of the site, while allowing the analysts to use the same data to help scope project costs.

Another great feature of SEOsource is that site-specific recommendations can be printed directly from within the toolset in a client-friendly manner, including specific explanations. For example, should a 301 redirect process or a URL rewrite be recommended, SEOsource will provide a multi-page document detailing the specific steps for the client’s developers to implement.

Tools Plus Brains Equal Evolution

Man evolved partially through learning to use crude hand tools and building fires. In this next generation of human evolution, tools are developed on a daily, if not hourly, basis to perform advanced calculations for scientists or provide accurate actuarial tables for insurance underwriters, for example.

SEO and SEM experts are fortunate in being able to benefit from a large amount of actionable data that exists “for free” within Google and other portals and analytics suites. In order for large agencies to evolve at the same pace or faster than the smaller and sometimes more nimble SEM firms, the creation of enterprise-level tool suites such as SEOsource and supporting technologies such as SiLC is paramount to continued success.

As usual, I welcome thoughts and opinions at the Search Engine Watch Forums thread dedicated to this column, “Just One Agency Point of View.”

Related reading

Search engine results: The ten year evolution
Five ways PPC customer support can help SMBs
#GoogleDoBetter The latest on internal issues at Google and Alphabet
Google Sandbox Is it still affecting new sites in 2019