Just Blogging

Peter Nisbet asked:


Among the many advanced SEO techniques that most people fail to put into practice on their websites is an internal linking strategy that can be used to improve the way search engine spiders crawl your website, and also to optimize the Google Page Rank (correctly PageRank) for each page. Not only is it important to make sure that spiders are not leaving your home page too quickly, but also that you are not wasting Page Rank on pages such as your Contact or Privacy Policy pages.

Collar the Spiders

You can collar and attach a lead to search engine spiders and make sure that they don’t stray away from your web pages too soon, particularly your home page that is liable to receive the highest search engine listings.

Spiders work from top left to bottom right, and if you have tables on your site, they start with the top left table, and scan its contents first, then go the next table and so on. If your website is designed using tables rather than CSS, and you have a left hand navigation table and then a table containing the bulk of your content, the spider will read your top navigation link first, and leave your home page. It will not return until it hits the home page link.

It will therefore miss most of your content. The answer is either to place your navigation table to the left of your content, so it is visited after your content, or to use an empty table top left, then a content table, then a left aligned navigation table. That way the spider will visit the empty table, then your content and then your navigation table, which is still showing to the left of your content, but comes after it in your HTML.

You then receive the full benefit of your great keyword and semantically optimized content, rather than waste it by the spiders looking elsewhere for your main content. When relevance to a search term is calculated, (keyword), spiders give most weight to what is contained within your H heading tags, the first 100 or so characters in the body of the text, and your final paragraph. Your Title Tag is also very important, and you could put your company name and the main keywords there. That helps your branding and the calculation of your listing from the keyword.

Internal Linking Strategy

Now that you know where your links to appear to spiders, where should they point to? If your site is silo structured, then your home page should link to each of the main silos. Do not link to every page in your website, but to Level 2 pages that provide further links to your level 3 pages. That is because but Google’s Page Rank is calculated on internal links as well as external links. Only link your Home Page to every other page if you want the maximum possible share of your site PR for your Home Page.

Your total site Page Rank is equal to 1 vote for every page on your site. So if you have 20 pages, you have a 20 PR votes to distribute. That does not mean that you have a Google PageRank of 20 - far from it. Nobody but Google know how many links or PageRank points/votes are needed for each Page Rank vote. It could be 10 for a PR of 1, 100 for 2, 1000 for 3 and so on, or something completely different.

The internal Page Rank for each page in your site can be calculated since it is a function of both the page rank of pages it is linked to and the number of other links leaving that page. You can use this to maximize the PR votes for any page on your site, or spread them around pages you want listed highest. This calculation involves both internal links and external links.

In fact, you can make an appreciable difference to your SEO and Page Rank if you use a sensible internal linking strategy. With a 10 page website, if every page is linked to every other page, then your internal PR votes are one for every page. However, if you link Page A to page B and then Page B to every other page, and all pages back to Page A, you can give Page A 3.42 PR votes, page B 3.06 and the rest 0.44, thus optimizing the PR of your first two pages (note how these figures add up to 10: 1 for each page).

If you want to give your Home Page maximum votes, link it to every page in your site, and every page back only to the Home Page. For the same 10 page site, Page A then gets 4.67 Pr votes, and the rest 0.59.

However, for a silo site, it is best to have the main silo pages with a reasonable share of the votes, so link the Home Page to the main silos, and then each main silo to the sub-pages in their silo. Everything links back to A. This gives your Home Page 3.60 and your silo pages 1.17, the rest 0.40. There are several options in between these, but the point is that you can use linking strategy to maximise the PR for any page on your site.

No Follow: Beat the Spiders

The nofollow attribute was devise by Matt Cutts of Google. Its intention is to enable you to link to a page without giving that page a share of your PR. This can be used when you are linking to pages that have no outbound links, and for which a PR would not be meaningful. Google claim that it uses the term literally and does not follow the link at all, but test results have been conflicting, and it appears to follow it, but not index it.

Different search engines interpret ‘nofollow’ differently: Yahoo do not include pages linked by use of the attribute in their rankings but does follow it, MSN does not count links with ‘nofollow’ in their ranking and Ask ignores the attribute and follows everything!

It therefore appears that you can use the attribute to prevent spiders from leaving your site by following every link. From my own experience, I seem to get few spider visits to pages attainable by means of a ‘nofollow’ link, and so can use this to prevent spiders going where I want them to go.

The easiest way to do this, though , is by means of the Robots Exclusion code in the HTML for each page. You should use that on pages such as your Disclaimer, Privacy Policy and About page, and also on pages with a good amount of duplication such as pages where products are sorted by name, price, application and so on with a different page for each sorting method. The same for the same page written in different languages and duplicates where only the keyword is changed. All of these can get you in trouble and you should use the Robots.txt exclusion or a specific exclusion for each page involved.

Summary

Advanced SEO techniques can be used to lead search engine spiders where you want them, and prevent them from being sidetracked by poorly positioned links to other pages on your site. You can use your internal linking strategy to optimize the Page Rang votes for each page in website, and can also prevent some pages receiving a share of your sites page rank.

It is important that Google should be able to spider or list all of your pages, since that can not only dilute your overall PageRank but affect your listing position, but if you have too may low value pages being navigated and indexed by search engines, they may put you on a reduced crawling status. Try not to have lower value pages put on a crawling par with your higher value pages that you want listed higher in the SERPS.

Frederick Townes asked:


Often pages or sections of your site have developed some link love that would better serve other pages you were prefer to having ranking or would rather have users spending time with. So what you need to do is get those bank links and PageRank where you need it.

Redirects and Search Engines

Redirects automatically send a visitor to a specific page once the visitor accesses the site. The redirect can be to another page on the site or to a completely different site altogether. Redirects come in a few flavors the most important for this discussion are 301 redirects which are viewed as permanent and 302 redirects, seen by spiders as temporary.

Redirects, regardless of type, have been identified by SEO professionals as PR killers. Matt Cutts, a Google Software Engineer, stated in a Search Engine Strategies conference that “…clear disclosure to machines won’t pass Page Rank [with] a 302 redirect.” Even today there are hackers and crackers using redirects to hi-jack websites, even though spiders are capable of detecting nefarious activity. The behavior of 301 redirects however is quite the opposite.

However, on-site redirects are commonly employed by site owners to increase site visibility on SERPs and to take the visitor to the “good stuff.” Here’s how this technique is employed:

On-Site Redirects: the Basics

It’s important to recognize that the structure of your website is pivotal to long term success in search engine result pages.

The site owner or SEO optimizes a deep site page to the max. The only point of this page is to rank highly on SERPs and in some cases, the text on the SERPs link is unusable by humans because it’s designed for SEO only. Botspeak. However, once the visitor clicks the SERPs link to the highly-optimized page, he’s automatically redirected to a page or a site designed for human use, i.e. optimized for conversion.

Note: Search engine algorithms can easily detect redirects and, frankly, they are still viewed with suspicion by spiders. In fact, a site page may be penalized. The entire site may be whacked if the redirect is deemed suspect.

Spiders are not opposed to redirects per se. In fact, if the redirect is considered valid - as in the case of a blog that delivers a “confirmation of receipt” page before redirecting the visitor to the blog post itself. This is viewed as an acceptable practice because it provides benefit to the search engine user.

Using Redirects Wisely

A link is a redirect - taking visitors to other pages of a site. Embed text links on high ranking pages deep in the site. Optimize these pages for SEO and employ redirects to take visitors to the appropriate zone or page of the site. Example: A site selling insurance products employs the following on a page deep in the site:

“Unfortunately, you never know what the future holds, but you can protect your loved ones now and in the future.”

The link is embedded in informational content, which ranks higher, deep within the site. By placing the text link within this informational context, the visitor is provided an immediate option to the now-recognized risk - the lack of term life coverage.

Internal linking is a critical consideration in site design and content architecture. Creating highly optimized pages designed to appeal to SEO bots deep within a site (the site archives, e.g.), with links to pages higher up in page heirarchy is a legit use of what in effect is a redirect, though bots won’t see these links as such.

Robot.txt and NoFollow

The redirect will be instantaneous (almost) to visitors, adding maybe a second to download times. However, bots will righteously crawl these sites whenever the opportunity arises - and that includes anytime a visitor with a Google toolbar installed visits your site. Googlebots are aggressive little letter-string junkies. A visitor can initiate a crawl so you want to be prepared for on-site bot activity.

Clearly identify those pages that are not to be indexed. This can be by individual page with nofollow or noindex tags, or if there are a number of pages designated off limits to spiders, create a robot.txt file.

A robert.txt file is developed identifying all pages off limits to spiders and placed in the site’s root directory. Remember to update the site’s robot.txt file as more pages are added to the site.

On-Page Optimization

The objective of redirects is on-page optimization - both SEO and conversion ratio optimization. At this stage in the purchase sequence, the prospect has demonstrated enough interest to click on a deep link to learn more about products and services. Once that link is clicked by a human or tracked by a spider, both bots and eyeballs must be happy with what they see.

A key point for non-tech site owners: bots wouldn’t know an attractive site if it bit its digital ****. Bots never see the presentation layer, which could be a mess. Bots read the black and white HTML and XML data that forms the site’s structure. On-page optimization will appear in the page’s code.

On-page optimization involves the deployment of links - which information links to what zone or page? Each page is optimized to drive the visitor deeper into the site and to, ultimately, convert.

Placement of “trip-wire” links is an essential aspect of page design. Trip wire links are embedded in content that evokes an emotional response in the reader, a response that demands immediate action. The example of the embedded insurance link above is a good example of trip wire links that optimize each page for conversion and for more expansive and complete indexing of the site by bots.

Remember, bots follow links. Embed links deep in your site on highly ranked pages. These links automatically redirect the visitor to less optimized but more human-friendly pages or sites.

Embed text links deep in site text with text embedded, intra-site links placed for maximum emotional response, i.e. a link to a sign up page embedded in an informational article about the benefits of jogging. Spiders and humans will follow well-placed, contextual links embedded in the body text.

This insures that spiders stay onsite longer, indexing more pages and more accurately placing indexed pages within the right folder of the search engine’s classification system (the taxonomy employed by Google, Yahoo, etc. determines how and where a site is classified).

Further, redirects can increase both page rank and site revenues when on-page optimization and nofollow and robot.txt files are used to direct the activities of bots. Remember, a Googlebot will try to crawl any site visited by a user with a Google toolbar installed so, virtually any visitor is a potential opportunity for Googlebot to spider a site.

Employ redirects to appeal to both spiders and humans and redirect a site’s page rank in the process.