Having a Website

    In Chapter 2: Having a Website, you will be given a checklist of what makes website a good one. Learn about the parameters that search engines measure to determine if optimization can be effective for that site. Perhaps you will need to make a few changes with your own site to jive with some of the easy-to-follow suggestions in this section.

 

  • Having a Website
    • Website SEO Quality Parameters
      • Search friendly Domain names
      • Directory Structure, File Naming & Page File Extensions
      • Navigation Menus & Drop Down Menus
      • Robots Exclusion Protocol (Robots.txt File)
      • Error Trapping
      • Google Site Maps & Image Maps
      • Cascading Style Sheets (CSS)
      • Server Side Includes (SSI)
      • Graphic-Heavy Pages, Flash & Intro / Splash Pages
    • Search Engine Rules
    • SEO Terminology
    • Dynamic Pages
    • Doorway Pages / Cloaking
    • Frame based sites and Tables
    • Is your website SEO Friendly

Robots Exclusion Protocol (Robots.txt File)

    Basically, a Robots.txt file lets the web crawlers or search engine spiders know what pages of your site should be indexed and which content should be excluded. In some way, as webmasters put it, you are giving the spiders and bots a “stop” sign, so they will not index the content of that said page.

    How significant is this? If for example, you are selling basketballs of different brands, each brand of basketball like Spalding, Mikasa, Nike, and the like will likely have its own web page. If you will be selling basketball-related items such as an air pump, it will be smart for you to mention that product in all the pages of your basketball brands, so that the visitor will surely not miss it.

    However, if you include the air pump in all those pages, it will be redundant or repetitive for the crawler or bot to index the air pump several times. If this happens, your page might not be indexed, and may even turn up as spam.

    The Robots.txt file can prevent the spider or crawler from indexing the additional pages that contain the air pump, so your site will still be in good standing if ever a user wants to search for that product over Yahoo, MSN, or Google.