Search engine optimization and submission
By Craig Goldwyn, visibility.tv
Search engines can bring you business but you cannot depend on them. You should make an effort to get listed near the top, but the sad truth is that, because everyone wants to be near the top, getting there is nearly impossible. But you can try, and in the process almost certainly improve your visibility and your business. Here's how search engines work and how to work the search engines.
Search engines are very popular because there are billions of pages of info on the web and finding what you need is a needle in haystack proposition. As a result, most people rely heavily on a favorite search engine to help.
The "Big Three"
There are more than 100 search engines, but only three matter in the US. In September 2010, according to Nielson Company, Google had approximately 65% of all searches, and is holding steady. Microsoft/Bing had 13.9% (increasing slightly) and Yahoo! had 13.1% (declining steadily). Microsoft/Bing and Yahoo! have a deal for Yahoo! to use Bing technology and it is being phased in through 2012. Ask had 2% (declining) and AOL had 2% (declining). The dozen or so others account for only 1% and aren't worth the effort (there is even a search engine called dogpile.com - watch where you step!). This all totals slightly more than 100% due to rounding. Worldwide, the number three search engine is China's Baidu.com, behind Google and Yahoo! but ahead of Microsoft/Bing.
How do they work?
Search engines have huge databases of info about billions and billions of web pages. How do they get this info? Each does it slightly differently. There are essentially three ways search engines compile their listings, (1) computer programs, (2) humans, (3) paid inclusion. Some use combinations of all three methods.
Crawlers (aka robots, bots, and spiders) are computer programs that follow links on websites. They "read" the page and categorize the info, rate it and rank it, and then list it. Google uses a crawler.
Human driven search engines actually have people who go to a website, poke around, rate it, and create a listing for it. These search engines are also called directories. Yahoo! is partially man-made.
Paid search engines take your money and your descriptions and list you. For obvious reason, they are not very popular all by themselves, but almost all of the other search engines have a paid component. If you pay them, they will list you prominently or in a special, highly visible location. Overture and Google AdWords are the most prominent paid listing services. Overture provides Yahoo! and MSN Search with their paid listings and Google AdWords are used by Google, AOL, and Ask Jeeves.
How do you get listed, and how do you get the highest possible ranking?
First, you make your site crawler friendly and then you submit yur pages to the major search engines. How do you make a site crawler friendly? The process is called Search Engine Optimization (SEO). Here's an overview.
Start with your domain name
If your business is portrait photography, name your site with something a machine can read, like ChicagoPortraitPhotos.com, not PetesPorts.com.
Metatags
Crawlers read the text on your site, even the programming code people do not see. Some look for "metatags." Metatags hide in the invisible html code in a section called the "head". The most important metatags for the crawlers are the "title" and the "decription". To see the metatags on a web page, on your browser select the menu option "View Source." If the webmaster has included them (and some forget), they will be at or near the top. They will be labeled "title" and the "description". Some searh engines (not Google) read a metatag called "keywords".
Title
The title of the page is a metatag that appears at the very top of a page. For example, on this page it is "Search Engine Optimization" and you can see it above. Google and other Search Engines hold it in high esteem. You must characterize the page in the title, and try to pack it into 70 characters because that's all Google reads. The words in your title should also appear in the description and in the body of the text on the front page. Search engines look for this. Clever, no?
Descriptions
This is the second most important metatag. A description is a regular English sentence that describes succinctly what the page is about. Google reads the first 155 characters. Some crawlers look to see how many of the keywords are in the description. And then they look to see if they show up in the text of the page.
Now write a sentence or two that describes the page using as many of the keywords people might use to search as possible. Keep in mind that this sentence will appear in your listing in the search engine, so it should be compelling, say a lot, and be front end loaded because the entire description may not appear when the results of a search appear in the "hit list."
Keywords
Keywords are not as important as they once were because Google and other search engines use software to pull keywords from your text. But they are worth doing just the same. There are a lot of tricks in writing keywords. Let's say I'm writing keywords for a pet photographer in a Chicago suburb. Hey, everybody knows "sex" is a popular search word, so why not use it as a keyword so people searching for sex will find my site. If I use "sex" as a keyword in the metatags of a pet photography site, the crawlers ask "is the site really about sex?" Should it be categorized as a sex site? Not if the word sex doesn't appear in the visible text of the site. How about "pets"? You bet. "Dogs" and "cats". Sure. Should I list all pets? Nope. Because some search engines use only the first ten keywords, and we have other important words to get on the list. How about "photography" and "photographer"? Better include them both.
If someone in Chicago is looking for a photographer to make a portrait of their pet, they probably don't know there are pet photographers. So they will probably search on "photographer" and "Chicago". So even though my client is in nearby suburb she will not be on the list unless I include "Chicago" in the keywords. Probably "portrait" should be on the list.
According to Danny Sullivan or Search Engine Watch "Your target keywords should always be at least two or more words long. Usually, too many sites will be relevant for a single word, such as 'stamps.' This competition means your odds of success are lower. Don't waste your time fighting the odds. Pick phrases of two or more words, and you'll have a better shot at success."
Use the plural or long form of a word for a keyword. If you use "pets" most search engines will find you if the person searches on "pet". But if you use "pet" and they search on "pets" you may not be found.
Try to get your keywords into the document title and the top level headline as well as the text. This is verey very important. Write your home page text carefully so it contains as many of the keywords as possible. This really tells the search engines that you really are all about the topics in your metatags.
To prepare a list of keywords, go to http://www.wordtracker.com. You can purchase the service for about $8 per day, but they offer a free sample and the free sample may do the job just fine for you. It will take you about 30 minutes, but it is time well spent. Just put in the obvious keywords such as "pet photographer" and you will be pleasantly surprised at the other keywords wordtracker suggests. As you are doing this, make believe you are a typical client looking for your services in a search engine. What would they search for? Think of typical synonyms, catch phrases, and even misspellings. Write down the keywords and list them in descending order of importance.
Then go to Yahoo! keyword search tool and search on a few of your keywords to generate some new ideas. Add them to your list.
Yahoo! Local and Google Maps
If you are strictly online ordering and do not need people stopping into you office, or if you are a service like a mergers and acquisitions attourney and you don't want walkins, then you can ignore Yahoo! Local and Google Maps. If your business depends on local sales, like a liquor store or a dog kennel, then click the links and fill out the forms. These can be very helpful traffic generators.
Recon
Check out your competitors. Enter some of your keywords and see which sites come up. In your browser choose "View Source" and see what keywords and descriptions they have used. Steal the best ideas.
Page titles
Crawlers read the title of each page so it should be meaningful and contain keywords. Also, name the pages with logical names. An name like "dog_photos.html" is much better than "page1.html" or "fidofotos.html". Avoid using symbols or spacesin URLs especially the ? symbol. Make sure your webmaster is doing this.
Page content
The text on the page can be crafted to be search engine friendly. Get the important keywords into the headline and the first paragraph. If the text contains a lot of the keywords and phrases from the description, and they appear often, you will get a high grade. Some crawlers even measure how close to the beginning of the page keywords appear. But don't overdo it. If you repeat the same keyword all over the page, the crawler may consider it spam and give you a lower score.
Frames are bad and databases are also a problem
Some crawlers cannot read pages that are produced with an html technique called frames. That alone is a good reason to avoid building sites with frames. None of the crawlers can read information kept in a database on your site. That's a big problem for sites that have thousands of listings such as a large catalog or restaurant review site.
Image maps are bad
Some webmasters use a technique for creating links to inside pages called image maps. These are pictures cut into puzzle pieces with each linked to a page. Some crawlers cannot find links this way so you should use text links at the bottom of the page to help the crawler.
Flash is a problem
Adobe Flash is a superb program for creating animations and multimedia. Very hip, cool, and techie. Problem is that search engines have problems reading links in Flash. For this reason I strongly recomment you NOT use Flash as your home page.
Image names and alt tags
What if the homepage is a jpg photograph with the type embedded in the jpg? crawlers can't read text embedded within graphics. Then it is important that the invisible name of the picture contain keywords like "dogs.jpg" and that it has an "alt tag" with keywords in it. In the home page at right, "Picture your pets" is embedded in the picture and is not readable by crawlers. So is the text on the buttons. But the blue text at the top and bottom are machine readable html text. Better still, the text at the top "fotos by felicia" is designated "H1" so the bot can tell it is the main headline. If the word "fotos" was spelled properly, it would significantly help the page's position in bot driven search engines.
Drop downs
If your site uses drop down menus created by javascripts, it is a good idea to create a navigation page with the same links as in the drop down menus. Sort of a table of contents. You should also have the top level menus in html links at the bottom of the page as a convenience to both your users and the bots.
Reciprocal links
Google also look to see how many other websites are linked to you so it is important to get your friends and business associates to link to you from their web sites. Likewise, you should have a links page to link to your favorite sites and to the sites of those people who link to you. Search on your keywords and ask the top sites to link to you and offer to link back. With crawlers considering who links to you, it's not just quantity, it's quality. To check for links to you, try these sites.
Sitemaps
Search engines also use a feature that you should use. You can upload a sitemap (a list of all the pages on your site) to them. The require a special .xml format. But it helps search engines list you, so you should do it (I will do it for you if you are among my clients).
Here's a service that will generate a proper xml sitemap for Google, Yahoo!, and Bing, and submit it for you whenever you want http://www.automaticsitemap.com
Backlinks
Google looks at how many websites link to your site. If there are a lot, and if they are big important sites like the New York Times, then Google thinks you are important. You should make a serious effort to ecxchange links with other websites. If you are a pet photographer, link to the ASPCA and shelters and vets and pet supply stores and get them to link to you.
Your home page
It is crucial that your home page contain as many of the keywords people will search with, and that those keywords appear in your title, and in your description.
Be educational
Search engines give more credibility to educational and news oriented sites than commercial sites in many cases. So if you search for "BBQ Ribs", my website AmazingRibs.com comes up #1, ahead of all the world's most famous ribjoints partially because it is more informative while the others are strictly sales. So if your site is selling bicycles, in addition to promoting your services, you might write several articles about how to select, maintain, and ride bicycles.
Do it yourself or not?
So what's the first step? Decide if you want to submit your site to the search engines yourself or if you want to have someone do it for you. I recommend you have someone experienced do it for you. Many webmasters own special software for optimizing sites and submitting them. There are also services that can help you.
Not surprisingly, I offer search engine services to my clients. When I build a site I start by making page names and titles search engine friendly. If you ask me to do SEO and submission, we will work together to develop the metatags. I then go into every page and make believe I am a bot. I then go back and make believe I am a human being looking for the products and services of this site. To keep from becoming permanently schizo, I use special software to help me make these transformations. It helps me fine tune the metatags without exceeding word limits and grades the bot friendliness of each page. My software and I then prepare the proper submission forms for each of the search engines.
As you can see, getting a prominent listing in the search engines is a black art. One thing is certain: Anyone who says they can guarantee you a high ranking in a search engine is a liar. But because good placement in the search engines can mean a big boost in business, you gotta try.
Pay to Play?
Non-commercial sites can be listed in Yahoo! for free, but commercial sites must pay $299 per year. Yahoo! is a very influential search engine, so if you are a commercial site, you should consider paying. The experts say it is a good investment for at least one year because other crawler-driven search engines will find you in Yahoo! faster that way. You can also pay Yahoo! and Google to expedite their things and list you faster.
You should consider paying to have your site displayed more prominently by search engines. For example, you can pay to have Google list you in the special advertising list on the right and at the top if people type in a keyword.
How to submit?
The next step is submission. You can save some time and effort and just wait for the search engines to find and catalog you. Some will find you within a few days. Others take weeks or months. But most wait for you to tell them you're out there and then they'll send out their crawler or reviewer. Each has a different procedure for submission (both meanings of the word apply here), but all require you to fill out a form. And no two are alike.
Google. You can submit at the Add URL page. You can add 2-3 urls this way. Google's crawlers count links to you, so it is important that you get as many as you can. Google also pulls some data from the Open Directory. Google Adwords costs $5 per month and monthly fees are typically about $20 if you've chosen your words well. Google AdWords are also used by AOL Search, Ask Jeeves/Teoma.
Yahoo! If you are a non-commercial site, go to Yahoo! search on some keywords to find the correct category for your site. Then click on the "Suggest a Site" link that appears at the top right-hand corner of the page and fill out the form. One can only hope that they know something about the subject and that they will evaluate you fairly. I am often puzzled about their choices. And if they ignore you, no matter how good you are, it is impossible to identify these people and plead your case. But if your site is not finished yet, it's best to wait. Yahoo will not give its best ratings to incomplete sites or sites with a lot of "under construction" signs. If you are a commercial site, go to http://add.yahoo.com/fast/add?+Business, fill out the form, pay the $299, and the Yahoo! editors will pick a category for you. For more info about the Yahoo! options, click here.
Overture. Overture is pay to play, but their listings appear on Yahoo!, MSN Search, and some other smaller search engines. You bid for each click through to your website. If you are the highest bidder, you are listed at the top. It is highly recommended to open an account at Overture with a $50 minimum deposit and pay the minimum of $20 per month.
Open Directory. This is a volunteer operated directory compiled by humans. Several other search engines check it in compiling their directories. Therefore it is important to submit to the Open Directory. Submission is free, but it can take a while for them get to you.
Ask Jeeves. This search engine became popular by claiming to be a natural language search engine, but, unfortunately, it doesn't do that very well.
Teoma. This up and coming search engine is owned by Ask Jeeves but there is no way to ask them to list you. You just have to wait for their crawler to find you, so make sure people link to you to facilitate the process.
AOL. Their search results and paid search results are provided by Google. They also get some data from the Open Directory.
MSN. Although it was Yahoo! and Overture driven for years, MSN has recently created its own crawler and search results.
Looksmart. A human driven directory that has a magazine article search feature.
Patience
Unless you pay the expedited listing fees, it may take a search engine up to four months to list you if at all, so don't wait! Even then, because you are new, you will not likely be near the top.
Check for results
You can go to a search engine to see how many pages it has listed. To see how to do this go to this page on Search Engine Watch.
Your web server keeps track of how many people visit your website and where they were before they came to you. By watching your web log, you can tell which search engine is working, and which ads are working.
Resubmit
The forms change occasionally, and the methods each engine uses change. If you don't get the ranking you think you deserve, work on optimizing your pages and submit again. Everytime you make major changes, resubmit. If you don't get the results you want, try again, but don't obsess. As Danny Sullivan says "Know when it's time to call it quits. A few changes may be enough to make you tops in one or two search engines. But that's not enough for some people, and they will invest days creating special pages and changing their sites to try and do better. This time could usually be put to better use pursuing non-search engine publicity methods."
Other resources
http://www.google.com/webmasters/ - Google tells you what you need to know to make your site Google friendly.
http://searchenginewatch.com - The guru of search engines is Danny Sullivan. All the top webmasters and SEO experts read his Search Engine Watch newsletter and read the latest reports on his site.
http://searchenginewatch.com/links/ - For a description of major search engines.
http://searchenginewatch.com/webmasters/index.php#essentials - Part 1 of several pages that tell you most of what you need to know.
http://www.highrankings.com/forum/ - A great message board where people help each other.
http://www.ihelpyou.com/forums/ - Another great message board where people help each other.
http://www.submit-it.com/subopt.htm - SEO tips from Microsoft.
http://www.highrankings.com - Sign up for their free newsletter.
http://www.SiteProNews.com - A newsletter and resource site for webmasters, updated daily.
http://www.seo-news.com - Articles about SEO.
http://www.seo-news.com/forum.html - A message board with give and take on SEO.
http://www.freewebsubmission.com/site-ranking.html - How high are you ranked?
http://www.trafficzap.com/searchsubmit.php - Search engine submitter.
http://www.searchengineworld.com/cgi-bin/robotcheck.cgi - Check the validity of your robots.txt file
http://www.freefind.com - Creates a search function for your website.
http://www.google.com/searchcode.html - Creates a search function for your website.
http://www.searchbliss.com - Some really useful tools to help you see how you're doing with the search engines.
http://www.quirk.biz/searchstatus - A Firefox plugin that allows you to see how your page ranks in Google and loads of other valuable info.
Other useful stuff
Want to see if Google's indexed all the pages in your site? Go to Google.com and enter this in the search field: "allinurl:www.yoursite.com". Look at the Results Number in the upper right.
To see how many sites link to you, go to Google.com and enter this in the search field: "link:www.yoursite.com" or even "www.yoursite.com/gallery.html" to see links to specific pages.
To find something on a site, enter "keyword site:sitename.com". For example, to find the word mustard on the site amazingribs.com enter "mustard site:amazingribs.com"
This page was revised on 9/15/2010
