Why Do I Need To Get My Web Site Indexed
The question of whether you need to get your web site indexed is frequently at the forefront of a business owner's mind. The reasons for this can be myriad: it might be that they have spent a substantial sum on getting their site developed, perhaps they have not had any increase in customers, or it may just be that they do not know what an index is and why it matters to them.
In order to resolve these queries, we will explore the process: what is involved? What should you expect? And we'll also talk about how important indexing your site really is. But first - let’s summarize the actual process.
How does it work?
There are several companies who offer to get your website placed on an indexing site. Sometimes this is done by an automated process - a software program which runs a crawler over the web site, checks if there are any broken links or other issues, and then sends you a report of any errors. Other times, you need to go in and check it manually. Most people will have a website developer do this for them, but if you've decided to go it alone, then I recommend that you start with an SEO company.
The first thing they want to do is check your site's content. When a person goes to your website, what do they see? Does it say anything about your business? Do you have any blog posts on your site? Are there images or any other media files that help the visitor find out more about your business and its products / services?
The SEO company will check these, as well as looking at your meta data (the information found in the html tag of each page of a website) and keyword density. This is not always easy, as sometimes you have to check these things manually; if you use an automated service, they will just take a snapshot of the site when it is new and then do another one at the end. This means that when you make changes to your site, the content will not be indexed. This is why it is important to use SEO experts if you are going to do it manually.
Now, once they have finished examining the web site, they will submit it to the search engines. The engines will then crawl the site and check to see if it complies with their criteria: are there links out of date? Is there too much keyword density (i.e., using words too much) in any one place? Are pages over 100kb? Etc., etc..
Once the process has been completed, you should get a report detailing any issues that were found, as well as suggestions of what changes you can make to your site to improve its overall performance. This is all part of the process - it is not always over. The search engines will send their crawlers back out again, and if more issues are found then they may make more changes, sending you a new report which will outline what needs to be done.
What do the search engines care about?
So - now that we know how it works, let's take a look at what indexing actually is: getting your site onto an indexing site.
The engine (let's just use Google for this example) does not want to list every site on the Internet. They want to list sites which are relevant, and provide their users with the information they need. They do this by ranking your site based on its content and how relevant it is to what they know their users are searching for.
So Google, when faced with a search query of "online computer games", will put the sites that most closely match that query at the top because they have content related to that subject. They will then go through and do this with each result until they populate their results page and all there is left is a link back to the main search page. This is why you might see the same site on different pages of the results - because it matched a different keyword, and so Google has decided not to include that result on that page.
Now, how does your site get into their database? They crawl their own database to see if there is any new or updated content. This means that when you make changes, the search engine will pick this up and add it to their database again. If they do not find any content which is new or updated, then they will check what sites are in their database which are related to yours (using keywords). If they find some, then they will go and fetch content from those sites and compare it to that on your site, also checking any links to or from them.
The search engine will then rank them based on how much content they have which is relevant to the query; if you have a lot of content with keywords which are similar to those used in the search query, then the engine will assume that your site is more relevant than others. If a lot of links point to your site from other sites using those keywords, then Google will believe that to be a good thing. They will then go through every page of your site and export the keywords so that they can be added to the index. The same process will then take place with any pages which have more than 100kb in their content, reducing it down to the size needed for indexing.
Eventually, your site should appear somewhere on Google - but how long it takes is dependent on a number of factors. You can achieve some success quite quickly - especially if you are starting with a low ranking for your keywords, or if you are already very well-known because of having lots of traffic or press coverage. If you have a site which is not doing either of these things, then it will take longer.
It is important to remember that Google likes to keep its index up-to-date, and so as soon as they find something they don't like they remove it from the database. This can happen when you update your site, or when the crawlers find something that is out-of-date. If your site does not come back after a few weeks, try checking for broken links - especially those leading away from your site.
Probably the most common mistake is to have out-of-date links to your site from other sites. If you have a blog post from a few years ago which still has a link to your site on it, then this may well be holding your site back. You can check for this using the following query in Google's webmaster tools:
site:mysite.com intitle:"website" -inurl:(htm|html|php)
The -inurl parameter stops the query searching within image files, so you can be sure that it is just looking at your site.
Conclusion
Hopefully, this has given you a better understanding of how search engines work, what indexing is and why this is important for your business. It is also a good way to prove to your customers or clients that you are engaged with them and want to give them the best possible service. Being able to provide relevant content which the engines find useful will help others find you on their searches more easily, and give you better rankings across other sites which use different engines (such as Bing).
This post was written by Gary Sexton, Internet Marketing Manager at MicroVision. Please feel free to contact us if you have any comments or questions about SEO for small businesses .