Jalaj

Google spiders crawl your site, index the pages, rank them by checking how many other sites think your site an important one by linking to it, and depending on how relevant your page is to a visitor querying Google, shows it on the search results. It was all controlled by Google with your site only playing the part of putting up the pages.

Now Google decided to give webmasters power to decide how Google spiders should crawl their site, to show which new pages have been added, and to know how their site is seen by Google. Here comes the Google Webmaster Tools. All you need to do is to have a Google Account and claim your site and after Google verifies your ownership you get to use these features.

You can use one of the two methods for verifying your site.
1- You can upload a page with name as suggested by Google
2- You can add a meta tag given by Google to your home page.

If you are unable to get your site even by using the above two methods then you need to check with your host. They may have made settings that sends a 200(OK) status even for pages that don’t exists for which either 404(unavailable but may appear) or 410(permanently removed) status. At least for the 1st method this anomaly will prohibit you from verifying your site. If you are a wordpress.com user you can get your blog verified by referring Get your WordPress.com Blog Google Verified.

Got your site verified! Then Welcome to Google Webmaster Tools.

The first page that you would be greeted with a Summary Page as below.

The summary page will show you when Google last crawled your site, whether pages from your site currently exist in Google index, and if Google encountered any errors while crawling you would get to know here and following the details link will show exactly which page encountered errors. These error detail pages are accessible from “Web Crawl” link of the sidebar.

Similarly “Mobile Web” link will show you errors if you have a mobile version of your site.

“robots.txt analysis” link shows you if your site contains a robot.txt file or not. And if it exists, shows whether it affects the crawling behavior of Google spiders or not. If you are unaware of role of robots.txt in instructing Automated crawlers as Search Engine spiders and desktop applications allowing you to download entire or a part of website, please follow this link.

“Manage site verification” allows you to instruct re-verification of all site owners. If you are the only verified site owner for your site, it simply displays the filename or meta tag that you used for verification with instruction that you need to ensure that it always remain in place.

“Crawl Rate” link takes you to a screen which shows Googlebot activity for last 90 days in terms of “Number of pages crawled per day”, “Number of kilobytes downloaded per day” and “Time spent downloading a page (in milliseconds)”. You get a “Maximum”, “Average” and “Minimum” values for each of above. This page also allows you to change the Googlebot crawl speed to “Faster”, “Normal(Default)” or “Slower”. The change that you make is applicable for next 90 days after which Googlebot returns to “Normal” and would require you to change it again if you like to.

“Preferred domain” page allows to indicate whether you would like your site pages with or without prefix “www.”

“Enhanced Image Search” page allows you opt for images from your site to be included in Enhanced image search, which uses a tool Google Image Labeler to label images making them more searchable.

Google removes pages from its index after it disappears for a considerable time or is blocked by site owner either using robots.txt for noindex metatag. But for any reason you want a speedy removal of one or many pages from Google Index you can use “URL Removals” page to place removal requests which would be used by Google to remove the requested page(s) from its index.

All the above pages are available under “Diagnostic Section”. Now let’s move to next section which is “Statistics”.

The default page under “Statistics” section is “Crawl Stats” which shows the distribution of PageRank among pages from your site indexed by Google. All pages newly indexed by google and others which are for some reason not ranked by Google are shown under “PageRank not yet assigned”. Pages with PR0-PR4 are shown under “Low”, and Pages with PR5-PR7 and PR8-PR10 are shown under “Medium” and “High” respectively.

“Query Stats” page shows you the top search queries that showed up your site in search results and the average position from top at which your site was placed on those search results. Similarly you are also shows those top queries which not only showed you in search result but also made the researcher to click and visit your site page. You can see these stats for “All searches”, “Blog Search”, “Mobile Devices” and “Web Search”. You can even classify it further location-wise with list of all location-specific google sites where your site page showed up, in a dropdown list. You can also download the comma separated values (CSV) file for a particular search-type/location combination or the overall data.

“Page Analysis” shows the link text from external links, variations and how they are seen by Google. You can also see content distribution as HTML/XML and ASCII/UTF-8 etc depending on content indexed by Google.

“Index stats” shows links to see your site content in index. These are also available if you use the Google “Advanced Search”.

Now let’s move to next section “Links”. The section though small is an important one, after all, it’s only the links that will decide you PageRank. The default page for this sections is “External Links” which shows the number of pages that are linked from outside and the number of overall such links. The table shows a summary which is also available as a CSV file if you click “Download this table” link. “Download all external links” link allows you to download the CSV file containing details of all external links with page that they are linking to and when was the first time that Google saw this link (PageRank depends on age of the link too).

“Internal Links” page shows you similar table/CSVs for all links from within the site or from its subdomains. Links from within the site and from subdomains are rated lower in comparison to external links (The biggest reason I immediately moved to my own domain jalaj.net).

The links shown in above pages contains data for all links including those which contain a Rel=”NoFollow” tag, which is not included when calculating the PageRank for pages.

The last section, “Sitemaps”, allows you to add a sitemap (an xml file confirming to Google Sitemaps specification) for your site. Google will refer to these when indexing pages from your site. Using a sitemap insures that the latest pages from your site are not missed by Google Spiders and they index on time. Google also support RSS feeds as and for Sitemaps.

That’s all for the Webmaster Tools as it exist now. Google may add more features with time. And if you believe me, Google Webmaster Tools is not just knowing how Google sees your site, but also how you can modify yourself to get into it’s focus.