Shot of Caffeine for Google Search

Google has enjoyed a nice and solid slice of the web search industry. They were good at indexing the whole Internet and providing relevant results.

However, the web is changing. More and more people are searching for real-time news and the web is getting richer and richer.

Microsoft has seen some really good success with their new search offering – Bing.

Google, naturally, has been working on making it’s own bread and butter business – web search – better. And today they announced the completion of a new web indexing system codenamed Caffeine.

So why did we build a new search indexing system? Content on the web is blossoming. It’s growing not just in size and numbers but with the advent of video, images, news and real-time updates, the average webpage is richer and more complex. In addition, people’s expectations for search are higher than they used to be. Searchers want to find the latest relevant content and publishers expect to be found the instant they publish.

According to Google. they were indexing the web in layers and each layer were updated with web pages at various rates. This means some parts of the web could wait to be indexed for days even.

With Caffeine, Google can index web pages on an enormous scale. How enormous?

* If Caffeine were a pile of paper – it would grow 3 miles taller every second.

* Takes up nearly 100 million gigabytes of storage in one database.

* Will take up 625,000 of the largest iPods

* Stacked end to end it would go for more than 40 miles.

With Facebook and Twitter delivering real time results and news, Google is looking to provide more relevant and real time results in their web search as well.

If you are a webmaster and wonder how this new indexing might affect your search rankings, Matt Cuts discussed about this back in August 2009 – a bit dated but still relevant about Caffeine.

we’re not looking to make huge changes in ranking with this new infrastructure. Some rankings will change, but that’s not the main thrust of the infrastructure.

It is good to see competition brewing innovation and it’s great to see the place i turn to to search anything on the web is constantly evolving. For the better.

[via Google blog]