Hadoop's brain power

In many ways, cloud computing looks like not much more than glorified Web site and application hosting. Yet in other, probably more important, ways it really is something altogether new. It's with this dichotomy in mind that telecom operators and cloud service providers are exploring Hadoop, which squarely fits in the altogether new category and holds the promise of turning tomorrow's clouds into giant brains capable of analyzing huge sets of data to uncover heretofore-unknowable truths.

What is Hadoop? Introduced originally by Google (whose version, called MapReduce, was built to help the search engine consume and analyze the entire Web) and forwarded as an open-source project by Yahoo! (which uses it to customize content for its 20 million users), Hadoop is a way to store and query against extremely large data sets. Today's relational databases and business intelligence tools are powerful, but what if you have 100 or 1000 times the data? Hadoop not only can analyze all that data, it breaks its work up over thousands of inexpensive servers, making it affordable, as well.

Hadoop began as a start-up toy, helping to power early cloud computing services from Amazon and Google. But now it's going mainstream. The first Hadoop World conference in October counted China Mobile, JP Morgan Chase and Visa among the companies talking about how to use Hadoop for vertical data-crunching applications. Now, telecom operators are not only considering providing Hadoop as a service as part of their cloud offerings, but using it internally to solve big-data problems, such as how best to configure their networks or understand their customers.

“Our view is that Google, Yahoo! and Facebook are not different than other companies, they just represent the future,” said Mike Olson, CEO of Cloudera, which is trying to commercialize Hadoop via a software and services model. “It used to be hard to get your hands on a terabyte of data, but not anymore. In the future, the companies that win will be the ones that understand data the best.”

Hadoop essentials

A cloud computing application for crunching huge data sets across inexpensive computers

Powers Google and Amazon cloud offerings

Can also be used outside the cloud for vertical enterprise applications

First Hadoop World event attracted mainstream companies, including telecom operators

Named after the pet toy elephant of Hadoop creator Doug Cutting

TO READ A LONGER VERSION OF THIS STORY, SEARCH FOR “HADOOP” ON TELEPHONYONLINE.COM.