I have been to several software code camps and conference where Hadoop was mentioned and demoed. I am still not 100% of all the uses of Hadoop, but with all the buzz words about computing moving to the Cloud, this is worth reading about. Hadoop is based on Java and is owned by the Apache Software Foundation.
Hadoop uses 2 main concepts:
– HDFS or Hadoop Distributed File System
– Map Reduce engine
The first, HDFS allows for rack aware distributed files across a large cluster of networked computers, or nodes. In some cases these files are replicated on 2 or more computers, which eliminates the need for a RAID setup. This file system is also ideal for very large files.
The second, MapReduce allows for distributing and/or sharing the processor workload across this large cluster of networked computers, or nodes. Since HDFS is rack aware, MapReduce can put the workload on the same rack or box the data is on.
There are several large companies using Hadoop in one aspect; like Yahoo, Amazon, IBM, Google, Sun and many others.
If you want to learn more, and I feel it is worth a read if you are interested in Cloud computing.