HP, Intel, Yahoo Join Government, Academia In Cloud Computing ResearchHP, Intel, Yahoo Join Government, Academia In Cloud Computing Research

Hewlett-Packard, Intel, and Yahoo on Tuesday said they have joined government and academia in launching a global, multi-data center test bed for experimentation and research in cloud computing, which many experts believe will be the dominant IT delivery model of the future.

The initiative aims at building a computing network comprised of six data centers spanning three continents. The idea is to have a large-scale platform for testing all technology -- hardware and software -- related to delivering application services over the Internet.

"This is a global collaboration that spans the industry, spans academia and government," Prith Banerjee, senior VP for research at HP, told reporters during a teleconference held by the three founding companies.

The other founders of the effort include the Infocomm Development Authority of Singapore, the University of Illinois at Urbana-Champaign, and the Karlsruhe Institute of Technology in Germany. The partnership with the University of Illinois also includes the National Science Foundation.

Each of the founding members will host a cloud-computing infrastructure largely based on HP computers and Intel processors. The infrastructure will include from 1,000 to 4,000 processor cores capable of supporting data-intensive research. The six facilities are up and running today in "bits and pieces" and are expected to be fully operational this year and accessible to researchers worldwide through a selection process.

How much access will be granted to researchers in other institutions and corporations isn't clear. Andrew Chien, VP of the Corporate Technology Group at Intel and director of company research, said the group was open to having other organizations join, but "obviously, there will have to be discussions."

"In general, the commitment is to exploit openness in the technology," Chien said.

That "openness" for now will revolve around the use of open source, distributed computing technology developed by the Apache "Hadoop" project, which is under the Apache Software Foundation. Other open-source technology that will be used by the group is called "Pig," a parallel programming language developed by Yahoo.

The implication is that if open-source technology is used in the cloud-computing project, than participants will have to give back at least some of what they develop to the open-source community. In addition, researchers from the academic institutions are expected to publish the results of their work in scientific publications.

For Yahoo, the latest initiative will be an extension of its work on delivering Internet services with Carnegie Mellon University. In that effort, Yahoo provided its M45 supercomputing cluster, which has about 4,000 processors, 3 terabytes of memory, 1.5 petabytes of disks, and a peak performance of more than 27 teraflops, or 27 trillion calculations per second.

The new research effort is only the latest partnership revolving around cloud computing. IBM and Google have been working together with several major universities in building a worldwide network of servers from which consumers and businesses would tap everything from online soccer schedules to advanced engineering applications. The companies announced in May that they planned to roll out the network over the next year. That network's infrastructure runs on Linux-based computers using Xen virtualization and Apache Hadoop, which happens to be an implementation of the Google File System.

The IBM-Google effort has a clear commercial side. IBM believes the cloud model will allow it to reach small and midsize companies around the world, which it says represent a $500 billion IT market that it has trouble serving profitably through the usual sales channels. Google and IBM could conceivably supply computer users -- both business and consumer -- with hosted offerings ranging from basic productivity software like word processing and calendaring to sophisticated management and security tools through IBM's Tivoli brand and Google's Postini unit.

Even though it is launching in concert with government and educational backing, the HP, Intel, and Yahoo collaboration is expected to eventually lead toward commercial markets as well.

Among the best examples today of commercial cloud computing is Amazon's Elastic Compute Cloud (E2C) and Simple Storage Service (S3). The services open Amazon's infrastructure to third parties that can build, deploy, and run application services on Amazon's platform for a fee.