Will Linux Hold The High-Performance Ground?Will Linux Hold The High-Performance Ground?

Linux may still have an uphill battle against Windows on the enterprise desktop. It's a different story, however, in the market for high-performance computing (HPC) systems, where Linux already enjoys an established presence and a solid technical reputation.

Linux is likely to remain a strong contender in the HPC market, just as it has built a lasting presence in the enterprise server and niche computing markets in recent years. As the HPC market continues to grow, however, vendors supporting other platforms are increasingly likely to take notice. The question is, will Linux continue to dominate the HPC market as it has in the past, or will competitors swoop in to fight for a piece of the action?

A Market Worth Fighting For
According to Yankee Group analyst Laura Didio, the HPC market is already worth billions of dollars in potential revenue, making it a lucrative target for major system vendors. In addition, Didio predicts the market will grow by 10-15 percent over the next two years, a rate she said would be even higher if not for the lasting effects of the last economic downturn. "Customers are still being cautious with cash," Didio said.

IDC also paints a rosy picture of the HPC market: The firm says sales revenue from technical high-performance systems will increase by 6.1 percent through 2007 and revenue from capacity systems will grow seven percent to $6.3 billion by 2007. IDC also predicts that Linux systems will shine in the cluster computing market, earning almost a billion dollars in revenue in the third quarter of 2004--a 40 percent increase over the previous quarter.

Bruce Toal, director of product and technology marketing for HP's high performance computing division, said Linux clearly plays an important role in this space. "Linux is growing faster in HPC than the commercial area," Toal said. "If you look at IDC growth rates, and I would think we have comparable experience with that, it's above and beyond what we find in the commercial Unix space."

Familiarity Breeds Success
According to Gordon Haff, an analyst for Illuminata, Linux enjoyed a head start in the HPC market due to the fact that many of the IT professionals responsible for high-performance systems already have strong Unix backgrounds. "If you know Unix, you are on your way to knowing Linux because it is a form of Unix," he said.

Haff also cited the availability of cheap commodity hardware as another big advantage for Linux in HPC environments. He said that while some high performance systems, such as offerings from Cray, use costly proprietary hardware, most Linux-based HPC systems today run on generic boxes. "If you look at HPC clusters in general, most are cheap two-way boxes hooked together with Ethernet running Linux," Haff said.

Don Becker, CTO and founder of Scyld Software, makers of the Scyld Beowulf cluster operating system, said Linux also does a good job adhering to HPC standards such as the Open Source Grid Architecture, a set of open standards covering a variety of security, management, communication and other key grid infrastructure services. "Since grid computing is not a big portion of the computer space, you cannot expect all tools will work on the grid," Becker said. "In order to do that, you need to modify these tools, and Linux is an excellent platform because it's open and users are more likely to share results and keep utilities consistent."

Becker added that open standards make it easier to modify software not necessarily designed to work in the HPC environment. "Linux is a good choice for grids and clusters. It's possible to modify software not intended to work on a grid and make it part of an interactive environment," he said. "Open source allows you to take tools not designed for use on the grid and cluster and verify they would work in that environment, or modify them if there are subtle or deep flaws."

Paola Lubet, vice president of marketing for Qlusters, a provider of enterprise software to manage Linux datacenters, also cited the advantages Linux provides through its use of open standards and commodity hardware. "There is cost/performance associated with Linux that is powerful," she said. "One of the biggest advantages of standardization, from the customer perspective, is it removes vendor lock-in. It doesn't force you to buy one type of machine."

Lubet cited one of her firm's customers as an example of this process at work. "There is a large IT manufacturer that makes routers," she said. "They have about 3,000 servers that are on Linux. They service 24 business units. Server utilization is currently at 15 percent. There are ways if they could insulate applications from hardware they could do far more with these 3,000 servers, and postpone acquisition of another 1,000 servers in the next nine months."

Competition Comes Calling
In a March, 2003 Gartner research note titled The Penguin and the Grid: Impact of Linux on Grid Computing, analyst George Weiss observed that today's high-performance systems involve a growing number of commercial applications alongside existing scientific inquiry and discovery applications. System architects, for example, are weighing high-performance grid solutions as more efficient and reliable alternatives to current .Net- and J2EE-based Web services applications. The question is whether these IT professionals will adopt Linux-based HPC solutions, or will they use high-performance versions of more familiar platforms?

According to Bruce Toal, HP has already acknowledged this possibility by introducing a Windows-based option for its high-performance computing solutions. Although the technology is still in the early stages of market development, Toal said it has been successful in environments where customers are predominately using Windows and don't want to stray from that platform.

Haff says Windows has not taken hold in the high performance market, however, because it does not have the support of the HPC community. "Windows has not done well at all in HPC, although Microsoft continues to try to do things about it," Haff said. "It's to some degree a flip side of Linux's success: [Windows is] not a Unix operating system, and many people in those environments are philosophically opposed to Windows because they have grown up with Unix."

Gartner said the HPC space is too immature to predict how Linux will do, but the firm does believe Linux will continue to dominate the scientific HPC market. The rest, however, is hard to predict: "By 2008, more than 70 percent of scientific grids built on server architectures will tie grid protocols to a Linux operating system," the Gartner report stated. "In the commercial and service provider sectors, the most important technology underpinning grid deployments will involve developments associated with a Web services environment, such as the ongoing evolution of Java 2 Platform, Enterprise Edition or Microsoft .NET services for Windows.

"Both grid computing and Linux are too immature for us to forecast that Linux will dominate in commercial grid applications," the report concluded. "Thus, by 2008, grid computing will be used tactically by 20 percent of enterprises--not enough to make Linux pervasive in commercial enterprise grid applications."

The bottom line is that Linux is likely to remain a major force in the scientific and technical HPC market, enjoying widespread support among IT professionals, the benefits of low-cost commodity hardware, and its embrace of open standards. Although those advantages may not be enough for Linux to dominate a new generation of high-performance business and commercial solutions, it seems likely to have at least a fighting chance--and perhaps considerably more.