TPC Results: Not Crystal ClearTPC Results: Not Crystal Clear

IBM is boasting that its one of its x86 servers beat an HP x86 server on a TPC benchmark by 27 percent. But when you look closer you see that life is not that simple.IBM has announced that its IBM System x3850 X5 server did 27 percent better than the HP ProLiant DL580 G7 on a Transaction Processing Performance Council (TPC) benchmark. And when we look at the TPC site we can see that its true.

But we can also see that, cost-wise, HP performed the benchmarked tasks at a rate that was more than a fourth less per task.

IBM and HP both submitted results on the TPC Benchmark C (TPC-C) which simulates a complex online transaction processing environment for a fictional wholesale supplier, with simulated executives making five types of queries or entries against nine different tables. The result is given in transactions per minute, or tmpCs.

The IBM system handled 2,308,099 tmpCs. The HP system handled 1,807,347 tmpCs, so the IBM system was 27.7 percent faster. But the TPC also included a price per tmpC. That was 49 cents for the HP system, versus 64 cents for the IBM system. So IBM was 30.6 percent more expensive, per transaction, than the HP system.

Oddly, the two considerations almost exactly balance out.

Meanwhile, there are considerations other than speed or price. For instance, the IBM system was running the SUSE Linux Enterprise Server 11 SP1, while the HP system was running Microsoft Windows Server 2008 R2 Enterprise Edition. The choice from a buyer's perspective could hinge more on in-house expertise.

Meanwhile, TPC has added an energy metric. The HP system clocked 2.46 watts per thousand tmpCs. IBM did not submit a result. The HP result was about half of its two previous submissions, which shows a trend in the right direction. IBM has never submitted an energy metric, and the only ones on the chart (which goes back a decade) are from HP.

With system prices falling and throughput rising-and energy costs becoming more paramount-the energy metric may become the most salient thing on the TPC results chart. Let's see if vendors start filling in that column.