Data Warehousing Takes Center StageData Warehousing Takes Center Stage

The story of the year in information management is clearly data warehouse scalability. Against a backdrop of about a dozen or so alternative database/data warehouse appliance vendors emerging over the past 18 months, Oracle and Microsoft both recently threw their hats into the ring of scale-out architecture. The two database giants are finally acknowledging that the scale-up approach that they have touted for years only goes so far. As data volumes and demands keep growing, that's often not far enough.

The question for practitioners is, which architecture and approach will meet your long-term data warehousing needs? As Richard Winter explains in this week's in-depth feature, proper data warehouse planning is not just a matter of estimating data volumes; you also have to assess query volumes, query complexity, the number of users, and data latency and availability demands. Only then can you truly prepare for the workloads likely to be experienced three to five years down the road.It's nice to know that Microsoft and Oracle have finally seen the light, but they still have a ways to go before they can prove themselves at the high end of the market. Microsoft is officially 18 to 24 months away from releasing its DATAllegro-based SQL Server upgrade, so figure 24 to 30 months before we see real proof of production performance.

Oracle, meanwhile, says it's taking orders and shipping product, but I'm guessing it will be several months before we really understand what we can expect from the HP-Oracle Database Machine and Exadata Storage Server. Sure, data capacity figures have been published, but what about the ability to handle high volumes of queries, complex queries, mixed loads or large numbers of users? Oracle has provided little official detail on pricing, architecture and performance characteristics.

Intelligent Enterprise contributor Curt Monash has gone deeper on the new HP-Oracle offerings than just about anybody, and he tells me now has in-depth briefings scheduled with Oracle. So stay tuned for more via his blog, but I trust many readers will want to see real-world case studies and talk to reference customers before plunking down six or seven figures.

My advice is to follow Richard Winter's prescription in planning your next data warehouse. If you're already running out of headroom or hitting a performance wall, I wouldn't wait for the promise of unknown solutions. As Winter points out, you shouldn't accept flimsy proofs that a solution will meet your requirements. Plenty of incumbents, from Teradata and IBM to Netezza, Sybase IQ and others, can point to plenty of successful deployments.The story of the year in information management is clearly data warehouse scalability. Against a backdrop of about a dozen or so alternative database/data warehouse appliance vendors emerging over the past 18 months, Oracle and Microsoft have finally acknowledged that the scale-up approach that they have long touted only goes so far... The question for practitioners is, which architecture and approach will meet your long-term needs?