Teradata to SAP: Our Memory Is SmarterTeradata to SAP: Our Memory Is Smarter

Teradata on Wednesday introduced a new in-memory database feature that it says offers a practical approach to exploiting random-access memory (RAM). The benefit, it says, is delivering the ultimate in query performance as needed without what it calls the "unnecessary and impractical" cost of an all-in-memory approach.

Teradata Intelligent Memory, the vendor's new database feature, is clearly being compared to SAP's Hana in-memory database. Teradata and SAP aren't exactly head-to-head competitors, with SAP primarily being an applications vendor and Teradata focusing exclusively on data warehousing. But with SAP thumping its chest about being the industry's fastest-growing database vendor, market leadership, mind share and industry buzz are clearly at stake.

Teradata Intelligent Memory is an extension of the vendor's years-old Teradata Virtual Storage software, which constantly monitors which data is being queried most often and then automatically moves that data to the fastest storage tiers available. Before Wednesday's announcement the choice was hot (on solid state disks or flash storage), warm (on fast hard drive disks) or cold (on slow hard drives). When different disk drives speeds aren't available, the software moves warm data to the outer tracks of the drives, where faster rotation delivers quicker access, and cold data to the inner tracks.

[ Want to hear about SAP's new Hana deployment option? Read SAP Cloud Push Starts With Hana. ]

Solid state disks and flash storage are certainly fast, but RAM is orders of magnitude faster, so Teradata Intelligent Memory adds "super hot" to the hot/warm/cold access-speed strategy by supporting an extended RAM storage layer. Because it's a database feature, currently in beta but set for an upcoming database release, the optional feature will be compatible with all existing Teradata deployments.

"We're delivering the performance of in-memory in a way that makes sense for data warehousing and in a package that's easy for our customers to implement," Scott Gnau, president of Teradata Labs, told information

Teradata previously used (and still uses) the RAM available in its servers for caching, but Teradata Intelligent Memory creates a dedicated memory space where information that's in the most demand can be pinned for super-fast access. The size of the memory available is configurable by the customer and depends on the RAM available on servers.

Customers with business-to-consumer Web applications and mobile applications are likely to be the first adopters of Teradata Intelligent Memory, Gnau predicted, because the in-memory advantage will ensure fast and consistent performance. These applications also call for an automated, dynamic approach because yesterday's super hot data (appropriate for RAM) is likely to quickly become warm data better suited to SSD or fast disk drives.

SAP has argued that the cost of RAM has plummeted, making an all-RAM approach practical and affordable. Gnau counters that prices of all other storage options are also falling, so RAM still stands out as the most expensive option.

"I've never lived in a world where businesses buy something that's more expensive than another option just because it's cheap," said Gnau. "We're also in a world where data volumes are growing faster than RAM prices are declining, so cheap RAM, cheaper solid-state and cheapest-option rotational drives are still relevant options."

Are you better off using all RAM or a hybrid approach? There are too many variables to discount either approach, and keep in mind that the Teradata database and SAP Hana aren't used for the same purpose. SAP is pushing Hana as a transactional database for running applications, such as its ERP and CRM systems, as well as an analytic database for running SAP Business Warehouse (BW). But where BW is a data warehouse used mostly in connection with SAP application data, Teradata is invariably an enterprise data warehouse platform used for a much broader swath of historical information.

Even Hana champions Hasso Plattner, chairman of SAP, and Vishal Sikka, a member of the company's executive board, have acknowledged that truly cold data -- infrequently accessed historical information in a warehouse-- might be better managed by conventional databases and disk drives. In this scenario they tout the Sybase IQ database, which is the closest thing SAP has to a head-on Teradata competitor, though IQ is most often used for focused data marts rather than enterprise data warehouses (EDWs).

Teradata isn't the first data warehousing specialist to exploit RAM. Kognitio, for one, has been supporting high-memory deployments for years. Kognitio customer Tivo Research Analytics, for example, pins as much as a third of its 15-terabyte data store in RAM so advertisers can quickly spot which televisions shows are most watched by the biggest buyers of specific products.

But Kognitio is a niche player compared to Teradata, a decades-old company that has a who's who list of prominent customers with massive petabyte-scale and 100-terabyte-plus deployments. What's more, Teradata is introducing a much more automated, dynamic and nuanced (super-hot-to-cold) approach than Kognitio offers. One other contrast: Kognitio sells a database whereas Teradata has a complete range of hardware platforms -- from large-scale EDWs to speed- and archive-focused appliances. In short, this is a pretty big deal for Teradata and its customers.

It's obvious even to SAP that high-scale data warehouses don't belong entirely in memory, but give it credit for sparking a lot of interest in in-memory performance. Teradata's press releases and executive comments make it clear that the buzz stoked by SAP helped spark the development of Teradata Intelligent Memory. Teradata isn't exactly leading the wave with its announcement, but it came up with a practical and sensible approach to in-memory performance for high-scale data warehousing.