Sybase IQ Upgrade Adds Support for In-Database AnalyticsSybase IQ Upgrade Adds Support for In-Database Analytics

Bolstering a growing trend, Sybase has released an upgrade of its fast-growing data warehousing product that enables customers to move analytic processing into the database for dramatically faster performance. Netezza and Teradata led the now-growing move into in-database analytic processing. But with this week's Sybase IQ 15.1 release, Sybase becomes the first to offer a column-oriented database that also supports in-database processing for data mining, predictive analysis, and other compute-intensive analytic functions. Besides faster processing, the vendor says the in-database approach brings increased accuracy.

"If you can process faster, you can run sophisticated analytics against more data," explains Lisa Hopkins, director of product marketing at Sybase. "If you are limited, because of the performance of your database platform, to using only six months' worth of data, versus three years' or five years' worth of data, the accuracy of those models will suffer."

Sybase IQ 15.1 offers three paths to in-database processing. First, the product's built-in library of statistical and data mining analytic functions has been extended. Second, standard-SQL OLAP extensions have been added for analysis on large data sets, including moving averages, ranking and correlations. Third, Sybase has launched a partner program that will enable statistical and data mining software from third-party providers to be plugged into the database.

"Vendors simply port their libraries to our API so those functions can be recognized by the database as if they are native to Sybase IQ," Hopkins explains.

The first vendor (and the only vendor to date) to join the partner program is Fuzzy Logix, with its DB Lytix library of descriptive statistics, Monte Carlo simulations and pattern recognition functions. Other "leaders and innovators" are expected to join the partner program, according to Hopkins.

Among other vendors promoting in-database analytic processing, Netezza launched its Netezza Developer Network in September 2007; it now has more than 100 partners, including SAS, SPSS, Catalina Marketing, Decision Intelligence, Epsilon, Ingrian and Tier-3. Teradata and SAS also announced an in-database processing partnership in 2007, and they released production-ready, jointly supported offerings this spring.

Greenplum and Aster Data Systems also support in-database processing, but Sybase asserts it's the only column-store database provider to make the move into in-database analytics. Unlike conventional relational databases that store data in rows, column-store databases store and query vertically, an approach that is ideal for OLAP analysis. If you want to explore a customer database for sales by ZIP code and product, for instance, a column-store product can interrogate only the ZIP code and product SKU columns rather than wading through all the names, addresses and other irrelevant data along each row. Queries run faster, because only the relevant data is accessed. In addition, compression is optimized – by up to 10 times over conventional databases -- because the data in each column is consistent (all ZIP codes, all product numbers and so on).

Having been on the market for more than 10 years, Sybase IQ is the most well-established column-store database, and the vendor claims to have more than 1,600 Sybase IQ customers. Competition has emerged in recent years from column-store database vendors including Vertica and ParAccel.

Other upgrades in the Sybase IQ 15.1 release include a new Eclipse-based integrated development environment (IDE) and built-in information lifecycle management capabilities. The IDE will help developers rapidly build their own analytical schemas, models and business intelligence applications to run on top of Sybase IQ.

"The toolkit includes a plug in for our PowerDesigner product in which you can do schema design, indexing, modeling, replication or reverse engineering from Oracle or DB2 into IQ," says Joydeep Das, a senior product manager. "Once that's done you can build your SQL code around the schema you developed with the aid of a built-in editor that fills in the required code automatically."

The IDE can also be used to tap the new information lifecycle management features, so developers can specify, for example, when to migrate aging data to lower-cost near-line and off-line storage platforms.

Sybase IQ 15.1 is slated to start shipping this week.