Gartner Business Intelligence Magic Quadrant Rankles SAPGartner Business Intelligence Magic Quadrant Rankles SAP

When vendors make it into the top-right "leaders" quadrant of a Gartner Magic Quadrant (MQ) report, they generally don't complain. But SAP isn't thrilled with the "ability to execute" positioning of SAP BusinessObjects in the 2010 Gartner MQ for Business Intelligence Platforms, which was released early this week.

"If you look at the results, it's nonintuitive and nonsensical that we would have less ability to execute, than, for example, Microstrategy," said Franz Aman, SAP's vice president, intelligence platform product marketing.

Never mind that SAP BusinessObjects ranked second on the "completeness of vision" axis (after IBM Cognos). SAP was so upset with the ability to execute ranking (where it was behind Oracle, Microsoft, IBM, SAS and Microstrategy, in that order), that it went to the extreme of commissioning a blind survey of 24 CIOs, conducted by the Gerson Lehrman Group, that shows a higher ranking for SAP Business Objects based on an open-ended, respondent-defined interpretation of "ability to execute."

"We basically ranked in the ballpark of Oracle and IBM in the blind study, and the other vendors were in far distant spots," Aman said.

Gartner says it defines and measures ability to execute across all MQ reports in terms of seven dimensions: product and service competitiveness, vendor viability, sales execution and pricing, market responsiveness, market execution, customer experience and operations. The weighting on each of these seven dimensions varies from year to year based on customer inquiries and market changes, said Rita Sallam, the primary author of this year's BI MQ. She defended the ability to execute rankings saying they were based in large part on nearly 900 customer interviews, including 91 SAP BusinessObjects customers, most of whom were referred by SAP itself. As stated in the MQ report, both IBM Cognos and SAP BusinessObjects lagged in the survey's customer experience ratings, which Gartner interpreted as a result of post-acquisition challenges.

"When there's an acquisition, there's often a lot of turmoil," Sallam said. "SAP, in particular, had significant challenges when it changed over [BusinessObjects] support [to SAP]... IBM cut over Cognos support to IBM in 2009, and it also realigned sales territories and changed the way Cognos customers buy products. Change by itself can have a negative impact on customer perceptions."

Oracle saw a similar decline it survey-based customer experience ratings in 2008 after its Hyperion acquisition, Sallam said, "but it now seems like they are beyond that transition and they had significantly better scores for 2009 than in 2008." Aman said SAP BusinessObjects has gone to great lengths to improve service support, and he said he believes perception has yet to catch up with reality. Citing research that shows that it takes up to seven positive interactions to overcome one bad customer service experience, Aman said survey's such as Gartner's are "a rear-view mirror."

"We're now measuring customer feedback after every support interaction, and we've shown continuous improvement in speed and quality of response over the last year," he said.

Recapping the MQ leaders quadrant, Oracle and Microsoft led on ability to execute while IBM Cognos and SAP BusinessObjects led on completeness of vision. The three other vendors in the quadrant where SAS, MicroStrategy and Information Builders, all of which issued the usual congratulatory press releases highlighting their achievements (SAS even included free access to Gartner's MQ report). QlikTech, TIBCO Spotfire and Tableau where in the "challengers" quadrant, reflecting high marks for ability to execute.

In its extensive commentary on each vendor, the MQ report on BI included more than a few "cautions" about each vendor. For example, the report cited "emerging signals that IBM's ability to continue to sell its BI platform into firms with application stack-centric sourcing policies may be limited... IBM does not have business applications and does not share the same operational BI vision or capabilities of Oracle and SAP."

The report cautioned that In comparison with competitors, Microsoft's BI vision "remains more narrow and focused on developers, reporting functionality and Microsoft Excel, coupled with Microsoft SharePoint functionality." And on Oracle, the report observed that "Oracle lags behind the competition in introducing new and innovative solutions, such as the ability to integrate interactive visualization, search and collaboration." It also noted Oracle's lack of an in-memory architecture for OLAP analysis, something most competitors have introduced.

Actuate was the lone open-source BI vendor in the Gartner MQ for BI. Sallam said Jaspersoft and Pentaho met Gartner's threshold of having more than $15 million in revenue, but they were not included because fewer than 30 customers responded to the qualifying customer survey. Sallam said the popularity of the open-source BIRT standard developed by Acutate led to higher response levels for that vendor.