Event Processing: The Next Disruptive TechnologyEvent Processing: The Next Disruptive Technology

Complex event processing (CEP) technology is aimed at many of the same challenges as conventional BI technology, it's just that the frame of reference is real-time analysis rather than a separate reporting loop built on historical data. Thus, CEP is another threat to BI as we know it, and it's pretty apparent that this will be one of the next competitive battlegrounds for the big infrastructure players.

As I detailed in this week's top story, CEP spots patterns in high-volume data streams while they're still streaming, rather than after the fact. Latency is measured in milliseconds or microseconds, and the volumes of data can exceed 50,000 messages per second. You build models or queries and the CEP system spots the related patterns as they emerge. You can respond while the activity is still in process rather than after the fact."As we're moving toward real-time scenarios, this is going to take away business from BI vendors," says Dr. Peter KÜrpick, the Sofware AG executive board member in charge of the company's just-formed webMethods business line. "People are tired of this big data warehouses, slicing and dicing and coming out with reports next Monday when you need the insight the right away. The old approaches aren't going to go away, but parts of the analysis are going to move upstream into real-time scenarios."

While CEP is currently dominated by a handful of specialty players in the BI vendor mold (StreamBase, Coral8, Progress (Apama), etc.), it's pretty clear that the infrastructure vendors are going to be all over this technology (through acquisition or organic development). IBM and Oracle are still waiting in the wings, but TIBCO entered the market in 2004 and BEA threw its hat in the ring last month announcing the WebLogic Event Server. Software AG, which just acquired webMethods to bolster its infrastructure play, also has its sights on CEP. "We have that in the research labs… and I'm not worried that we can't easily come up with CEP," says KÜrpick.

Not so fast, and not so easy, say vendors that have been pioneering CEP for five years or more. Taking a swipe at BEA's recent market entry, John Morrell, director of product marketing at Coral8, says, "all BEA did was OEM an open-source product called Esper and put it inside their product. We've never lost a deal to Esper, and when you go down to Wall Street, you won't find a TIBCO or BEA [enterprise service] bus because they're way too slow for those demands."

BEA's response? "Early on in WebLogic Event Server development, BEA purchased a perpetual source code license to Esper and used it as a starting point, but we've modified it heavily and built significant functionality around it," says Mike Piech, BEA's senior director of product marketing. "Thus, [Esper] is not a major component of WebLogic Event Server, nor is it maintained intact so as to reincorporate updates from the open-source version." In other words, BEA controls its own product destiny in CEP.

Morrell's larger point is that some CEP vendors talk microseconds and hundreds of thousands of transactions per second while others talk milliseconds and tens of thousands of transactions. That difference may be crucial on Wall Street and in the Intelligence Community, but it's splitting hairs where the broad market potential for CEP applications is concerned. In manufacturing, shipping, RFID, clickstream analysis and many other mainstream applications, anything CEP looks like warp speed. If you read about what practitioners such as Con-Way and eCourier are doing, you quickly recognize that CEP will be a game-changing technology.Complex event processing (CEP) technology is aimed at many of the same challenges as conventional BI technology, it's just that the frame of reference is real-time analysis rather than a separate reporting loop built on historical data. Thus, CEP is another threat to BI as we know it, and it's pretty apparent that this will be one of the next competitive battlegrounds for the big infrastructure players.