CEP Moves From Wall Street To Main StreetCEP Moves From Wall Street To Main Street

Amid the economic turmoil that's gripped Wall Street the past few months, Adam Honoré, a senior analyst at Aite Group who specializes in brokerage and financial services technology, declares that "human factors such as poor risk management, not computerized trading," are primarily responsible for the market's current volatility.Wall Street firms have used trading algorithms to search patterns in data for buying or selling opportunities for decades but, Honoré maintains, the present instability is because of the risk posed by collateralized debt obligations and other unregulated asset-backed security and structured credit products. "People should have better insight into the risks they are taking," he says, citing new complex event processing (CEP) technology that can solve many of the risk management and compliance problems that financial firms currently face. According to Honoré, CEP engines can remove much of the risk in the market because they can absorb a wide range of information and normalize, aggregate, and analyze it in real-time. This can provide greater transparency and visibility to market information, not only for regulatory agencies but also for enterprises that want to better manage risk and stop regulatory infractions before the regulator spots them.

Once available only to big financial institutions and government agencies that could afford custom development projects, real-time data analysis is finding its way into the hands of more IT enterprises with the emergence of off-the-shelf CEP products from a handful of pure-play CEP vendors such Progress Apama, StreamBase, Aleri, and Coral8 that first made names for themselves in capital markets. Much of the technology isn't yet mature and major SOA and middleware vendors such as Tibco, IBM (who recently acquired CEP vendor AptSoft), and Oracle (through its purchase of BEA) are beefing up their CEP offerings through in-house development and acquisitions. As you'll see in this article and a series of upcoming rolling reviews by information's Cristian Sturek, who will take several CEP products for test drives, CEP is gaining more mainstream momentum and expanding outside industry verticals such as finance, telecommunications, transportation, manufacturing, and the military, where CEP established its bailiwick. Open-source projects such as NEsper for .NET and Esper for Java, as well as "try-before-you-buy" offerings from numerous CEP vendors, even makes this technology accessible to small and midsized businesses that want to develop whiz-bang real-time customer tracking applications.

IDC analyst Maureen Fleming pegs the CEP market at $150 million in 2008, and projects yearly growth to exceed 50% for at least 5 years, which would make CEP one of the fastest-growing segments of the middleware market. She qualifies her projection by saying it's not clear what the impact of restructuring of U.S. financial markets will have on CEP growth. "The question is how slow financial markets will be in figuring out how to re-build themselves, along with how much this will hurt the economy overall," says Fleming.

John Bates, who is a CEP industry pioneer and founder of market leader Progress Apama, responds, "We've seen a big growth in capital markets in the last 18 months -- despite the market conditions. While we're concerned about the current turmoil, firms still have to make money and differentiate so we believe that uptake for Apama's products in capital markets will continue to grow." Echoing Honoré, Bates sees a role for CEP in regulating chaotic financial markets and points out that the U.K.'s Financial Services Authority is using Progress Apama's CEP platform to monitor stock market transactions and detect insider trading and other market abuses as they occur. "The power of CEP is to monitor events, discern salient patterns, and take responding action. So if there is risk-inducing behavior in a particular set of market conditions, CEP can help detect it by monitoring the market for conditions that suggest that such things may go awry or are at an increased risk to do so." Adds Bates, "But a key corollary to this capability is to do it quickly and to do it with a light touch -- in other words, to do it in a fashion that does not have to be obtrusive to the normal course of business."

Early Days For CEP
Rourke McNamara, director of SOA Products at Tibco, who offers a competing BusinessEvents product, says "CEP is really in its early days. The advantage of CEP is that it allows a company to harness the know-how of the smartest 10% of its employees and automatically scale their best practices across the entire organization." He compares the adoption rate of CEP to that of Enterprise Service Buses (ESBs), which many companies use to implement SOA. "Now, just about everyone has an ESB -- the great majority of which are standards-based and very flexible," McNamara says. "We're not there yet with CEP engines, but we're getting there."

"The inexorable growth of Web-based e-commerce is the dominant trend fueling the adoption of real-time, on-demand, just-in-time technologies across many verticals," says Forester analyst James Kobielus. "Real-time customer relationship management, marketing campaign management, customer service, closed-loop business-process optimization, help desk, Web experience management, online fraud detection and prevention, recommendation-based cross-sell/up-sell -- these are among the leading "live" online, event-driven, analytics-rich, interactive transactional environments that thrive on real-time response."

CEP vendors, who also call their technology event stream processing or just plain event processing, say the spreading adoption of CEP in industries other than finance is fueled by some of the same market drivers that have made the technology invaluable in capital markets -- firms seeking to gain a competitive edge in analyzing skyrocketing volumes of customer data. "This is especially important in a business environment where, increasingly, 'a customer is a click.' " says John Morrell, director of product marketing at Coral8, a leading CEP vendor. Morrell cites the example of Sallie Mae, the nation's largest student lender, where 90% of Sallie Mae's loan applications over the last five years have shifted away from paper-based applications to Web-based forms, which lead Sallie Mae into online CEP monitoring systems to help it manage its sudden shift into an online business.

"Complex event processing has fundamentally changed the way we do business," says Jo Lee Hayes, VP of enterprise technologies at Sallie Mae, which was once a quasi-government entity but is now a private company. She says that in a little over 9 months, Sallie Mae's IT staff was able to construct 35 fraud-detection patterns that Coral8 can watch for, as well as create modules that can spot when the loan application process is moving too slowly and reach out to the customer to ensure their student loans are processed correctly. Using CEP to track online applications, and when an application process seems stalled, following through with a multichannel strategy to reach out to the customer, is the wave of the future, agrees Honoré. "Like most people these days, I tear up the credit card offers that are sent to my house via mass mailings from old-school business intelligence apps." He elaborates, "I have multiple relationships with my bank and other financial service providers -- say when I walk into a bank branch or use an ATM card from one or the other -- and new multichannel CEP applications allow these companies to contact me wherever and whenever our paths cross."

Harrah's Entertainment, the $10.8 billion resort and gaming company, is another company developing CEP systems that have the ability to suggest services to customers based on transactional information received in real time, rather than waiting until the customer has checked out. During a discussion at this year's information 500 Conference with information VP and editor-in-chief Rob Preston, CEO Gary Loveman said Harrah's has a history of analyzing customer transactions to determine patterns of spending and better anticipate and influence spending. A former Harvard Business School professor, Loveman is the author of an influential May 2003 Harvard Business Review article "Diamonds in the Data Mine" that described Harrah's approach to using real-time analytics at the point of sale much like an 'old-time casino boss.' Loveman says Harrah's has the capability to monitor the gaming habits of a regular customer who spends $1,000 or so per month at one of Harrah's 50 casinos, and to signal shift workers to interrupt a customer who has lost that amount of money by offering her a special deal such as a steak dinner, a free night in the hotel, or tickets to a stage play. In the HBR article, Loveman boasts that Harrah's has come out on top in the casino wars precisely because of its ability to "run these kinds of marketing experiments, and then use the results to develop and implement finely tuned marketing and service-delivery strategies that keep Harrah's customers coming back."

All of which leads to the inescapable conclusion that savvier stock brokers need to steal a page or two from Harrah's playbook as they try to re-build customer confidence and bring disappointed investors back to the game in the wake of the events that continue to unfold on Wall Street.