Reinventing Business IntelligenceReinventing Business Intelligence

I've felt for quite a while that business intelligence tools are due for a revolution. But I've found the subject daunting to write about because -- well, because it's so multifaceted and big. So to break that logjam, here are some thoughts on the reinvention of business intelligence technology, with no pretense of being in any way comprehensive.

Natural language and classic science fiction

Actually, there's a pretty well-known example of BI near-perfection -- the Star Trek computers, usually voiced by the late Majel Barrett Roddenberry. They didn't have a big role in the recent movie, which was so fast-paced nobody had time to analyze very much, but were a big part of the Star Trek universe overall. Star Trek's computers integrated analytics, operations, and authentication, all with a great natural language/voice interface and visual displays. That example is at the heart of a 1998 article on natural language recognition I just re-posted.As for reality: For decades, dating back at least to Artificial Intelligence Corporation's Intellect, there have been offerings that provided "natural language" command, control, and query against otherwise fairly ordinary analytic tools. Such efforts have generally fizzled, for reasons outlined at the link above. Wolfram Alpha is the latest try; fortunately for its prospects, natural language is really only a small part of the Wolfram Alpha story.

A second theme has more recently emerged -- using text indexing to get at data more flexibly than a relational schema would normally allow, either by searching on data values themselves (stressed by Attivio) or more by searching on the definitions of pre-built reports (the Google OneBox story). SAP's Explorer is the latest such view, but I find Doug Henschen's skepticism about SAP Explorer more persuasive than Cindi Howson's cautiously favorable view. Partly that's because I know SAP (and Business Objects); partly it's because of difficulties such as those I already noted.

Flexibility and data exploration

It's a truism that each generation of dashboard-like technology fails because it's too inflexible. Users are shown the information that will provide them with the most insight. They appreciate it at first. But eventually it's old hat, and when they want to do something new, the baked-in data model doesn't support it.

The latest attempts to overcome this problem lie in two overlapping trends -- cool data exploration/visualization tools, and in-memory analytics. Tableau and Spotfire are known more for the former; hot BI vendor QlikTech is known for both. And many vendors -- established or otherwise -- are going to in-memory OLAP.

Collaboration and communication

The reason I'm finally buckling down and posting on this subject is the announcement of Google Wave, which I think foreshadows a revolution in communication and collaboration technology. Google Wave augurs two primary advances. First, it shows how to make email, instant messaging, microblogging, and so on much more useful. Second, Google Wave could evolve in a way that -- finally -- makes it truly practical for end-users to set up ad-hoc mini-portals that combine arbitrary URL-possessing resources, exposed to arbitrary workgroups of people.

If and when both of those promises are fulfilled, it will become vastly easier for people to reason together about analytic questions. That may take a little while, as Google Wave obviously wasn't designed with business intelligence in mind. But whether from Google or from a frightened Microsoft redoubling its SharePoint efforts, there's hope that we'll see a leap forward in general collaboration technology. And since BI vendors are doing a generally decent job of exposing queries, charts and so on as portlets, it seems likely that business intelligence will benefit from the collaboration arms race. That's important. The first time I heard that reporting was as important for communication as for analytics was from Pilot Software a quarter-century or so ago, and it's just as true now as it was then. In its first incarnations it probably will be a little too dumb for my tastes, focusing more on mindless reporting and same-old KPIs than on deeper analysis. Still, it's a move in a good direction.

Other directions

As I said at the beginning, I find it too daunting to try to cover all facets of this subject in one post. So I'll leave out, at a minimum:

plus some hobby horses you probably don't want to hear about anyway until I work out a better way of articulating my opinions.

But by all means please comment on what I've left out just as vigorously as on what I've included.

This post is just the first of many to come.I've felt for quite a while that business intelligence tools are due for a revolution... Here are some thoughts on the reinvention of BI, with no pretense of being in any way comprehensive...