BI Scorecard: Administration 2BI Scorecard: Administration 2

To date in this series, I've looked at many of the end-user features of BI suites that help users create powerful reports and analyze data to discover business opportunities. Administrative features might not garner the interest of the business users, yet they are equally important. The best deployments consider both end-user requirements and administrative issues of BI tools. Without considering both, companies can end up with either a tool that might look nice but requires significant IT resources to maintain or a system nobody uses.

Security

I confess: I hate BI security. It's not that I don't see the need for it; it's that I loathe keeping track of yet one more user ID and password! Nothing kills the success of a BI implementation quicker than when a BI user happily accesses a slick dashboard only to be plagued with "incorrect password" error messages when trying to refresh it. There's a lesson here: You can spend a significant amount of time selecting a BI tool, but if you don't spend an adequate amount of time planning security, the tool will be blamed for both breaches and login errors.

Security can be broken into two phases. The first is authentication — the validation of a user name and password. The second is authorization — what someone is allowed to access after authentication. The use of Lightweight Directory Access Protocol (LDAP) services promises to minimize the problem of multiple user IDs and passwords. In theory, a company will have one directory server with all employee usernames and passwords. All company systems — whether network, BI, or ERP — use the directory server for authentication. Time to change your password? It's changed in one place, for all corporate systems. New employee? It's added in one place, providing instantaneous access to everything the new hire needs to be productive from day one. Sound too good to be true? Probably. But we're getting there.

Currently, there's no clear standard for directory services. Sun's iPlanet, Microsoft Active Directory, and Novell's eDirectory are some of the leaders. BI vendors may support some or all of these. Microsoft Reporting Services, for example, supports only its own Active Directory out of the box. (Note: Microsoft claims it supports third-party directories through its APIs, but this capability isn't documented.) Business Objects only recently added LDAP support in version 6.1 (released September 2003).

TABLE 1 Scorecard comparing administration capabilities in several BI suites.

For both historical and practical reasons, most BI tools continue to have their own authentication mechanisms. If your company hasn't yet implemented a directory server, you need these mechanisms. If you do have a directory server, however, you want the BI tool to authenticate against it. For this reason, I haven't scored the item "Optional Proprietary Authentication" in Table 1: How you perceive this aspect is completely dependent upon how far along your company is in implementing a directory server.

Cognos has been the most aggressive in eliminating proprietary authentication, completely removing its own security with ReportNet. Authentication is done solely against third-party LDAP servers. Similarly, Microsoft Reporting Services doesn't have an additional reporting authentication database, but rather, authenticates directly against Windows. All the other BI products reviewed here continue to offer proprietary authentication for companies that don't yet have a directory server, and some require you to use the BI security. Cognos PowerPlay, for example, continues to require a separate namespace in a third-party LDAP.

Authorization is even messier than authentication. Within authorization, you may want to restrict users to certain business views or metadata layers (see part one, Query, in Resources), individual reports, software functionality, and data. Ideally, you want to define roles or groups of users so these authorizations can be maintained at a group level rather than for thousands of individual users. Herein lies a significant challenge: Even if you have an LDAP server for authentication, you may have to replicate all those individual user IDs within the BI tool for authorization purposes! With replication comes the risk that something — a user ID or password — will get out of sync. If, however, you can define groups within your directory server and the BI tool can read those groups, there's hope. Clearly, many BI vendors are moving in this direction, but most of them aren't there yet.

While it's clear that authorization for BI-specific capabilities must be defined in the BI tool itself, row-level security doesn't necessarily need to be. Row-level security refers to which rows of data any user or group can see. For example, within the subject area of sales, the European users may see sales only for countries in Europe; North American users may see sales for only the United States and Canada. These restrictions could be defined in the data source (either ERP or data warehouse) or in the BI tool. For operational reporting, you want the BI tool to leverage the row-level security already defined in the source system.

Metadata

Metadata integration holds out multiple promises. First, it should ease the administration of business views. Second, it should bring greater clarity to business users who will have consistent, accurate definitions for each metric's source, transformations, and calculations. At the moment, these are just promises.

In reality, each component within the BI infrastructure has its own metadata and uses it for different purposes. In part, this situation exists because metadata has been treated as proprietary to each component. It's also, in part, because each component has its own constituents. Business users who use the BI tool want business terminology. The IT department, which uses the ETL tool, wants to know the exact source system, table name, and field from which a data element originated. Assigning the data element a business term may or may not be important to the IT user.

From the BI suite perspective, you need to consider what metadata you want to share and between which components. Are you modeling in ERwin, for example, and want to use that model to build a Cognos ReportNet package? Or do you want to update a Business Objects universe based upon a data mart you've built with an ETL tool such as Informatica PowerMart? In the past, vendors within the BI infrastructure have taken a different approach to sharing their metadata, often providing proprietary APIs. As the Common Warehouse Metamodel (CWM) from the Object Management Group is gaining acceptance, vendors are increasingly supporting it. More recently, both Business Objects and Cognos have used metadata bridges provided by Meta Integration Technology Inc. (MITI) that are CWM compliant. Although not reviewed in depth as part of this series, SAS's Enterprise BI Server version 9 (released in March 2004) has also been innovative in pursuing metadata exchange. While these are good first steps to metadata exchange, in reality, they seem to help more with initial builds than with maintaining metadata over time. Metadata exchange seems to be a requirement that many companies list in their selection criteria, while in reality few actually use the limited capabilities currently available.

Metadata exchange can also be a challenge within a suite when the vendor has multiple tools. As shown in Table 1, a vendor that has multiple products may have administration per tool for security and metadata. For Business Objects (which acquired Crystal) and Hyperion (which acquired Brio) their recently acquired products take top priority on both companies' roadmaps. Even Cognos, which has both PowerPlay (for MOLAP) and ReportNet (for query and reporting), has long been challenged to synchronize metadata between its OLAP and reporting environments. With ReportNet 1.1 (released in February 2004), its metadata design tool, Framework Manager, can generate query definition files that an administrator uses to build cubes. You still, however, must further model the data using an OLAP-specific design tool, called Transformer.

Security and metadata maintenance pose less of a problem for ROLAP vendors MicroStrategy and Informatica, which have integrated toolsets and architectures.

FIGURE 1 Cognos Framework Manager can import metadata from other design tools and provide integrated impact analysis.

Metadata is important for a number of reasons, particularly in ensuring that users understand the data they're accessing. Oddly enough, even when BI vendors provide the ability to store help text or long descriptions for individual data elements, accessing this information is largely limited to query authors. As shown in Figure 1, Cognos Framework Manager nicely allows an administrator to provide additional help text on how "Revenue" is calculated and what it means. However, only query authors and not report consumers see this help text. The same limitation exists in the other BI tools reviewed here. So not only do BI suite vendors have a way to go with respect to sharing metadata between components, they have a lot of work ahead of them to make metadata useable by more than just IT.

Impact Analysis

Impact analysis — or the ability to know which reports are affected when you change or delete a data element — is critical. Impact analysis is relevant at different points in the BI architecture. If something changes in the source system, how do you know what to change in the business view, and ultimately, the reports? If you change something in the business view, is that change automatically propagated to the reports? At a minimum, an administrator needs the ability to identify interdependent BI elements within the BI suite. The scores in Table 1 reflect impact analysis capabilities within the BI suite and not across the total BI architecture.

Business Objects' ETL tool, Data Integrator, has good impact analysis with the metadata layer or universes. However, the impact analysis within its metadata design tool, Designer, is minimal. An administrator would need to run a separate report to identify report dependencies. As shown in Figure 1, Cognos Framework Manager shows you report dependencies within the design tool. MicroStrategy takes impact analysis one step further: When you try to delete an object, it actively warns you which reports are dependent upon it.

FIGURE 2 Informatica PowerAnalyzer dashboard lets administrators track BI usage.

Usage Monitoring

Failing to monitor usage of the BI system is like driving your car at night with the headlights and the dashboard turned off. At worst, you (or the server) will crash. At best, you'll run out of gas (or the queries will fail). As BI vendors increasingly target enterprisewide deployments, they're beginning to provide usage monitoring. Initially, vendors recorded BI activity in log files that were rarely useful for analyzing usage. The ideal is when the data is captured in a relational database, and the vendor provides prebuilt reports. Furthermore, administrators should be able to determine which activities they want to monitor, ranging from the number of logins down to which individual objects are accessed by whom.

MicroStrategy, with its server-centric architecture, has been a leader in providing tools to monitor BI usage. Business Objects introduced its Auditor product in 2001. Cognos, Crystal, and Informatica have all introduced capabilities in the last six months. As shown in Figure 2, Informatica PowerAnalyzer uses its dashboard capabilities to provide administrators a prebuilt dashboard that highlights current activity as well as historical trends such as longest running reports.

Hyperion Essbase and Microsoft still record information in log files only and provide no easy way for IT to interpret these logs. Although not reviewed in depth here, Hyperion Performance Suite provides some usage tracking stored in relational tables with prebuilt views.

Keep in mind that database monitoring and BI tool monitoring aren't the same thing. At the database level, you may be able to track how often the database field ORDER.QTY is accessed; within the BI application, you want to know which calculated metrics (such as "revenue," "average order amount," or "order quantity" that all access ORDER.QTY) users access most frequently, which standard reports, which viewing format, and peak load times. As I discussed in the Information Delivery segment (see Resources), the database load may be very different from the BI application server load.

Additional Requirements

In the first segment of this series, I discussed business views that help users build queries. The design tools for building the business views or OLAP cubes differ dramatically among the suites and there are a host of criteria to consider, such as version control, graphic representation of schemas, and wizards to build the models. Most design tools are client/server based, but a few such as Informatica and Hyperion Administration Services are Web-based. Also consider if the suite facilitates moving elements from development to test to production.

Next: Architecture and Wrap Up

Next is the seventh and final installment of this BI Scorecard series. I'll look at architectural differences between the suites. Having dug into the details of what these features mean and how the products differ, I'll finally answer the burning question: Who has the best BI suite for your company!

Cindi Howson is the president of ASK, a BI consultancy. She co-teaches The Data Warehouse Institute's "Evaluating BI Toolsets," publishes BI reviews at biscorecard.com, and is the author of Business Objects: The Complete Reference (McGraw-Hill Osborne Media, 2003).