VMware's Tool Play Doesn't Scare RivalsVMware's Tool Play Doesn't Scare Rivals

VMware Vs. Microsoft: 8 Cloud Battle Lines

VMware is rushing in where cooler heads have been wary: the rapid data accumulation field of server log file analysis. VMware's new vCenter Log Insight became available July 11 as the latest addition to VMware's push into data center management.

It's sure to meet resistance from the well-established vendors already at work on log file management. They include the widely used Splunk system, SolarWinds' Log and Event Manager, TripWire Log Management, and Sumo Logic. But it seems clear there will be no future software-defined data center without automated server log file analysis, and VMware is volunteering to take on the role from the vantage point of virtualized operations. Splunk has returned the compliment by offering Splunk for VMware, a product that collects log file data on the VMware environment.

VMware, like other vendors in the field, not only collects data but reviews it in real time, sorting out what indicates normal operations and focusing on abnormalities or patterns that might indicate the start of trouble. To do this for many servers with a low latency is no small task.

The typical vSphere server log file in the VMware environment generates at least 250 MB of data a day; a Microsoft Exchange Server will generate a GB. Keeping up with large numbers of these servers, along with virtual machine hosts, firewalls, Web servers, load balancers and database servers becomes a big data problem.

[ Want to learn more about VMware's entry into log file management? See VMware Tackles Server Log Data Pain. ]

Nevertheless, Splunk welcomed VMware into the marketplace, saying the addition of VMware's Log Insight "really validates our market," said VP Sanjay Mehta. Splunk better collects and analyzes machine data from a variety of systems in the data center, including alerts and click-stream data, not just virtualized server log files. It also offers specialized products looking at Hadoop or relational database systems.

"Data is only as good as the value that can be derived from it. Splunk allows many users within an enterprise to extract tremendous business value from their machine data," Mehta said in response to an email query about VMware's entry. "Imitation is the sincerest form of flattery," he added as a parting shot. VMware is listed as a Splunk customer on its website.

Sumo Logic CEO Vance Loiselle said in an interview that it's important to know what to do with server log file information, once you've got it. He called the VMware offering "very limited" and said it's an on-premises product aimed squarely at the existing VMware customer base, not the larger data center, operations management market.

Like Sumo Logic's server log file data collection, VMware's is searchable. But Loiselle adds, "For search to be effective, you need to know what you're searching for." Sumo Logic, in the field since 2010, has an edge in that regard, he said. Splunk started shipping its product in 2006.

Sumo Logic has applied for a patent on its LogReduce sorting and analysis system, which reduces "millions of log lines into a handful of human digestible patterns," and allows its users to decipher more useful results. It's looking at Cisco network switch logs as well as server logs, for example, and sorting out the sequences and patterns that lead to system errors. Its LogReduce analysis system is looking for patterns that indicate trouble is starting to happen, not after it's already underway.

Loiselle said his firm "took a significant amount of time, two years, to develop LogReduce" and has spent the last 18 months refining what it can do. It's offered as an online service, and will analyze 500 messages a day for new prospects for free.

"The Splunk team has capabilities that surpass VMware," said Loiselle, but both Splunk and VMware are limited to on-premises installed products, unlike Sumo Logic's service.

A Splunk spokesman responded that it also offers a cloud-based version, called Splunk Storm.

Use of such products is likely to increase our knowledge of how to manage servers before they go down and how to keep an automated data center out of trouble. But server log file management is a field that is just starting to get sophisticated knowledge of what it can do. It may one day be a major contributor to continuous operations in the software-defined data center; right now it's a field of fierce competitive efforts.