Microsoft Hires Database Pioneer, Opens Database Development LabMicrosoft Hires Database Pioneer, Opens Database Development Lab

In an effort to forward fundamental database research and push the limits of database scalability, Microsoft has hired parallel database pioneer David DeWitt as the company's newest technical fellow and will open an advanced development laboratory in Madison, Wis.

The laboratory, called the Jim Gray Systems Lab, will perform fundamental research on databases and work closely with Microsoft Research and the company's development teams to do advanced development of database technologies that the company will roll out in the midterm.

DeWitt will head up the lab as part of his duties. "We have Microsoft Research, but the things David's embarking on are things that are much more applied in systems in the near-time," Quentin Clark, Microsoft's general manager for SQL Server, said in an interview. "Their primary output is not papers, it's leading in code."

The new laboratory will be staffed largely by University of Wisconsin-Madison graduate students who will have full access to SQL Server source code and Live data centers, among other Microsoft technologies, to help facilitate their research.

DeWitt, who is one of only a few dozen technical fellows at the company, has a long history in the database field. He began teaching database courses at the University of Wisconsin-Madison in the 1970s, where he would later help develop some of the first parallel databases, Direct and the Darpa-funded Gamma, research that formed the foundation for many of today's parallel database systems. He also created an early database performance benchmark known as the Wisconsin Benchmark and did early research on XML-based and object-oriented databases.

That expertise could prove invaluable to Microsoft as it builds dozens of massive, $500 million-plus data centers to handle an emerging array of Web service offerings from the company.

"If you look at my career, I've almost always been interested in ideas of scale," DeWitt said in an interview. "I am very interested in cloud computing. We know how to make parallel databases work on hundreds to the low thousands of computers, but we really don't know how to make it work on tens of thousands of computers." Before being hired, DeWitt had been part of an outside advisory board helping Microsoft's Live Platform Services group build scalable services.

DeWitt said Microsoft has generally given him carte blanche to focus his research and development wherever he wants. He's interested in a number of technologies and use cases, such as analytics to target online ads, query optimization at megascale, building database systems that can better exploit multicore computers, and expanding the types of data databases can handle at scale.

DeWitt joins several of his former students at Microsoft, including technical fellows Peter Spiro, who's working on future Microsoft storage platforms; Rakesh Agrawal, who heads up Microsoft Research's Search Labs; and Bob Herbert, who's said to be working on increasing scalability for a future version of SQL Server.

A good chunk of Microsoft's technical fellows come from database backgrounds. Dave Campbell, another technical fellow, is working to develop SQL Server Data Services for data storage and querying in the cloud. Campbell was one of the key developers who helped turn SQL Server from an also-ran into the top-selling database in the industry.

Jim Gray, for whom the new lab is named, was a Microsoft employee and Turing Award winner for his contributions to database research. Also a close friend and mentor to DeWitt, Gray went missing on a sailing trip off San Francisco in 2007 and was never found.