Amazon Debuts Public Data CloudAmazon Debuts Public Data Cloud

Amazon.com has introduced a new service in which it hosts large data sets -- economic, demographic, scientific, and medical data, for example -- that are open for anyone to access. It's an interesting proposal, but one that casts Amazon in the potentially difficult role of having to be an information gatekeeper.The new offering, hosted on Amazon's recently introduced Elastic Block Storage, is called Amazon Web Services (AWS) Hosted Public Data Sets. Amazon first described the service a few weeks ago; today marks the official launch.

So far, Amazon has assembled a half dozen or so data sets from a variety of sources, including a repository of 3-D chemical structures, and census, labor, transportation, and economic stats from the U.S. government. It's looking to expand the number and types of data sets hosted on AWS.

Here's how it works: Data sets are hosted for free on Amazon's Elastic Block Storage, and users then use the data set to create their own volume, which they can modify and manipulate. The catch is that users need to have an EC2 account, and they'll pay for any compute and storage resources consumed in the process. Amazon says most data sets range from 1 GB to 1 TB in size; it can accommodate larger data sets by divvying them into 1 TB volumes.

What kind of data qualifies? Amazon says, vaguely, that the data must be "useful and interesting" and that the person or organization sharing it must have the right to do so. That seems straightforward enough with data made available by the feds, but it could get dicey depending on the nature of the data or its source. For example, many kinds of health care, financial, and demographic information may have privacy or governance implications, and there could be copyright issues with other kinds of content.

When I asked Amazon VP Adam Selipsky where the company would draw the line between what data gets accepted as a public data set and what doesn't, he admitted that Amazon doesn't have clear-cut guidelines. "We use judgment there," he said. It will be interesting to see how well Amazon adapts to this role as information steward and how long it takes before someone cries foul.