Amazon.com Database To Cut Lag TimeAmazon.com Database To Cut Lag Time

Amazon.com Inc. faced a tricky problem last fall when its Marketplace unit, which sells used books and other products from third parties, was gaining popularity. As business increased, so did the amount of time it took IT staffers to update Web pages. The result was out-of-date product and inventory information--and unhappy customers.

The online retailer this week will disclose details of how it solved the problem. Using Excelon Corp.'s ObjectStore database as a cache-management system, Amazon cut update time for the Web pages from eight hours to two minutes.

"With the growth of this business, we needed a zero-latency solution," says Paul Kotas, engineering director for Amazon's Merchants@Group. Third-party retailers, ranging from Circuit City to Mom's Books, a used-book store in Elkview, W.Va., link to Amazon's main servers to provide pricing and inventory updates. Amazon had been using a slow batch-download process to send these changes from the servers to dozens of Web servers running Marketplace Web pages. "As the business grew, so did the cycle time," Kotas says. "We were unable to get it to scale to the levels that we needed it to."

That meant if the last used copy of a Harry Potter book were sold, Web sites couldn't reflect the change quickly. In some cases, customers put items in their shopping carts, only to find them unavailable at checkout because the checkout system relies on the main servers, rather than the Web servers.

In September, Amazon overhauled the system by installing ObjectStore. The database's distributed-caching architecture propagates each Web-page change from the main servers to the Web servers immediately, meaning Web-page data is updated every minute or two, Kotas says. The retailer chose Excelon after considering other approaches, including using in-memory relational databases or proprietary cache technology.

During the holiday shopping season, the Marketplace pages handled up to 20 million requests per hour, while pages underwent up to 1 million updates each day. There was a quantifiable reduction in the number of times customers tried to buy a product that was unavailable, but Kotas declined to provide figures.