Nirvanix Shutdown: HP Steps In To Move DataNirvanix Shutdown: HP Steps In To Move Data

Years ago, Velocity Network Solutions helped one of its clients put data in the Nirvanix storage service. When Nirvanix declared bankruptcy and announced it was shutting down Oct. 15, Velocity wanted to help its client exit. But it needed assistance to get the data out.

Nirvanix initially gave its partners, including IT service providers, such as Velocity, only two weeks' notice. "The large size of Nirvanix, along with its rapid cease of support, makes this closure one for the books," wrote a Velocity staffer posting to the company's blog Sept. 19.

Moving terabytes or petabytes of data out of a cloud service takes time, possibly more time than Nirvanix seemed to be willing to allow. "An extremely short migration window combined with a massive amount of client data is enough to cause an IT consultant to keel over from a heart attack," the blogger wrote.

Velocity put the data in Nirvanix on behalf of one of its large clients, who must remain unnamed, according to Velocity CEO Thad Gerber in an interview. Velocity first realized the trouble both it and its client were in when Nirvanix informed Velocity that it could continue to load data into its service only for another 24 hours -- the "get lost" warning for new data. After that, all uploads would be refused. And the option to download would be shut off in two weeks, Gerber said.

[ Want to learn more about the Nirvanix shutdown? See Nirvanix Declares Bankruptcy: Customer Data Pours Out. ]

The client had both normal archival data and time-stamped snapshot images of data. The snapshots would leave Nirvanix in one format, move through an intermediary (Panzura Cloud Storage System, from which they had been uploaded in the first place), and land in an OpenStack Swift storage provisioning service in HP's Public Cloud. As the snapshots exited the Panzura hop, said Gerber, they would -- in transit -- be the only remaining copy of the snapshots. It was crucial to decrypt and restore the images in a safe manner upon arrival at HP after their journey over the Internet.

The archival data and snapshots represented 8 TBs of data, not an overwhelming load for the time available, especially after deduplication and compression reduced the total to a little over 3 TBs.

For Velocity's client, its business required that a history of the snapshots be maintained to vouch for the state of its customers' data at different points in time. To lose the integrity of the images would not only put Velocity's relationship with its client at risk, but also its client's relationships with its customers. Nirvanix had seemed the ideal, high level, sophisticated cloud storage service in which to maintain the images. Now disaster had struck.

"HP being our partner, they jumped to help us. They did all the heavy lifting," said Gerber.

Janette Hausler, HP manager of partner marketing, said an HP developer downloaded the version of the Python SDK that works with both Nirvanix and HP cloud storage, then composed a migration script that would move the data out of Nirvanix to HP. The script took 10 hours to develop and test, Hausler said.

"The circumstances were certainly unusual," Hausler recalled in an interview. "We typically don't develop one-off scripts for our customers but rather provide tooling that enables them to use our cloud services," she said.

Given the limited timeframe and business-critical nature of the data, HP didn't leave it to the customer to map the migration. Once the Python script was available, Hausler said, HP was prepared to receive data at any rate that it could be delivered to it. But there was a bottleneck at the other end. "What we did experience was a data egress rate from Nirvanix of about 100 Mbps," she said.

Gerber was more emphatic in his description of the bottleneck. Nirvanix claimed its standard exit capacity was available, but Velocity found in initial downloads that it was only able to achieve a transfer rate of 50 Mbps. That indicated possible contention for network resources as other customers tried to get their data out of Nirvanix.

The data needed to move from an unspecified Nirvanix location -- Velocity didn't know exactly where it was stored -- to an HP Public Cloud in its U.S. West region 200 miles away, Hausler said in a follow-up email message. Cloud service providers often prefer not to mention the location of specific facilities. HP's was probably in Las Vegas or Phoenix, or some similar location.

Gerber said HP completed the migration in four days, with the data now stored, with the snapshots intact, at its new home. "I thought it was phenomenal that HP got everything moving so quickly," said Gerber. "Without them, we would have lost all the snapshots," he said.

Hausler described the event in more routine terms. It was "a seamless experience" for Velocity and its client. However, that's just one extraction done. "We are working on several other migrations," she said.

Spokesmen for CoreSite, which leases data center space to Nirvanix and other cloud storage services, said they are making high-capacity lines available when the migration is to another service in the same CoreSite facility. They said Amazon Web Services was the main beneficiary in their facilities.