Surviving The Business Data Tidal WaveSurviving The Business Data Tidal Wave

Can technology alone enable your company to stay ahead of its data storage demands? In many cases, the answer appears to be no.Informationworld blogger Matt Prigge recently pondered the implications of an amazing IT statistic. He notes that according to IDC research, the amount of enterprise data doubles every 18 months. The implications are disturbing -- and the fact that disk vendors continue to offer greater capacity at shrinking costs only serves to disguise the true magnitude of the problem: Personal, corporate, governmental -- it doesn't matter. We're keeping and maintaining way more of it than we can possibly ever use. The fact that an 18GB disk available in 1998 is roughly the same size, weight, and cost as a 2,000GB disk you can buy today is only serving to hide this problem and make us lazy about policing our data growth. "The sad truth," says Prigge, "is that no technological silver bullet exists today that will solve this problem for you. You'd never completely trust an automated system to decide what data you don't need any more. In the end, you need to do it."

It is true that a variety of new technologies offer a partial solution to this challenge. But if Prigge is correct, then even the best data deduplication, automated archiving, and data analysis tools will not suffice to turn back this tidal wave of business data.

Don't Miss: NEW! Storage How-To Center

Yet this leaves IT managers stuck on the horns of a major dilemma. It is one thing to declare that companies must implement organizational changes to fight a "data hoarding" mentality and to make hard decisions about when, where, and how to discern between essential and non-essential business data. It is quite another, however, to enforce such distinctions in real-world business environments where nobody wants to take responsibility for these tough -- and potentially hazardous -- decisions.

Having said that, I'm inclined to agree that technology alone will not solve this problem. And clearly, the worst thing a business can do is to assume that cheaper, bigger storage solutions will accomplish anything except postponing the inevitable.

Solving this problem will require a deft mix of technological innovation and organizational change. And it will definitely require companies to start making hard decisions about their data storage policies -- the sooner the better.