Think Over Data's Value When Deciding What To StoreThink Over Data's Value When Deciding What To Store

Are you a data pack rat? A new report from the Society of Information Management's Advanced Practices Council says companies should be as careful in considering what information they discard as they do about what they keep.

The new report, "Information Lifecycle Management Concepts, Practices and Values," is the fruit of year-long study looking at the information retention practices and policies of corporations in multiple industries and is based on an online survey of 345 IT and storage managers, plus interviews with IT professionals and executives at 15 case study organizations.

Among the 12 lessons from organizations that have been early in launching information lifecycle management initiatives -- and one of those key findings is it's as important to figure out what data you can throw out as it is to decide what to store indefinitely.

The study found that the majority of companies examined had "inconsistent and incompatible" policies for discarding information, says the report's author, Dr. Jim Short, research director of the global information industry center at the University of California, San Diego. So, that results in most companies being inclined to hold on to increasing volumes of data for much longer periods than ever before -- and that volume of information that's retained will only get bigger, he says.

"We're at the very beginning of retention growth," he says. That data retention explosion is due to a variety of reasons, ranging from the lack of consistent policies within companies about what to keep; uncertainty about what data needs to be saved for legal and regulatory purposes; the availability of storage devices that can more keep hold vast volumes of data more affordably; data being stored on multiple personal and company devices; and the hope that old data will have new uses.

The evolution of new data mining and data analysis tools are breathing life into old data for new-found business opportunities, says Short. The development of new analysis tools is also driving companies to shift data storage strategies, so that larger volumes and types of data are available in an "active mode" longer, rather than being tossed out completely or relinquishing to long-term storage media, he says.

"Companies are not only data-mining sales data, but are also keeping surveillance data much longer," he says. What kind of surveillance data? That includes data that's being created in warehouses and on retail shelves from sensor technology, like RFID, he says. Some companies are beginning to analyze that data for trends, and looking for ways to put those insights into good use.

So just like that old and ugly plaid suit still hanging in the back of your closet that you keep hoping will come back into vogue someday, companies are also inclined to think there's possible value in old data and would rather not risk throwing it out.

There are no hard-rules that work for all industries about what's ok to discard and what's best to keep, he says. Obviously, sectors like financial services and health care have regulatory and compliance issues that contribute significantly to decisions about what information to keep and for how long. Also, there appears to be no hard-rules about who's involved with data-keeping decision and policies from one organization to the next.

"On average, companies have six to eight people who play some part in policy, but some companies even have committees," he says. Those people could include compliance directors to storage managers to database manager, he says.

But even with multiple players and multiple organizations getting involved in policy setting, information retention issues can get fuzzy, he says. "A database administrator's job is to not destroy data," Short says. "So those organizations are storage drivers retaining information," and aren't inclined to dump data, he says.