NASA Report On Air Safety Draws CriticismNASA Report On Air Safety Draws Criticism

The release of a significantly redacted NASA survey on airline safety and security has prompted criticism from Congress and industry groups for the survey's lack of scientific rigor and the pre-holiday timing of its release.

"NASA today dumped three-year-old, unanalyzed data from more than 25,000 interviews of commercial and general aviation pilots on aviation safety," said Rep. Brad Miller, D-N.C., chairman of the Investigations and Oversight Subcommittee, in a statement issued Monday. "According to NASA, they never intended to analyze the data, just test the 'methodology' used for the interviews. The taxpayers who paid $11.5 million for the surveys and the pilots who spent 12,000 hours answering the survey questions hoped for more. They hoped the survey would lead to improved aviation safety."

Having received three Freedom of Information Act requests from The Associated Press in 2006 and 2007, NASA initially declined to release the survey on the basis of respondent privacy and because the findings could undermine public confidence in air transportation safety.

The National Aviation Operational Monitoring Service (NAOMS) survey, based on data gathered from April 2001 through December 2004, includes comments from more than 24,000 airline pilots about air transportation problems and perceptions. Among its findings: The rate of engine failures was four times higher than expected.

Such aberrant data, said NASA administrator Michael Griffin, called the survey methodology into question more than casting doubt on the safety of air transportation.

"From the conclusions that arose out of the survey itself, it was reported, for example, that the survey unearthed approximately four times as many engine failures as the FAA believes that it has cognizance of," said Griffin, according to a transcript of a Monday press conference provided by NASA. "Engine failures, as I am sure you know, are a very high-profile item. This is an area where if someone comes in and says we are seeing four times as many engine failures as are being otherwise reported, it calls into question the reporting mechanism rather than the underlying rate of engine failure, which we believe we understand."

It was on Monday that NASA reversed its decision and released some of the NAOMS survey information. "The goal was to release as much survey response information as possible before the end of the calendar year, but only release information that does not contain confidential commercial information or information that could compromise the anonymity of individual pilots," said Bryan O'Connor, chief safety and mission assurance officer, during the press conference.

"NASA's initial determination in response to the request for NAOMS data relied on Exemption 4 under the [Freedom of Information] act, which protects confidential commercial information," said a NASA spokesperson via e-mail. "NASA Administrator Michael Griffin reviewed the initial determination and asked a panel to review how to publicly release the NAOMS information in a timely manner using a redaction process. NASA regrets any impression that the agency was in any way trying to put commercial interests ahead of public safety. That was not, and never will be, the case."

NASA has said it will release all the unreleased data that it's legally permitted to disclose at some undetermined point in the future.

Rep. Miller, however, expressed skepticism that protecting pilot privacy was a valid reason to redact the data. "Eighty percent of the pilots asked to participate in the half-hour survey agreed," Miller said in his statement. "Those pilots obviously wanted others to know their observations and experiences. It is hard to imagine that those pilots would want useful information scrubbed from the data out of a fear that stray bits of information might somehow be connected to identify a specific pilot's answers to the survey."

A spokesman for the Air Line Pilots Association defended redactions as a way to protect its members against retaliation for expressing safety concerns. "We are not, have never been, and I don't think every will be, advocates for the public release of raw safety data," he said, noting that raw data can easily be misconstrued or cherry-picked to fit a specific agenda.

He stressed that the ALPA supports ongoing programs to collect and analyze airline safety data and said that issues of pilot fatigue raised by the report are of real concern. "We believe that fatigue is an ever-present and growing problem," he said, citing antiquated regulations, understaffing of pilots, and schedules that push legally permissible limits.

Characterizing the U.S. air transportation system as the safest in the world, James C. May, president of the Air Transport Association of America, an industry group that represents air carriers, said in a statement, "While we appreciate any insight that might be derived from the NASA study, as the FAA noted, the study was not designed to capture real-time, verifiable data. As such, it is important that the report be viewed in that context."