Culture / Storage

Air Force Scrambles to Recover 100,000 Lost Data Files

19 Jun 2016 8:15am, by

A corrupt database wiped out 12 years worth of Air Force files, permanently destroying records on 100,000 Inspector General investigations dating back to 2004. But after weeks of worry, and embarrassing headlines around the web, there was a second news story this week, which reports that the Air Force had found the files again.

So what exactly happened? According to an article at Government Executive, the culprit appears to be a corrupted database run by Lockheed Martin. It contained some particularly sensitive information, specifically, the records from the Air Force’s inspector general and legislative liaison divisions, which investigate fraud, waste, workplace disputes and abuse within the U.S. Air Force.

A spokesperson for the Air Force told the site that the database contained the complaints, the findings, and whatever actions were then taken — all the way from 2004 up to the present. Ongoing investigations were being delayed, slowing their response time to Congress and the Inspector General. “The Air Force is assessing the immediate impact of the data loss,” read one statement from an Air Force spokesperson. “We’ve opened an investigation to try to find out what’s going on, but right now, we just don’t know.”

Of course, this raised the inevitable discussion about government databases and faulty backup procedures. “There has been too much recently of government agencies losing data due to ‘crashes’ or ‘inadvertent’ deletions,” posted one user on Hacker News. “For example the IRS, state department, and now the Air Force.”

“Please note the severity of this,” added another user. “It’s not ‘We had a crash and lost a weeks worth of data’… This is ‘We had a crash and lost over a decade’s worth of data’ — yeah, that shouldn’t happen. Ever. If you don’t have a working backup from sometime in the past 13 years, you are doing something wrong. That’s not a small series of failures or mistakes. That’s either staggering levels of incompetence or maliciousness.”

Allegedly, Lockheed Martin was in charge of the database and spent a full two weeks trying to recover it before notifying the Air Force. The Air Force then reached out to experts at the Pentagon, as well as private contractors; a Pentagon spokesperson told Government Executive. “We’ve kind of exhausted everything we can to recover within [the Air Force], and now we’re going to outside experts to see if they can help.” Virginia Senator Mark Warner complained that Congress wasn’t notified until five days later — in a tersely-worded note. “The five-sentence notification to Congress did not contain information that appeared to have the benefit of five days of working the issue…”

Was it espionage? A malicious hacker? The Air Force says they had no evidence it was anything more than a corrupted database. “[W]e’re doing our due diligence and checking out all avenues of the investigation to find out if there’s anything that we’re not aware of.” It’s always hard to get details about military data. Lockheed Martin said only that they were aware of the issue, and that they were “working with the Air Force to identify the cause, and restore the lost data.”

But by Wednesday, the Air Force was back with a new announcement. “After aggressively leveraging all vendor and department capabilities, the Air Force made a full recovery of the Automated Case Tracking System database…” The service cited an “extensive data recovery efforts over the weekend and this week” which included Lockheed Martin, Oracle, and the 24th Air Force, the “cyber division” of the Air Force. “They’ve been working nonstop since they got called in here a few days ago,” the Air Force chief of staff told a group of defense reporters, adding that the only lasting impact of the incident will be “making sure we understand exactly what happened, how it happened and how we keep it from ever happening again.”

The Air Force promised that they’d implement every necessary “corrective measure” — and that there’d be an independent review.

What is the state of your backups this weekend?


WebReduce

Feature image via Pixabay.

A newsletter digest of the week’s most important stories & analyses.