CrowdStrike says bug in quality control process led to botched update

A CrowdStrike software update that crashed computers globally last week hitting services from aviation to banking and healthcare was caused by a bug in the U.S. cybersecurity firm’s quality control mechanism, the company said on Wednesday.

Friday’s outage happened because CrowdStrike’s Falcon Sensor, an advanced platform that protects systems from malicious software and hackers, contained a fault that forced computers running Microsoft‘s  Windows operating system to crash and show the “Blue Screen of Death.”

“Due to a bug in the Content Validator, one of the two Template Instances passed validation despite containing problematic content data,” CrowdStrike said in a statement, referring to the failure of an internal quality control mechanism that allowed the problematic data to slip through the company’s own safety checks.

CrowdStrike did not say what that content data was, nor why it was problematic. A “Template Instance” is a set of instructions that guides the software on what threats to look for and how to respond. CrowdStrike said it had added a “new check” to its quality control process in a bid to prevent the issue from occurring again.

The extent of the damage from the botched update is still being assessed. On Saturday, Microsoft said about 8.5 million Windows devices had been affected, and the U.S. House of Representatives Homeland Security Committee has sent a letter to CrowdStrike CEO George Kurtz asking him to testify.

CrowdStrike released information to fix affected systems last week, but experts said getting them back online would take time as it required manually weeding out the flawed code.

Wednesday’s statement was in line with a widely held assessment from cybersecurity experts that something in CrowdStrike’s quality control process had gone badly wrong.