CrowdStrike, a cybersecurity company, has released its analysis on the Falcon Sensor software update crash that affected millions of Windows devices worldwide.
The “Channel File 291” incident was traced back to a content validation issue. This issue came up when a new Template Type was introduced. The purpose of this new Template Type was to detect and identify novel attack techniques that exploit named pipes and other Windows interprocess communication (IPC) mechanisms. This incident was originally highlighted in the Preliminary Post Incident Review (PIR).
The problem is related to a content update that caused multiple issues, leading to a crash. There was a mismatch between the inputs passed to the Content Validator and the Content Interpreter.
CrowdStrike stated that the parameter mismatch was not detected during the rigorous testing process, which involved multiple layers of scrutiny. This was partly because the 21st input during testing and the initial IPC Template Instances delivered between March and April 2024 utilized wildcard matching criteria.
The new Channel File 291 released on July 19, 2024, was the first IPC Template Instance to use the 21st input parameter field. The absence of a specific test case for non-wildcard matching criteria in the 21st field was only highlighted after the Rapid Response Content had been sent to the sensors.
“Sensors that received the new version of Channel File 291 carrying the problematic content were exposed to a latent out-of-bounds read issue in the Content Interpreter,” the company said.
“At the next IPC notification from the operating system, the new IPC Template Instances were evaluated, specifying a comparison against the 21st input value. The Content Interpreter expected only 20 values. Therefore, the attempt to access the 21st value produced an out-of-bounds memory read beyond the end of the input data array and resulted in a system crash.”
“CrowdStrike addressed the issue by verifying the number of input fields in the Template Type at sensor compile time and implementing runtime input array bounds checks in the Content Interpreter. They also fixed the number of inputs provided by the IPC Template Type.”
“The added bounds check prevents the Content Interpreter from performing an out-of-bounds access of the input array and crashing the system,” it noted. “The additional check adds an extra layer of runtime validation that the size of the input array matches the number of inputs expected by the Rapid Response Content.”
On top of that, CrowdStrike said it plans to increase test coverage during Template Type development to include test cases for non-wildcard matching criteria for each field in all (future) Template Types.
Some of the sensor updates are also expected to resolve the following gaps:
The Content Validator is being modified to add new checks to ensure that content in Template Instances does not include matching criteria that match over more fields than are being provided as input to the Content Interpreter
The Content Validator is being modified to only allow wildcard matching criteria in the 21st field, which prevents the out-of-bounds access in the sensors that only provide 20 inputs
The Content Configuration System has been updated with new test procedures to ensure that every new Template Instance is tested, regardless of the fact that the initial Template Instance is tested with the Template Type at creation
The Content Configuration System has been updated with additional deployment layers and acceptance checks
The Falcon platform has been updated to provide customers with increased control over the delivery of Rapid Response Content
CrowdStrike has hired two independent software security vendors to review the Falcon sensor code. They are also conducting a review of the quality process from development to deployment.