Last week, Cybersecurity and Infrastructure Security Agency officials spoke candidly about the challenges they faced tracking the use of F5 products across the civilian federal government. While CISA knows there are thousands of instances of F5 currently in use, it admitted it wasn’t certain where each instance was deployed.
The uncertainty came as the agency issued an emergency directive related to F5, instructing other government agencies to find and patch any F5 instances. The urgency stemmed from the fact that F5 itself had revealed a nation-state had gained a long-term foothold in its systems.
One of the main goals of the directive: “help us identify the different F5 technology in the federal network,” as one official told reporters.
CISA didn’t already have a complete picture of that despite the billions of dollars spent on a program, Continuous Diagnostics and Mitigation (CDM), designed for, among other things, “increasing visibility into the federal cybersecurity posture,” which CISA’s website for the program states is one of its main four goals.
CISA’s lack of awareness about the extent of the F5 vulnerability’s presence in the federal government highlights a weakness in a program that is, by and large, a well-regarded one. But the fact that CDM did not automatically identify F5 prevalence is a circumstance of fast-changing technology and a shortcoming in the part of CDM that’s focused on keeping track of digital assets, according to current and former CISA officials and cyber industry professionals.
Missing the edge
“CDM has been highly focused on typical assets, like computers and servers, and they’ve struggled on the network side in many cases,” Jonathan Trull, CISO at Qualys, told CyberScoop.
Sean Connelly, an 11-year CISA veteran, said he’s been familiar with CDM dating back to its earliest days. The program began in 2012 with $6 billion in contracts. The Department of Homeland Security had received billions to administer the program over its first decade and as of 2022, planned to spend billions more on it over the next 10 years.
“A lot of the CDM capabilities initially were more focused on internal networks and what was internally going on inside the agencies themselves, and more about mission-oriented systems, not systems that were directly connected to the internet,” Connelly, now executive director for global zero trust strategy and policy at ZScaler, told CyberScoop. That contrasts with F5’s presence on edge devices, he said.
“Those type of devices, just those firewalls and those type of devices at the edge, typically, they don’t have the same type of reporting capabilities as the internal networks, or ones where you can put some type of agent on the device,” Connelly continued. “A lot of those edge devices, they’re proprietary. They don’t have the ability to be able to put a sensor on.”
Edge devices also happen to be a favorite target of hackers right now, especially China-linked ones, said Matt Hartman, the former deputy executive assistant director for cybersecurity at CISA.
“Given the scale and diversity of agency networks, there are visibility gaps, especially for technologies like network edge devices,” Hartman, now chief strategy officer at Merlin Group, told CyberScop in written answers. “Devices like F5 BIG-IP load balancers often reside in demilitarized zones, or DMZs, that sit between an agency’s internal network and the public internet. These environments aren’t always monitored by the same inventory and telemetry tools as agencies’ enterprise networks. And because of where these devices sit and the nuances of how they are monitored, they are an ideal entry point or pivot point for adversaries.”
Connelly said the CDM team is constantly focused on expanding its visibility into federal networks. And he said there are other programs that can help with that outside of CDM, like CISA’s cyberhygiene service, CyHy.
The F5 vulnerability isn’t the only one to illustrate where CDM needs work to increase visibility.
Matt House, CDM program manager, said last year that CDM was looking to tackle cloud infrastructure. But he said “we are largely blind” when applying existing CDM tools for platform-as-a-service and software-as-a-service, and that a first step to changing that was looking at how CDM defines assets. He also said not every agency has 100% identification and monitoring of assets for even on-premise hardware.
Trull said that “the consideration of what an asset is has changed tremendously.”
A June Government Accountability Office report found that CDM had “partially met” its cybersecurity visibility goals, with CISA and the Office of Management and Budget creating helpful “dashboards” for agencies. But a lack of CISA guidance is holding back those goals, the report concluded.
“The dashboards are intended to visualize information from each of the capability areas, providing insight into the cybersecurity posture associated with assets, users, networks, and data,” the report states. “However, officials from 21 of the 23 civilian … agencies stated that they had not fully implemented capabilities within the network security management and data protection management areas. According to officials from several agencies, they are awaiting additional guidance from CISA regarding these capabilities.”
CISA also doesn’t consider CDM adequate for identifying systems for migration to post-quantum cryptography, it concluded last year.
Hartman — who, like everyone CyberScoop interviewed for this story, praised the CDM program — emphasized that any current shortcomings do not reflect the program’s future capabilities.
“Today, CDM excels on traditional IT assets, like servers and workstations, but is not fully optimized for specialized systems like OT and IoT, or cloud-native resources and containerized workloads that change dynamically,” he said. “The positive news is that CISA fully acknowledges these gaps and they are on the CDM deployment roadmap.”
How CDM works with the directive
In lieu of automated CDM identification, an emergency directive like the one issued for the F5 vulnerability triggers a series of federal agency responses.
“The process typically begins with inventory validation: confirming what assets are in place, where they reside and how they are configured,” Shane Barney, CISO at Keeper Security and former CISO at U.S. Citizenship and Immigration Services, said in a written reply. “To address issues related to devices like F5 systems, agency teams rely on multiple data sources, including network scans, asset inventories and procurement records, to ensure complete visibility and to account for every system.”
Hartman said “CDM has streamlined how CISA and agencies are able to quickly and comprehensively inventory their environments,” improving the speed of the emergency directive process.
A CISA spokesperson, Marci McCarthy, said the emergency directive “was a great example of CISA working with a technology partner to communicate important and timely information.”
CISA officials who briefed the media last week said federal agency budget and personnel cutbacks wouldn’t have any impact on response to the directive. Ohio Rep. Shontel Brown, the top Democrat on the Oversight Subcommittee on Cybersecurity, Information Technology, and Government Innovation, said she was concerned about how those cuts might affect CDM’s performance.
“As Ranking Member, I support the Continuous Diagnostics and Monitoring program,” Brown told CyberScoop in a written statement. “However, the Trump Administration’s mass firings and funding cuts have jeopardized the effectiveness of this program. I will continue to monitor the situation at CISA and demand transparency and oversight to support the agency’s efforts in this nation’s cybersecurity.”
The emergency directive process works in large part because of agency collaboration, said Jeff Greene, former executive assistant director for cybersecurity at CISA.
“CISA’s dramatic improvements in protecting” federal civilian agencies “and the effectiveness of the ED process specifically was one of the biggest surprises when I got to the cybersecurity division,” Greene, who since has joined the senior cyber leadership team of Cambridge Global Advisors, told CyberScoop. “It worked because we partnered with the other agencies, and the team was careful only to go use an ED when it was truly necessary.”
CDM’s existence, however, is vital overall to responding to incidents like the F5 vulnerability, said Bill Wright, global head of government affairs at Elastic.
“Without a mature, comprehensive CDM program, federal agencies could be flying blind,” he said in a written statement. “This incident demonstrates that CDM is the foundational capability that makes a rapid, government-wide response like this possible.”
The F5 vulnerability exposure “highlights a critical tension between CDM’s intended outcomes and real-world execution,” said Ensar Seker, CISO at SOCRadar.
“The fact that agencies are now scrambling to inventory thousands of F5 instances, many potentially exposed, shows the gap that still exists between data collection and actionable insight,” he told CyberScoop in a written comment. “CDM may technically collect asset data, but if that data isn’t normalized, federated, or readily queryable across agency environments, it loses operational utility in moments like this.”
Matt Kapko contributed reporting to this story.
The post F5 vulnerability highlights weak points in DHS’s CDM program appeared first on CyberScoop.
The Continuous Diagnostics and Mitigation program is oft-praised, but there are areas where it doesn’t yet excel, as a recent CISA emergency directive shows.
The post F5 vulnerability highlights weak points in DHS’s CDM program appeared first on CyberScoop. Read MoreCyberScoop
