Governance: Who has authority? Who is accountable? These are perhaps the two most important questions in reducing cyber risk to operations. There are “big G” Governance questions such as: who should set the overall OT cyber security agenda?; what metrics should be achieved?; who should have authority to make the ultimate risk trade-offs? and who has the accountability if an incident occurs? There are also “small g” governance questions such as: who will decide whether to patch a specific device or create a mitigation plan?; what tools will a business use to address certain cyber risks?; should a particular device be replaced because its firmware is out of date or can it wait until the next upgrade cycle? More than talent, tools, or tactics, governance is the most fundamental decision to get right if we are to achieve success in defending critical infrastructure.
We often hear debates about IT vs. OT. Should the CISO or head of operations or CIO be in charge? Who should control the security decisions on the OT assets within a plant or SCADA environment? If the CISO is accountable, should not he or she have the authority to make decisions? If he or she has authority and accountability, should not the budget and resources be aligned with those as well?
In today’s large and complex industrial organizations two themes emerge: 1) there is no ‘one-size-fits-all’ answer – the right governance structure depends on the culture and existing model of the rest of the organization; and 2) there is no “single point of authority and accountability” for all the above decisions – the right governance involves coordination and shared decision-rights across IT, security/risk management, operations, and finance. Although it would be nice to have a standard construct where accountability and authority are vested in one person or organizational function, this is almost impossible given the realities of managing operations assets and processes.
So, if the right answer is so critical, yet so varied, how do you design the right approach for a specific organization?
We believe that there are five key principles to establishing the right governance model for OT cyber security in your organization.
1. Start with alignment at the top
Achieving the right governance model requires clear alignment of the C-suite as to the real risks to operations, the risk appetite of the senior team and board of directors, rough estimates of cost to achieve different levels of security maturity, and how the senior team will make decisions on key trade-offs in these areas.
The natural leader for this exercise is the CISO. Of all the many hats that a CISO must wear, this is perhaps the most important. This is not to say that the CISO will have the authority to make all the decisions. To the contrary, in most successful exercises we have seen, the CISO plays an influencing-rather than a determinative - role in bring the senior team to alignment on the best path forward, taking into account the various trade-offs across the business.
Although specific governance models often focus on the definition of where authority and accountability reside, we have seen many RACI charts become paper exercises unless there is true shared understanding of objectives and priorities at the top. This alignment ensures budgets, metrics and resources are based on an agreed-set of objectives. We often encounter clients part of the way down the OT cybersecurity journey, but with no clear alignment at the top. In most cases, the best choice is to reset and ensure the team takes the time to establish this basis of understanding or future progress may slow.
2. Go with the flow of the current organization model, not against it
One of the most successful OT cybersecurity executions we have seen came from a utility holding company with a culture of business-unit independence and ownership of results. The company’s incumbent governance model uses the classical distributed business-unit P&L ownership model made famous by Emerson Electric, Illinois Tool Works, Danaher, and many other industrial companies over the years. The principle is to make clear accountabilities around the “what” – i.e. targets and objectives. Then let management of each BU have full authority as to the “how”- strategies and tactics to deliver.
In the case of cybersecurity, the senior team established a very clear top down directive as to the objective and standards they expected each of the BU’s to achieve – in this case the CSC top 20 controls - down to specific maturity levels by each sub-control. They put in place a company-wide review process to ensure progress to the objective. The CISO was very involved in helping shape both the objective as well as the process. Then the “how” was left to each BU. Decisions such as: what tools to deploy, how to balance compensating controls, the specific approach to achieving least-privilege settings, specific approaches to incident response, etc. The BU had the authority to make decisions, but within an overall construct of a set of objectives and metrics.
I can already hear the complaints with this approach: duplication of effort, inefficient use of underlying tools, not applying corporate best-in-class approaches to each BU, need for duplicate cyber security expertise in a world where cyber talent is limited, too focused on a set of standards rather than real “security” and reduction in threats or time to remediation. All of these limitations are absolutely true and were addressed through other measures. However, the organization did not have a culture of centralized experts or top-down directives of shared tools or infrastructure. To create such a model would have meant going against the primary mode of operation for the organization. Had the CISO tried to push in this direction, he most likely would have ultimately failed because it was not in the organization’s DNA.
He knew that no governance model is perfect. Successful OT cybersecurity leaders take the time to understand the overall governance culture of their organizations and will build a model that works with the flow, rather than trying to force-fit a theoretically better governance model. They will then address the gaps unique to that approach to ensure the limitations do not become hindrances.
3. Follow the money
One of the most challenging aspects of governance is to ensure alignment of budgets with accountability. In many organizations, “cybersecurity-related” spend is distributed across the company – plants may be responsible for the budgets of their OT systems including updates, patching, and management; corporate IT, however, may manage the budgets of network gear and possibly segmentation; the CISO may manage spend on security-specific initiatives such as anti-malware or monitoring logs for threat detection; HR may have the budget for training and awareness development; and facilities management may be responsible for the building systems such as warehousing, chillers, freezers, etc. which may be critical to operations. In this kind of distributed environment, capturing current spend related to cyber security, as well as prioritizing additional spend on new protective or detective measures is difficult.
We have seen clients adapt to this situation in different ways. Some have created a shadow accounting system which aggregates spend from different business units into a “holistic cybersecurity budget.” Others have established clear objectives and asked business units to achieve those objectives while managing their overall budgets in line with typical year-over-year increases, essentially making trade-offs of spending on cybersecurity vs. other items. Still others manage security compliance at a plant-by-plant level and ensure that the budgets for the plants take into account cybersecurity as one key element of its metrics.
Whether they use one of the models above or some alternative, organizations first need to gain visibility to total cybersecurity spend and second to align budget authority with security accountability to manage risk effectively.
4. Adopt operations’ use of balanced scorecards & KPIs
Successful operations organizations run on metrics, targets, detailed procedures, and tactical results monitored on hourly, daily and weekly basis. All too often, cyber security objectives are subtle or aspirational: reduce vulnerabilities, identify potential malware, identify attackers, improve incident response by X%, etc. Successful OT cybersecurity approaches will work with the flow of operations management and transform these subtle objectives into very tactical targets and metrics that can be shown on simple red, yellow, green charts.
One of our clients used this operational approach to great effect. They adopted the NIST CSF as their cyber security framework and went to the next step and implemented a set of measures that could be tracked on a weekly, monthly and quarterly basis. Each control area had a set of targets and metrics (e.g., number of critical patches not deployed, number of machines without a backup in the past week, number of false positive alerts, time spent by operational personnel responding to false alarms, etc.). Importantly, they treated the corporate SOC that was analyzing threat data as if it were an upstream supplier of material. They were held to targets relating to threat detection quality and timeliness. These data were shared regularly between operations and the SOC to ensure the teams had accountability to one another. When items were not “in the green” remediation plans were put in place, as they would be if it were a product quality or throughput metric.
Operations is used to managing a balanced scorecard of KPIs beyond just production volume and cost. They already manage occupational safety, environmental quality, product quality, etc. in parallel to their operational metrics. By working with the flow and making cyber security an additional element of that balanced scorecard, organizations can align accountability with the authority to assign resources and take action.
5. Get tactical
If we consider the NIST CSF, it contains 5 core areas and 98 specific subcategories. CSC 20 has over 140 sub-controls. It is not practical that a high-level governance model will succeed across all of these sub-elements. Just as operations does, the team needs to build detailed procedures identifying accountable parties and their levels of authority around specific deliverables.
In many instances, governance breaks down at the micro-level. For instance, in the identify component of NIST CSF: who is in charge of maintaining the asset database with the required information? The IT team may believe it should do so, but OT may argue that running the IT tools on the OT networks is not safe or appropriate. Furthermore, in some organizations the asset information required at the plant level may be well in excess of what is necessary at corporate from a cyber security management point of view. Or in another example, the decision to patch a critical device immediately or leave it until an outage or perhaps leave it semi-permanently until the device can be upgraded is a debate we see almost daily with our clients.
In critical operations where a wrong – or perhaps even a correct, but delayed – decision can lead to lost production, injury, or even death these detailed decision-rights are critical to assign upfront. Successful operators take the time to document in detail not only the decision rights but also who will take the necessary actions in areas such as maintenance, quality, etc.
We have found that following these five principles helps define an OT cybersecurity governance model that works with an organization’s methods of running operations. We’d love to hear about your success (or war) stories as well.