Governance: Who has authority? Who is accountable? These are perhaps the two most important questions in reducing cyber risk to operations. There are “big G” Governance questions such as: who should set the overall OT cyber security agenda?; what metrics should be achieved?; who should have authority to make the ultimate risk trade-offs? and who has the accountability if an incident occurs?
There are also “small g” governance questions such as: who will decide whether to patch a specific device or create a mitigation plan?; what tools will a business use to address certain cyber risks?; should a particular device be replaced because its firmware is out of date or can it wait until the next upgrade cycle? More than talent, tools, or tactics, governance is the most fundamental decision to get right if we are to achieve success in defending critical infrastructure.
We often hear debates about IT vs. OT. Should the CISO, Head of Operations, or CIO be in charge? Who should control the security decisions on the OT assets within a plant or SCADA environment? If the CISO is accountable, should not he or she have the authority to make decisions? If he or she has authority and accountability, should not the budget and resources be aligned with those as well?
In today’s large and complex industrial organizations, two themes emerge:
First, there is no one-size-fits-all answer. The right governance structure depends on the culture and existing model of the rest of the organization. Second, there is no single point of authority and accountability for all the above decisions. The right governance involves coordination and shared decision-rights across IT, security/risk management, operations, and finance. Although it would be nice to have a standard construct where accountability and authority are vested in one person or organizational function, this is almost impossible given the realities of managing operations assets and processes.
So, if the right answer is so critical, yet so varied, how do you design the right approach for a specific organization?
There are five key principles to establishing the right governance model for OT cyber security in your organization.
1. Start with alignment at the top
Achieving the right governance model requires clear alignment of the C-suite as to the real risks to operations, the risk appetite of the senior team and board of directors, rough estimates of cost to achieve different levels of security maturity, and how the senior team will make decisions on key trade-offs in these areas.
The natural leader for this exercise is the CISO. Of all the many hats that a CISO must wear, this is perhaps the most important. This is not to say that the CISO will have the authority to make all the decisions. To the contrary, in most successful exercises we have seen, the CISO plays an influencing-rather than a determinative - role in bring the senior team to alignment on the best path forward, taking into account the various trade-offs across the business.
Although specific governance models often focus on the definition of where authority and accountability reside, we have seen many RACI (responsible, accountable, consulted, and informed) charts become paper exercises unless there is true shared understanding of objectives and priorities at the top.
This alignment ensures budgets, metrics and resources are based on an agreed-set of objectives. Many organizations find they are somewhat down the OT cybersecurity journey, but without a clear alignment at the top. In most cases, the best choice is to reset and ensure the team takes the time to establish this basis of understanding or future progress may slow.
2. Go with the flow of the current organization model, not against it
One of the most successful OT cyber security executions we have seen came from a utility holding company with a culture of business-unit independence and ownership of results. The company’s incumbent governance model uses the classical distributed business-unit P&L ownership model made famous by Emerson Electric, Illinois Tool Works, Danaher, and many other industrial companies over the years. The principle is to make clear accountabilities around the “what” – i.e. targets and objectives. Then let management of each business unit have full authority as to the “how” - i.e. strategies and tactics to deliver.
In the case of cyber security, the senior team established a very clear, top-down directive as to the objective and standards they expected each of the business units to achieve – in this case the CSC top 20 controls - down to specific maturity levels by each sub-control. They put a company-wide review process in place to ensure progress to the objective.
The CISO was very involved in helping shape both the objective as well as the process. Then the “how” was left to each business unit. Decisions such as what tools to deploy, how to balance compensating controls, the specific approach to achieving least-privilege settings, specific approaches to incident response, etc. came down to the business unit, but within an overall construct of a set of objectives and metrics.
I can already hear the complaints with this approach: duplication of effort, inefficient use of underlying tools, not applying corporate best-in-class approaches to each business unit, need for duplicate cyber security expertise in a world where cyber talent is limited, too focused on a set of standards rather than real “security” and reduction in threats or time to remediation.
All of these limitations are absolutely true and were addressed through other measures. However, the organization did not have a culture of centralized experts or top-down directives of shared tools or infrastructure. To create such a model would have meant going against the primary mode of operation for the organization. Had the CISO tried to push in this direction, he most likely would have ultimately failed because it was not in the organization’s DNA.
He knew that no governance model is perfect. Successful OT cyber security leaders take the time to understand the overall governance culture of their organizations and build a model that works with the flow, rather than trying to force-fit a theoretically better governance model. Then they address the gaps unique to that approach to ensure the limitations do not become hindrances.
3. Follow the money
One of the most challenging aspects of cyber security governance is to ensure alignment of budgets with accountability. In many organizations, cyber security-related spend is distributed across the company – plants may be responsible for the budgets of their OT systems including updates, patching, and management; corporate IT, however, may manage the budgets of network gear and possibly segmentation; the CISO may manage spend on security-specific initiatives such as anti-malware or monitoring logs for threat detection; HR may have the budget for training and awareness development; and facilities management may be responsible for the building systems such as warehousing, chillers, freezers, etc. which may be critical to operations. In this kind of distributed environment, capturing current spend related to cyber security, as well as prioritizing additional spend on new protective or detective measures is difficult.
We have seen clients adapt to this situation in different ways. Some have created a shadow accounting system which aggregates spend from different business units into a holistic cybersecurity budget. Others have established clear objectives and asked business units to achieve those objectives while managing their overall budgets in line with typical year-over-year increases, essentially making trade-offs of spending on cyber security vs. other items. Still, others manage security compliance at a plant-by-plant level and ensure that the budgets for the plants take into account cyber security as one key element of its metrics.
Whether they use one of the models above or some alternative, organizations first need to gain visibility to total cyber security spend and second to align budget authority with security accountability to manage risk effectively.
4. Adopt operations’ use of balanced scorecards and KPIs
Successful operations organizations run on metrics, targets, detailed procedures, and tactical results monitored on hourly, daily and weekly basis. All too often, cyber security objectives are subtle or aspirational: reduce vulnerabilities, identify potential malware, identify attackers, improve incident response by X%, etc. Successful OT cyber security approaches will work with the flow of operations management and transform these subtle objectives into very tactical targets and metrics that can be shown on simple red, yellow, green charts.
One Verve customer adopted the NIST CSF as their cyber security framework and went to the next step and implemented a set of measures that could be tracked on a weekly, monthly and quarterly basis. Each control area had a set of targets and metrics (e.g., number of critical patches not deployed, number of machines without a backup in the past week, number of false positive alerts, time spent by operational personnel responding to false alarms, etc.).
Importantly, they treated the corporate SOC that was analyzing threat data as if it were an upstream supplier of material. They were held to targets relating to threat detection quality and timeliness. These data were shared regularly between operations and the SOC to ensure the teams had accountability to one another. When items were not “in the green” remediation plans were put in place, as they would be if it were a product quality or throughput metric.
Operations is used to managing a balanced scorecard of KPIs beyond just production volume and cost. They already manage occupational safety, environmental quality, product quality, etc. in parallel to their operational metrics. By working with the flow and making cyber security an additional element of that balanced scorecard, organizations can align accountability with the authority to assign resources and take action.
5. Get tactical
If we consider the NIST Cybersecurity Framework, it contains five core areas and 98 specific subcategories. CSC 20 has over 140 sub-controls. It is not practical that a high-level governance model will succeed across all of these sub-elements. Just as operations does, the team needs to build detailed procedures identifying accountable parties and their levels of authority around specific deliverables.
Governance tends to break down at the micro-level. For instance, in the identify component of NIST CSF: who is in charge of maintaining the asset database with the required information? The IT team may believe it should do so, but OT may argue that running the IT tools on the OT networks is not safe or appropriate. Furthermore, in some organizations the asset information required at the plant level may be well in excess of what is necessary at corporate from a cyber security management point of view. Or in another example, the decision to patch a critical device immediately or leave it until an outage or perhaps leave it semi-permanently until the device can be upgraded is a debate we see almost daily with our clients.
In critical operations where a wrong – or perhaps even a correct, but delayed – decision can lead to lost production, injury, or even death these detailed decision-rights are critical to assign upfront. Successful operators take the time to document in detail not only the decision rights but also who will take the necessary actions in areas such as maintenance or quality.
We have found that following these five principles helps define an OT cyber security governance model that works with an organization’s methods of running operations. Next, find out how to achieve maturity using the NIST cyber security framework: