The Equifax breach got me thinking about the whole subject of change control and change risk. What makes this event significant is that Equifax suffered an IT related issue caused by not doing something; in a word: delaying an upgrade.
If they can’t manage their risk, how can they manage ours?
I have spent most of my IT career (> 30 years) working in or for large financial institutions. Lately, two of the UK banks I have worked with have been in the news for customer affecting system failures that have followed upgrades or system configuration changes. This has made already conservative cultures even more risk-averse. What outsiders see are companies whose Unique Selling Proposition and role in the marketplace are to manage others’ risk, but are unable to manage their own IT risk. In financial planning the “do nothing option” carries with it a risk weighting. How much money will I lose if I don’t switch my investments? In IT change control systems, the “do nothing” status quo has no measured risk. This might have been true for systems that were isolated. Now that all IT, even that deep in the bowels of a bank, is exposed in some way to the internet or outside forces, this position needs to be revised.
You can’t wait till the weekend to apply changes
The increased security threats require constant patching, resulting in more frequent changes. There are further change control procedures and mindset changes required here. Waiting until the weekend to apply changes can and will introduce risk, and will limit the time available for competing systems to get upgraded, causing queuing and delays. The public is becoming more aware of the necessity to patch. Their appreciation of an online service will not be compromised by a short blip in service while a system is patched, but will certainly be affected by their personal data being stolen. The focus for change control needs to be of the total effect that patching a system has on the end user. In the past, the measure has been the outage while systems restart and not the benefits of knowing that your personal data is safe and sound. This does not detract from the hard work that an IT professional puts into designing robust, resilient IT systems. It still behooves IT professionals to carry out changes with agreed published steps, in a controlled manner with fixed blackout points.
You can’t afford to have change control to slow you down
During my IT career, I have seen change control systems start with a simple handwritten one side of an A4 sized paper and then move to the all-encompassing asset management change scheduling apps that we see today. I have seen the way change policies are written and implemented. I have seen regulation, in particular, financial regulators come into this space. Initially, they were concerned that a change system adequately promoted and protected disaster recovery plans. This was in response to 9/11. Later, as IT systems became pervasive, regulatory concern expanded into all aspects of change control. The goal here was to make sure that a financial institution could not fail because of a change. The rise of social media to make individual complaints more visible and customers more aware has upped the profile of IT incidents. Together these forces are a drag on change. They effectively slow down the rate of change possible in a large organization while not actually reducing risk in the overall IT system.
Good governance should triumph over change control
Given what has happened at Equifax, it now behooves both the financial institutions themselves and their regulators to look at the policies they set. They need to encourage and enable a change process and policy that does consider the risk of not applying a patch. They need to prioritize and report on security patch status so that the change system is there to enable change, not to prevent it. They need constantly to review policy decisions on when changes can take place; the concept of “out of hours” has changed over time. They need to review the policy with respect to the different systems and protocols. I have seen changes circumscribed to a particular technology because of a specific issue with it. When the IT staff responsible for the technology fix this issue, the change system doesn’t recognize this fix and continues to limit changes to this system based on the original shortcoming—in effect shutting the stable door after the horse has bolted, been recaptured and put in a new stable. Change needs to be controlled, but not at the expense of good governance and protection of customer data.
We in IT are responsible for the safekeeping of personal data. We need to proactively be making the necessary procedural and mindset changes to enable this, not merely react based on customer complaints and lawsuits. Customers are much more tech-savvy and will be able to see the real effect on their data if they put their trust in financial institutions that can’t look after it.