On Crowdstrike and automatic updates
Don’t use automatic updates!
Don’t use automatic updates!
Problem solved: A system can’t crash from an update you don’t install.
Brilliant!
Now let's see if there are any second order effects from such a decision…
You no longer have security updates until you manually install them.
On laptops.
On servers.
In your platform’s apps.
The amount of hours spent on an ongoing manual patching process would be a massive hit to engineering productivity. These are hours you are no longer building better products, a more resilient platform or reading up-to-the-minute posts about the election.
Making it a binary decision to auto-patch or not is like distrusting the stock market and coming to the obvious solution to keep your life savings as cash in the freezer.
It turns out that we live in a world with nuance and any number of decisions exist between zero and one.
Any decisions need to balance:
A) The risk reduction from having a given group of devices (Servers, workstations, apps, etc) patched to latest at all times
B) The risk of having a patch cause production or operational issues
C) The work that will be incurred by switching to updates as a manual process
Even with these three, there’s a lot things to keep in mind:
⚖️ You don’t have to have the same policy for every class of patchable software - not all software is equally impactful nor are all systems equally critical
♻️ Having and testing rollback processes helps build overall system resilience - this mitigates risk not just from failed updates but a broad range of potential issues
🛠️ Not all vendors are equally reliable with their updates - Much like security at a Trump rally, Crowdstrike is probably going to be tightening things up quite a bit in the near future.
🪦 Most patches don’t pose the risk of leaving you with an unbootable system - do you know which ones could?
🦑 You can do manual testing before releasing the automation kraken - Again, it’s not a binary decision to do or don’t do automated patching
🧪 Increased test coverage can better catch breaking updates in your apps’ 3rd party dependencies in the CI/CD pipeline before they roll out to the world
💥 Staged rollouts to portions of users/customers/systems reduces the potential blast radius of a failed update.
As with most things in both life and security, there are a lot of different paths available.
No single way of doing things will be a fit for everyone.
Figure out what works for your team, systems, business and risk tolerance and make sure there’s transparency into what the tradeoffs being made are.
Make sure to periodically reassess the decision to ensure assumptions and reality are still accurately represented, adjusting as necessary.
People proclaiming the One True Way tend to not understand nuance and should be considered with the same weight you would consider crypto advice in the replies to Elon’s latest tweet.