Subscribe / Unsubscribe Enewsletters | Login | Register

Pencil Banner

How to create a company culture that can weather failure

Mary Branscombe | Aug. 16, 2017
In technology, things go wrong all the time, sometimes catastrophically. But if you stop paying attention after you fix the immediate problem, you’re missing out on the benefit of learning from experience.

When you look at what you can learn from an incident, look at what information is available to the people working on the problem and how quickly they can get it, so you can develop clear guidelines to avoid compounding the problem due to stress, confusion or fatigue.

"What can go wrong in high pressure situations is that people can essentially lose sight of the goal of fixing the problem," warns Lambert. "You can also lose a lot of context and focus by having too many sources of information so we've learned to be very targeted about the information you pick."

To avoid late night confusion, Nather suggests that "it's good to train until it becomes a reflex so you don't have to think so hard about who you're supposed to call; it comes more automatically."


Don't ignore technical debt

Technical debt can be the reason you fall prey to ransomware, or it can just make key processes slower and less efficient.

"Assess your assets for business criticality, level of non-compliance with security hygiene, cost to remediate, and risk to the business if the asset is compromised, and develop lower cost, lower risk mitigations while you work on the most complex infrastructure renovations," advises Luta Security CEO Katie Moussouris. "Then develop a plan to keep the org healthy on an ongoing basis and make sure this plan itself is also reviewed for relevance and adjusted. Much of the technical debt that built up in the first place was due to an incorrect notion that whatever is working on the network shouldn't be touched in case it breaks."


Do use all your resources

There are plenty of templates for incident response, though fewer that cover how to lean from incidents. Etsy's Morgue tracker is open source and the company has also published an excellent debriefing facilitation guide.

The learning review process is as much about as communications as technology. "Business executive coaches who normally tackle lines of communications within the organization can address this area as well; not the technical aspects of where you need to pull information from, but what you do with it afterwards," says Nather.


Do spread the word - inside and out

Part of making sure the knowledge you can gain from an incident is applied is passing on what you've learned.

"Make sure the resulting lessons are simply explained and made available for the entire organization to learn from," says Hinchcliffe. "It's this last part that is frequently omitted and can doom organizations to proverbially relive IT history over and over again."

You also want to share the lessons beyond your organization, suggests Nather. There are formal organizations like the Information Technology Information Sharing and Analysis Center, as well as similar organizations for financial services, oil and gas, healthcare, automotive, retail and legal, and plenty of informal routes for sharing intelligence. There's a value in supporting and formalizing that, suggests Nather.


Previous Page  1  2  3  4  5  6  7  Next Page 

Sign up for CIO Asia eNewsletters.