The array then failed catastrophically, taking the entire bank's system offline for nearly a week and costing millions of dollars in lost transactions. That's when they called in Anthony R. Howard to troubleshoot.
There were three big screwups, says Howard, a best-selling author ("The Invisible Enemy: Black Fox") and independent technology consultant for Fortune 50 companies and the U.S. military. One, of course, was that the vendor shipped the wrong unit. The second was that the bank's IT staff tried to install the array itself without waiting for the vendor to send out a qualified technician to do it for them.
The third problem was the big one, though. Almost everyone involved in this screwup lied about it, says Howard. Only one staffer had the courage to admit what really happened.
"When the IT staff saw their jobs were in danger, they began to try to protect themselves and blaming the tech support staff of the vendor," says Howard. "After the bank's internal team was done with its investigation and found out that only one person told the truth, he was the only one who managed to keep his job."
Moral of the story: If the crime doesn't get you, the cover-up will.
Fatal IT mistake No. 4: The porn identity
Late one evening a couple years back, a network admin for a Fortune 100 firm was looking for an empty backup tape. He grabbed one from the desk drawer of a senior system administrator and popped it into the drive, but was surprised to find it was already full of data. What, he wondered, could be on it? So he looked at it.
You can guess what he found.
"It was filled with porn," says Dave Amsler, president and CIO of Foreground Security, which was called in to handle the incident. "And so were dozens of other 'blank' tapes in the admin's desk. There was nothing illegal on any of the tapes, thank goodness. Still, he was terminated on the spot."
Yet that's hardly the worst Amsler has seen in his 14 years with Foreground, which provides managed security services for major U.S. corporations and government agencies. Amsler says he's been called in to deal with porn problems for at least 10 clients. Twice he found IT employees running adult sites on company servers. In those cases, the personnel suddenly found themselves with lots more spare time to pursue their hobbies.
Porn filters are useless against this kind of behavior because the IT guys know how to turn them off. Even when organizations have strict policies and filters in place, high-level admins often exempt themselves from these restrictions, says Amsler.
Sign up for CIO Asia eNewsletters.