Taking Out Sun Microsystems. By Accident.
Here is a little story about how I accidentally took down Sun Microsystem’s marketing department.
Once upon a time, back in 1997, I worked for a small company known as Sun Microsystems. They were hot shit then. For me, getting a job there was an enormous thing. Sun had been the be-all and end-all of unix-style computing: the holy grail, as it were.
I was hired on a contract to be the NerdBitch for the marketing department. Back then, this was called “webmaster” but that term has since seen pollution and has fallen out of favor. I had many and varied duties, but my main ones were overseeing the care and feeding of two major, huge servers used by the marketing department: rock-n-roll and infobahn. (dot sun dot com). rock-n-roll was a huge machine. It was the primary server for all of Sun’s marketing.
I was “root” on these boxes.
(For the non-nerds, that means that I was the equivalent of “God” on these two major computers that ran everything).
From time to time, we’d have contractors come in to do small gigs. Usually, a contractor would get a full @sun.com account. However, sometimes the IT department was slow on the uptake and the contractor needed to be productive immediately. So, in these rare instances, I would simply create a user account for them on rock-n-roll, independent of the global account structure. They would then log in directly to rock-n-roll through via XDM from a workstation.
So it goes.
One day, a contractor of this type got fired. For cause, actually, which is where it gets goofy. At the time, if someone got fired for cause we would isolate the machine they were working from for later forensics. So, she got the boot, and I had to shut the machine down that she was working on.
Only, I forgot that she was remotely logging into rock-n-roll. My password was the same on all machines. I seriously forgot that she was remote, so I comandeered her console, grabbed a window, logged in as myself, and typed the following:
shutdown -h now
That turns the computer off. And normally I’d expect to see the workstation go blank and shut down.
That’s not what happened.
2 seconds later I realized that I’d shut down rock-n-roll, a machine that hundreds of people depended on, instead of her workstation.
This would normally have been recoverable except that rock-n-roll didn’t come back up. She stayed down, even after the IT department tried to bring her back up. Her disk controller had fried out a couple weeks back and no one knew.
So the marketing server was down for about five business days because they needed to get a new part and didn’t have one on hand.
I was not fired, though I have no clue why I wasn’t.
End of Line.