That accursed server at the heart of our Enco network has given us a series of bizarre, traumatic experiences, but none quite so weird and infuriating as this week’s situation.
It started Monday, when I moved the Beast (our standby server) from its old rack space to a new space underneath the main server to make room for the fancy new replacement main server. Shortly afterward we started hearing complaints about audio playback stuttering and behaving strangely in the production studios. We checked the usual error logs and indicators and what-not and couldn’t see anything out of the ordinary. We checked the connectors and lights and cable paths, but everything looked fine.
More complaints came in during the day yesterday. Again, we checked everything we could think to check. We had Enco specialists on the phone and dialed into our system to check everything they could think to check. Still, no problems could be found.
Complaints were widespread this morning, so we began an even more intensive search for the cause. While, for the fifth time in two days, monitoring the SCSI drive array controller… between blinks of the eye… two of the drives went “bad.”
Huh?
So we ended up downing the server, after some tinkering and deliberation. When we brought it up again, six drives were marked bad… out of eight.
Huh?
Going into the array controller’s configuration software, it was indicated to us that all eight of the hard drives were now offline and unusable.
HUH?
We tried everything. We stole the cables from the new server. We removed the drive enclosures and re-seated them. We completely power-cycled every part of the server system. No matter what, the drives were all coming up as Offline.
Great, we’ve lost the main Enco server array. Again. For the Nth time. Argh. And as always, the array has impeccable timing: In two weeks we were to have migrated gracefully to the new server.
So right now we’re on the backup, aka Beast. Tomorrow we’ll call Enco’s tech support and figure out the best way to get us online with the new server ASAP.
But wait, there’s more!
For some bizarre reason, our locally-hosted Qwest Dex software decided to nag everyone who uses it, insisting they download the latest version directly to their computers. Argh! I don’t think so! So between rounds with the Enco server I was frantically trying to update the Dex software on the server. I have the client updated, so at least now it looks better and nags in a reasonable fashion. Tomorrow I’ll try to get the actual updated phone book data from those fine, fine folks at Qwest.
But wait, there’s even more!
Before I could work on the Qwest problem, I had to get some semblance of a working office! That’s right, folks, when I got to work this morning I was greeted with an office crammed full of boxes. I couldn’t even see my desk, let alone get to it. A few hours’ (intermittent) unpacking later, and some cussing while I tried to figure out who the idiot is that neglects to label his network ports… oh wait, that would be me… anyway, I finally got my main workstation running. A few hours after that, I got my Linux box running.
There’s good news and bad news, here. Good news? I have a 19″ flatscreen on my Linux/shared workstation side. Bad news? My webcam died. Argh! So still no OfficeCam!
So, yeah. Tomorrow had better damned well better be a happy shiny fun day, or I’m going to go insane…
