Goddammit.
At about 3:15 this afternoon, our Groupwise server’s main mail storage volume ran completely out of disk space. All attempts to salvage the situation with the server code running failed utterly and miserably.
As of this writing I’ve finally managed to semi-gracefully reboot the server, check and mount its volumes, delete an unneeded installer directory from the volume in question, and start a purge of sent items, trashed emails and (most importantly) oversized emails.
I don’t know if this is going to work, mind you. Groupwise does funny things with purged emails, like (for instance) hanging on to them even though you’ve clearly told it you don’t want them anymore.
It’s going to be a long, long evening here at the office, folks. I’ll update this entry as (or more accurately “if”) the situation progresses… or regresses as the case may well be. *sigh*
And in case you’re wondering: No, I don’t have disk space alarms set on this particular server. SNMP troubles, doncha know. The sort of thing I’d have time to tackle if, say, I had an assistant… but we all know that’s a pipe dream, eh?
Wish me luck. Lots of it.
UPDATE: The first purge took 3 hours 20 minutes and brought the server from 99% full (I started the purge after deleting the client install directory so I’d have that one percent to work with) down to about 93%. The second purge took exactly one hour and brought the server down to 65%. How, you ask, did this miracle occur during the second purge? Easy. I was deleting any email larger than 1.5 megabytes. That’s right, folks. About 30% of a 34 gigabyte drive was taken up with very large emails, essentially emails with large files attached. Lovely.
There’s gonna be some pissed-off folks around the building tomorrow when they realize that their mp3s, videos and PowerPoint files are all missing out of the email system… where they didn’t belong in the first place.
Me, I’m going home now. It’s about damned time. Two 12-hour days in a row is not my cuppa, baby.
Comments
6 responses to “When Email Servers Implode”
The only thing we can throttle is the maximum size of in- and out-bound attachments. Here’s what happens:
“Hey, email me that Powerpoint presentation, will ya?”
Yes, there’s a perfectly good network drive with lots of storage space. No, there’s no amount of /pointing this out/ to these salesfeebs that will work to change their ways. They /live/ in their email windows, so naturally all transactions should take place there! And gods forbid they /learn how to use the network storage effectively/!
This morning, however, will see (yet) another round of sharp realization as they discover (yet again!) what I’ve told them often: Your inbox is /not/ a storage medium!
Argh. This should be a /fun morning/, you betcha.
So basically, what you’re saying is: Your email server went postal. hehehehehe
Postal is describing the hell otherwise known as Groupwise mildly. Evil, I tell you, mass evil!
Groupswise is a collosal pain in the ass.
Perhaps it’s time to throttle down the maximum size of attachements allowed?
Can’t throttle it down, they still need to send that stuff back and forth. The problem is that they are not saving the files to their local machine and deleting their emails like good little minions.
You mean, I shouldn’t be storing that 8 gig email that included all those pics I wanted? Geez… Why didn’t you ever mention that? Who’d a thunk it?