Where Anna the Piper geeks out
(Those of you who follow me on Facebook already saw me posting about this, but this is for everybody who missed the story!)
Thursday morning, around 4am, I got woken up of a sound sleep to discover that the power had gone out. How my sleeping mind knew this: the air filter had gone off.
I need the soft noise of the air filter to sleep–I have a very hard time sleeping in total silence. So when the filter went out and the room was in fact silent, that was enough to wake me up.
Current status of newmoon: fucked.
I had discovered that the reason the box wasn’t ssh’able upon bootup was that ssh was segfaulting. So were the imap and pop3 daemons, and all of them were bitching about libcrypto.so.0.9.8. Fine, I think, and as per 
solarbird
Which went swimmingly until the part where I tried to cut into the extraction from the tar file, which must have been before the damn files were actually extracted properly. Because when I copied them in on top of /usr/lib on newmoon and rebooted, they bitched about being “too short”. And now I can’t even log in to the damn box, even from the console. I try to log in as root, and I never get a password prompt, and it tells me “Invalid login”.
So now I’m going to have to create a Debian Live CD, in the hopes that booting from THAT will let me fix the damn system, and if that doesn’t work I’m going to have to ctrl-alt-fuckit and see if I can just upgrade to the new Debian 6 release. And that’s going to take oh so very long given that the Murkworks’ bandwidth is CRAP. And this is assuming we even keep door and lodestone running, given that we have a Wind Advisory up until 9pm this evening, with the potential of 50mph gusts.
All of which, given that a) I am fighting off a cold and really wanted to curled up under a blanket for most of this afternoon, and b) I wanted to be editing my novel today rather than doing emergency system repair, does NOT have me in a pleasant mood right now. I swear, right here and right now, that the next person who tries to preach to me about how awesome Linux is and how much better off we’d all be running Open Source software is going to get a keyboard shoved down their throat. When it comes to Open Source vs. The Damn Software Works Without Me Having to Google How to Do Every Step of the Process, you know what? I’ll take The Damn Software Works.
We just had a couple of power spikes at the Murk, like we do. This time though they appear to have taken out one of our UPSes–and it was the one that newmoon, our mail server, was plugged into. When I tried to bring newmoon back up, it worked for a while and then hung. I am not currently able to log into the system to restart its services.
So this means that as of this afternoon, mail services on murkworks.net are down. This impacts not only personal accounts but the various mailing lists we run. Please spread the word if you’re on any of our mailing lists, and if you have backup personal email accounts, let people know they should use them this weekend if they need to get in touch with you.
solarbird
Sorry about this, folks. :/
But at least this one was short.
Woke up around 5:30, and failed to get back to sleep in the Comfy Blankets of Comfiness, so around 6:30 I thought “screw it” and got up with the intention of having a nice leisurely breakfast and then heading out for work. So there I was boiling water for tea and opening up the laptop to start my morning perusal of the Intarwebz, when suddenly–PANG! NO POWER!
Couple of nervous-making flickers, too. So I went to shut down the server core, and while spazzkatsolarbird
So I headed on into work a little early, getting market berries on the way for breakfast nomming, and shortly after I got to work Dara jumped on IM to let me know that we were back. And that I’d accidentally shut the servers down for reboot, not shutdown. Oops. *^_^*;;
Anyway folks, those of you with murkworks.net-hosted resources, please to be doublechecking them and make sure that they’re functioning correctly. We should be fine, but never hurts to doublecheck. And oh yes, for those of you who log onto it, the MurkMUSH is up!
Back up a little behind schedule, but solarbird
We now return you to your regularly scheduled Internetz, already in progress.
The power came back on not long before it was time for me to get up anyway–which was long enough of a delay that I got to fail to go back to proper sleep for about three and a half hours, yay! Kept dozing and dreaming that the power was back on NO WAIT NO IT ISN’T no wait do I hear power trucks outside AUGH zzzzzzz.
Set the alarm on my iPhone to go off at my usual get up time, only to discover that my bedside clock had gotten fast by 20 minutes, so it actually woke me up first. Stumbled awake to get ready for work, resurrect the servers, and reset the blinking clocks.
Still don’t know what exactly happened; Puget Sound Energy’s automated message said they were on top of things and damage crews had been dispatched when I called around 3am. No notice on their web site though about what might have happened. I’ll have to see if I can find some news or something.
Anyway, newmoon, lodestone, and door are alive again, so all Murkworks.net services should be functional. Anybody with mailing list subscriptions or web pages or DNS or whatever on our site, if you see any issues, let me know!