View topic - Website outage

Website outage

Announcements, complaints and other discussion about all aspects of our website.

Website outage

Postby GregS » Mon Feb 17, 2014 1:54 pm

I would like to apologize to everyone who was affected by last week's outage of tuc.org. The hosting company is moving to a new, improved data centre, but in the process of moving this server, it seems that something went wrong, because when it was powered up at the new location the hard drive was slow to respond. It took some time to convince them that it was a real problem, then it took longer than it should to make a copy of the hard drive, then the new drive wouldn't boot. They started a second copy of the drive, and that was taking even longer, so at my request they cancelled that and instead set up a fresh copy of the operating system, which they then restored the off-site backup onto. But the guy doing that followed the wrong procedure the first time around, adding more downtime, and then when it was finally done, it turns out that they neglected to refresh the off-site backup from the old drive as I had requested, so some data was lost. And along the way there were various delays where I'd finally have to call them to see what the status was and remind them to get going on the next thing.

I'm surprised that they had the problems that they did, as their support to date has always been first-rate and prompt. They are, as I said, in the midst of physically moving all of their servers, which is going to take a lot of their time, and their staff is physically split between the two facilities right now, which caused other issues, and there was a major snow storm there which prevented the guy that best knew how to solve the final problem from just going directly to the server and doing what needed to be done (he was in the old facility). But when a client has been down for a couple of days, you really need to prioritize them and pay close attention to get them back up ASAP, and that didn't happen here.

I have now gained access to the old hard drive, and restored some of the key lost data, specifically emails, registrations and BBS posts that happened during the window between the last backup and when the site was finally taken down. I can also look at score submissions and schedules, but it seems likely that the easiest way to "recover" those is just for captains to re-submit the scores and conveners to recreate the schedules. If there are other things you find missing, please let me know (email webmaster@tuc.org is probably best), and I'll see what I can do.

Thank you for your patience and understanding through this time!
Did you get that thing I sent you?
User avatar
GregS
TUC Webmaster
 
Posts: 1291
Joined: Thu Apr 01, 2004 1:45 pm

Postby Edk001 » Mon Feb 17, 2014 2:43 pm

I read the second word in your subject title as "outrage". Glad nobody was outraged at the outage.
Are you photo worthy? Then lay out for the camera!
Edk001
 
Posts: 693
Joined: Thu Nov 23, 2006 12:50 am

Postby GregS » Mon Feb 17, 2014 5:08 pm

Edk001 wrote:I read the second word in your subject title as "outrage". Glad nobody was outraged at the outage.

I'm sure some were, but it's not like they could post on the BBS about it at the time...
Did you get that thing I sent you?
User avatar
GregS
TUC Webmaster
 
Posts: 1291
Joined: Thu Apr 01, 2004 1:45 pm

Postby DjDATZ » Tue Feb 18, 2014 2:47 pm

That sounds like an ordeal and a half! Glad you got it sorted out and the site's back up!
Dario

Aka the one with the Lego shorts and muscle tights. Except the tights are retired and the Lego shorts come out only once in a while.
DjDATZ
 
Posts: 66
Joined: Tue Jan 18, 2011 3:53 pm

Postby larrypmac » Wed Feb 19, 2014 9:13 pm

Is everything up? Because my game tomorrow night are not yet scheduled, so I wondered if my volunteer commissioner was unable to get in and set it. This is the Thursday Monarch hat league.
larrypmac
 
Posts: 175
Joined: Thu Apr 06, 2006 7:17 pm

Postby larrypmac » Thu Feb 20, 2014 12:20 am

larrypmac wrote:Is everything up? Because my game tomorrow night are not yet scheduled, so I wondered if my volunteer commissioner was unable to get in and set it. This is the Thursday Monarch hat league.


Must be up now, schedule finally posted.
larrypmac
 
Posts: 175
Joined: Thu Apr 06, 2006 7:17 pm


Return to TUC Website

Who is online

Users browsing this forum: No registered users and 1 guest

cron