# Database lost?



## Thanee (May 10, 2006)

Havn't seen any definite announcements about this (might have missed them, of course).

Is there any hope that the database can be restored, or is the current EN-World what we are going to continue with?

Bye
Thanee


----------



## Keith Robinson (May 10, 2006)

Thanee said:
			
		

> Havn't seen any definite announcements about this (might have missed them, of course).
> 
> Is there any hope that the database can be restored, or is the current EN-World what we are going to continue with?
> 
> ...




I'm pretty certain that this is it.  I don't think they'd upload an old database, do loads of work on it (as well as letting users update their own stuff) and then upload a newer version, over-writing everything.

Just my opinion.


----------



## Iku Rex (May 10, 2006)

I doubt if the database is completely lost. (Unless it's been deleted/overwritten.) The boards were up for a short while after about a day of downtime (not sure about the exact time), and the latest posts and threads were available then. I even made a new post. Clearly it wasn't a complete meltdown.

My guess is they went with the simple "start over" solution rather than try to fix whatever it was that was causing the problem(s).


----------



## buzz (May 10, 2006)

Might I ask why it is that the most recent database backup is six months old? I'm honestly not trying to be snarky; it just seems like there should be more frequent backups.


----------



## shaylon (May 10, 2006)

buzz said:
			
		

> Might I ask why it is that the most recent database backup is six months old? I'm honestly not trying to be snarky; it just seems like there should be more frequent backups.




According to Spoony this was the last one they did, back when he was doing admin work for ENWorld.

I think given the situation they will probably do a better job of backups from now on.  A lot of times it takes something like this to change people's minds about processes.


----------



## buzz (May 10, 2006)

shaylon said:
			
		

> According to Spoony this was the last one they did, back when he was doing admin work for ENWorld.
> 
> I think given the situation they will probably do a better job of backups from now on.  A lot of times it takes something like this to change people's minds about processes.



I hope so. I know this is just a gaming site, but given its preeminence among gaming sites and the sheer amount of content, backups should be done *weekly*.


----------



## freebfrost (May 10, 2006)

Yeah, I'd have to agree.  For such a large site, the level of disaster planning is obviously lacking.  I have a small website that I use for my gaming groups and even I do backups every two weeks or so. 

Losing 4,000 accounts?  Having to reload Community Support information manually?   Five months with no backup?

No business (and yes, EnWorld is a business at this point) can afford to run this way.  Luckily EnWorld has fostered a lot of goodwill while growing in the community, but they can't continue in such a haphazard way.

I'm hoping we'll see a post by Morrus and Co. that will address this soon.


----------



## Morrus (May 10, 2006)

There is a possibility we can use a much more recent backup, but we need to check carefully it is not corrupted at all.  We only tracked this backup down in the last few hours (I didn't even know it existed!).  Hang tight for more news on that!


----------



## Arnwyn (May 10, 2006)

*ahem* On the bright side, ENWorld is now running blazingly fast...


----------



## freebfrost (May 10, 2006)

Arnwyn said:
			
		

> *ahem* On the bright side, ENWorld is now running blazingly fast...



(chokes on coffee)


----------



## Thanee (May 10, 2006)

Morrus said:
			
		

> There is a possibility we can use a much more recent backup, but we need to check carefully it is not corrupted at all.  We only tracked this backup down in the last few hours (I didn't even know it existed!).  Hang tight for more news on that!




That would be cool! 

Even if it cannot be used for the actual forum, maybe it is an option to clone the forum and put that recent backup up as read-only for a week or so, so people can fetch the lost posts (many aren't found in the search engine cache) from there and save them!

Bye
Thanee


----------



## GnomeWorks (May 10, 2006)

I concur with Thanee's idea. If the backup from the 8th is still corrupt, making it available as read-only so we can copy things over would be greatly appreciated.

Just speaking from the PbP forums, we lost a ton of information over there... it'd be a lot easier on us if we could at least cut-and-paste it from a read-only copy of the boards from a couple days ago. Trying to recreate everything is something of a pain.


----------



## sniffles (May 10, 2006)

Same goes for story hours as PBP. My two story hours have lost months of information - I have one of them backed up myself but not the other (my bad; I should have known better).
And another story hour I participate in but don't "own" is completely gone. 

BTW, does this incident have any impact on supporting memberships? I signed up during the discount period and I don't seem to have any supporting member privileges.


----------



## BSF (May 10, 2006)

Morrus said:
			
		

> There is a possibility we can use a much more recent backup, but we need to check carefully it is not corrupted at all.  We only tracked this backup down in the last few hours (I didn't even know it existed!).  Hang tight for more news on that!




Ouch!  That could be a messy thing to try to integrate.  Good luck with it and if I can help somehow, don't hesitate to ask.  *shrug*  Not sure I can help, but the offer is out there.


----------



## Knightfall (May 10, 2006)

Thanee said:
			
		

> Even if it cannot be used for the actual forum, maybe it is an option to clone the forum and put that recent backup up as read-only for a week or so, so people can fetch the lost posts (many aren't found in the search engine cache) from there and save them!




I think this would be the best option. People are already updating their important threads, and if some may feel robbed of allthat work if it is overwritten. Running a cloned backup would allow people to cut and paste only what everyone really needs.

The "new" EN World messageboard shouldn't be overwritten, just to try and save the "old" board. Are we so desperate to have it all back that we can't move forward without losing more time over it?

I say, let it be, and just provide the cloned board, if possible, for those that really need some of those old posts (i.e. the PbPers).

Cheers!

Knightfall1972
Moving forward


----------



## Bront (May 11, 2006)

I agree on the read only archive.  Perhaps something that would allow us to pull the formating off the posts as well, though if not, that'd be undestandable.

Or perhaps a way to request or import individual threads.


----------



## MavrickWeirdo (May 11, 2006)

Latest word (it's not the best news) http://www.enworld.org/showpost.php?p=2817332&postcount=13


----------



## Thanee (May 11, 2006)

Knightfall1972 said:
			
		

> The "new" EN World messageboard shouldn't be overwritten, just to try and save the "old" board. Are we so desperate to have it all back that we can't move forward without losing more time over it?




Yes, actually, I would say, that losing the little done in the last few days is clearly worth it (especially since there was an advance notice about it already), if a stable backup from may can be used. Having the old posts somewhere backed up is at least something, but having the threads intact (mostly speaking of PbP) is worth a lot.



> I say, let it be, and just provide the cloned board, if possible, for those that really need some of those old posts (i.e. the PbPers).




That's, however, a good option, if the backup does not work, as said above.

Losing the posts completely would be harsh for many PbPs, since they are also used as an archive of what happened and (at least in my case) often checked back.

But as it seems from the link above, that's what's going to happen... :\

Bye
Thanee


----------



## Knightfall (May 11, 2006)

MavrickWeirdo said:
			
		

> Latest word (it's not the best news) http://www.enworld.org/showpost.php?p=2817332&postcount=13




That's too bad.


----------



## Pinotage (May 11, 2006)

Darn! At least it's still running, even if it is older posts.

Pinotage


----------



## jcfiala (May 11, 2006)

I just wanted to throw in my own two cents - as someone who does both programming and dba work, I'm shocked at how old a backup this is.  What am I donating money for if the admins aren't taking care of what I'm paying for?


----------



## Twin Rose (May 11, 2006)

jcfiala said:
			
		

> I just wanted to throw in my own two cents - as someone who does both programming and dba work, I'm shocked at how old a backup this is.  What am I donating money for if the admins aren't taking care of what I'm paying for?




There have indeed been other backups, but those had some problems as well.  Backing up the entire database takes something like 2 gigabytes worth of space - approximately 20 hours to download even at high speed.  Obviously, this is something that can't be done every week.  

In the past, a backup every 3 months or so seems to have done the trick - but this time, there's been problems for the last month or so with the database.  The most recent backup was, in fact, also a bit corrupt - and I suspect it could lead to another crash in a couple weeks or so.

Also remember that the previous admin left us recently, and I'm still rather new at the gig - if assiduous in it.  

What donations go for is the hosting and bandwidth.  Even still I'm planning to donate one of my other hard drives to the server so we have more free space for more regular backups.  (If I can manage it, i'll be trying to keep weekly backups instead).

In the next few months, I'd prefer to buy a very large hard drive to put in down there.  Hopefully, that means through money gained at the EN World GameStore.


----------



## GlassJaw (May 11, 2006)

Twin Rose said:
			
		

> Backing up the entire database takes something like 2 gigabytes worth of space - approximately 20 hours to download even at high speed.




2 gigs isn't that much space - why does it take so long?  That doesn't seem right.



			
				Twin Rose said:
			
		

> What donations go for is the hosting and bandwidth.  Even still I'm planning to donate one of my other hard drives to the server so we have more free space for more regular backups.  (If I can manage it, i'll be trying to keep weekly backups instead).
> 
> In the next few months, I'd prefer to buy a very large hard drive to put in down there.  Hopefully, that means through money gained at the EN World GameStore.




Why not pay to have the site hosted rather than spending the money on hardware?  That seems a lot more reliable to me.  I was in the network industry a few years ago but haven't really kept up on fees but I would guess that hosting a site that is primarily a message board (albeit a somewhat active one) shouldn't be too server intensive for even an average webhost.


----------



## James Heard (May 11, 2006)

I think ENWorld eclipses the "somewhat" active status by quite a lot. Crothian's posts alone would tally up the sum total of most of the most active of the lesser boards I take part in, and maybe beat them out. It might be worth it though, to try batting doe eyes at a corporation that really DOES have a lot more traffic. I'm picturing Google ads


----------



## IronWolf (May 12, 2006)

I will say up front that its always easier to be afforded the luxury of an armchair quarterback than to be the one that actually is doing the work.  I certainly don't have all the details of the situation, which can certainly change observations to one way or another.



			
				Twin Rose said:
			
		

> There have indeed been other backups, but those had some problems as well.  Backing up the entire database takes something like 2 gigabytes worth of space - approximately 20 hours to download even at high speed.  Obviously, this is something that can't be done every week.




As Glassjaw said, 2GB isn't that much space really.  Is this a compressed or uncompressed backup?  

20 hours to download 2GB?  Someone in another thread I believe mentioned that seems like a long time to download.  I don't know the bandwidth allocation on the hosted side for EN World though, so maybe it is the bottleneck.  Most cable modem connections and even a fair number of DSL connections should be able to come in well under 20 hours.

Downloading aside, I would think you could keep at least a couple of backups on the server itself.  Then those can be shuffled off to an offsite location automatically as needed for extra safety.  Even if it took 6 hours in the middle of the night to shuffle them offsite, that shouldn't be too big of a deal.  All of this would be automated, so once its setup it wouldn't even require that much maintenance save for a test restore here and there to be sure the backups were good (and even that can be automated to some degree).



			
				Twin Rose said:
			
		

> In the past, a backup every 3 months or so seems to have done the trick - but this time, there's been problems for the last month or so with the database.  The most recent backup was, in fact, also a bit corrupt - and I suspect it could lead to another crash in a couple weeks or so.




If decisions were made that 3 months loss of posts, new memberships, etc was reasonable then I can see why that might have done the trick.  As with everything one must weigh the options that one has and determine what acceptable risk is.  I would think though that with existing resources one could pull off a weekly backup.  As mentioned before though, I am only operating on knowledge I have of this place picked up from various threads over the past 2 years.


----------



## buzz (May 12, 2006)

GlassJaw said:
			
		

> Why not pay to have the site hosted rather than spending the money on hardware?  That seems a lot more reliable to me.



The bandwidth, traffic and services offered on ENWorld easily merit a dedicated server. A hosted solution, even a leased server, with consulting, would probably cost way too much.


----------



## Morrus (May 12, 2006)

buzz said:
			
		

> The bandwidth, traffic and services offered on ENWorld easily merit a dedicated server. A hosted solution, even a leased server, with consulting, would probably cost way too much.




Yep.  A co-loc dedicated server is waaaaay cheaper.  EN World used to be hosted on a leased server, but the cost became excessive.


----------



## BSF (May 12, 2006)

IronWolf said:
			
		

> Downloading aside, I would think you could keep at least a couple of backups on the server itself.  Then those can be shuffled off to an offsite location automatically as needed for extra safety.  Even if it took 6 hours in the middle of the night to shuffle them offsite, that shouldn't be too big of a deal.  All of this would be automated, so once its setup it wouldn't even require that much maintenance save for a test restore here and there to be sure the backups were good (and even that can be automated to some degree).




That might be a worthwhile approach.  Bring the boards down long enough to backup the database to disk; bring the boards back up; then download the backup on disk.  Minimum time for the boards to be down and perhaps some pain with the download competing with traffic.   But that might be worth it.  

Of course, I am not sure how admins access lower level functions.  The options will be much different if you have full CLI access as opposed to access via a browser tool interface.  But the idea is certainly a decent option that Morrus could consider.  

As you said though, most of us don't really know what's going on with the backend.  Some of our suggestions might not make sense, or might be a heavy burden for volunteers.


----------



## Staffan (May 12, 2006)

Twin Rose said:
			
		

> There have indeed been other backups, but those had some problems as well.  Backing up the entire database takes something like 2 gigabytes worth of space - approximately 20 hours to download even at high speed.  Obviously, this is something that can't be done every week.



2 gig taking 20 hours? If I have calculated correctly, that's something like 30 kB/s, which is pretty much what I get with low-end DSL. A 10 Mbit connection (which is what I would consider "high speed") would get it done in half an hour.


----------



## BSF (May 12, 2006)

Staffan said:
			
		

> 2 gig taking 20 hours? If I have calculated correctly, that's something like 30 kB/s, which is pretty much what I get with low-end DSL. A 10 Mbit connection (which is what I would consider "high speed") would get it done in half an hour.




Geographic diversity and personal budgets stand in your way.  My last house couldn't even get DSL and it was in a well-populated area of the city.  I could get ISDN or I could get cable internet access.  I don't watch much television and just to get hooked up for cable internet was $60+/month.  In short, not an expense I would want to afford.  

My new house can get DSL and we do have that.  Even so, it is considerably more expensive than comparable DSL rates in other areas of the country and world.  

I don't know what anybody's personal budget and geographic limitations are for greater than modem access is, but it is entirely possible that Twin Rose, MM and others would be administering from a DSL connection.


----------



## jcfiala (May 12, 2006)

BardStephenFox said:
			
		

> Geographic diversity and personal budgets stand in your way.  My last house couldn't even get DSL and it was in a well-populated area of the city.  I could get ISDN or I could get cable internet access.  I don't watch much television and just to get hooked up for cable internet was $60+/month.  In short, not an expense I would want to afford.




I've got cable internet and a 200 GB external hard drive for my birthday.  I'm willing to help on setting up some better way of backing up data.  I've worked with mySQL, Microsoft SQL, and PHP in the past.


----------



## Thanee (May 12, 2006)

I'd do monthly backups and maybe (if possible) bi-weekly partial backups of the subforums only, where the content is more trouble to lose (pbp, story hours and stuff like that).

I guess the long time for the backup is not because of the transition speed for 2 gigs, but rather for the backup data to be created by the db.

Bye
Thanee


----------

