[
Date Prev][
Date Next][
Thread Prev][
Thread Next][
Date Index][
Thread Index]
[
List Home]
|
[eclipse.org-committers] Database server outage -- download stats
|
Greetings,
About 24 hours ago our secondary database server suffered a segfault,
which required a hard reset. Unfortunately, that led to a corrupted
filesystem that required a fresh install of the underlying OS.
Not taking any chances with our data, we decided to wipe it clean and
re-sync all the data from the master. This is the first time we've had
to do this since 2007.
The server is now back up; however, for several hours a single database
server was powering our site, and it could barely keep up with the
demands placed on it, as we normally operate with three database servers
(DB server #3 suffered a series of hardware failures last December and
was retired).
The added stress of syncing 200GB of data made it impossible for me to
sync the massive download stats content while the eclipse.org website
was up. My only choices were to turn off www.eclipse.org for two hours,
or truncate the download stats. I've chosen to truncate the download stats.
I apologize for terminating this data. Hindsight being 20/20 I should
have provisioned a replacement third DB server sooner. This would have
allowed us to re-sync from a slave, preserving the download stats and
site performance. We'll work on that in the weeks to come.
If you have any questions or concerns, please don't hesitate to contact
Matt or I at webmaster@xxxxxxxxxxx
Denis
--
Denis Roy
@droy_eclipse
http://www.eclipse.org/
