Today something fubbed in our raid disk arrays that caused them to start resynchronizing themselves. This is a heavy process and severely limits throughput.
The load however kept increasing drastically and when we reached pop cap the throughput wasn’t enough to handle the resynch along with the pop cap. This caused the first crash.
We did some reboots that failed due to the massive load of members connecting at the same time, which is about the most intensive database process the player does, so we ended up starting with a limit, that we set to 5, 15 and 20, now ending it at 25.
We will slowly try to increase the number of members allowed but we will keep it limited until the raid array resynchronization process has finished. After that, we should hopefully return to the performance we witnessed last night … and hopefully able to double that – soon.
We’re pretty much prepared for the next ice-age down here in hell now, nothing short of an apocalypse would shake us up. We sincerely apologize for all this mess and appreciate your patience in this matter.