secondary server is down

UnFreeZe Game Servers News forum.
User avatar
adminless
Site Admin
Site Admin
Posts: 5966
Joined: Thu Nov 03, 2016 19:05
in-game nick: not available
Location: Spain

Re: secondary server is down

Post by adminless »

yesterday night I finally finished checking and setting up/monitoring the broken server and now I just moved this site there so that means that then now all that is left is just to review some dns names and similar minor stuff and everything should be finally set with the "new" servers.
User avatar
adminless
Site Admin
Site Admin
Posts: 5966
Joined: Thu Nov 03, 2016 19:05
in-game nick: not available
Location: Spain

Re: secondary server is down

Post by adminless »

this morning I think that I finally completed at least the initial setup/monitoring/diagnosis for this new configuration so hopefully if later today or tomorrow I finally write that pending report for the "new" server then everything should be good to go for some new UnFreeZe games and a potential UnFreeZe season as planned in the coming months. from that I saw I think it finally turned out pretty good so considering all the combined resources (three dedicated servers in three different locations and providers, 1.35+ gbit of combined bandwidth, six different enterprise class or better hard disks from different providers etc etc) I assume that even if there's some outage/failure at some point this should continue to run more or less "as usual" somehow. I mean, as a matter of fact, this time a entire disk died out of nowhere and I didn't even need to go for the local emergency physical backup to rescue and for the most part there hasn't even really been "downtime" but more like "waiting" time so as usual check it out and if anything just let me know.
User avatar
adminless
Site Admin
Site Admin
Posts: 5966
Joined: Thu Nov 03, 2016 19:05
in-game nick: not available
Location: Spain

Re: secondary server is down

Post by adminless »

just reporting here that the now web/workstation server first disk has went offline again and it's not coming up after trying to bring it up. considering the disk this time was brand new that then finally means the fault with that server is at the motherboard southbridge (death/broken port/s). I just set it up to continue operating just through the secondary disk again for as long as it lasts in the meantime while I evaluate further steps. good news is that I already scouted a solid affordable replacement deal, but unfortunately it's sold out at the time of witting, and in any case I can always just migrate this server into any of the other main servers at any time if necessary so beside potential small cuts there should really not be downtime. as usual if anything I'll keep this updated.
User avatar
adminless
Site Admin
Site Admin
Posts: 5966
Joined: Thu Nov 03, 2016 19:05
in-game nick: not available
Location: Spain

Re: secondary server is down

Post by adminless »

as a update on this I think I have good news. after some days running this reworked web/workstation server here now just with one disk and after producing a bunch of videos as well it appears like the server is finally running stable (i.e. no more ata bus errors so far). additionally after further investigation I performed a couple of low level kernel controller resets this morning and it appears that brought back online the first disk again so it seems like the server can probably still be repairable/workable after all with some care. in light of that today not that there's a game scheduled for this night and tomorrow in the morning neither that thus there's going to be videos to produce here but after that I believe that then I'll brought back the raid array and will spend some time doing some further diagnosis which will take this server (and consequently the forum) down keep that in mind (approx cut time 1 hour). I mean, there's definitively something broken with it as this is the same exact system image I run in all the other servers and none of them exhibits such behaviour but as it's paid for in the meantime I'll keep to operate it. again if anything I'll just keep this updated.
User avatar
adminless
Site Admin
Site Admin
Posts: 5966
Joined: Thu Nov 03, 2016 19:05
in-game nick: not available
Location: Spain

Re: secondary server is down

Post by adminless »

server repair and check up complete now so now everything should be up and running again with both disks properly raided and functional again. additionally I updated to latest kernel/system just to be safe so we'll see how long it lasts this time. by the looks of it seems like probably a faulty southbridge sata controller due age that just struggles to raid two last gen hdds as now that I recall it this server ran without problems for about a year when the installation was split across both disks and the problems seems to have arose once I mirrored them instead. anyway as long as it can be repaired/workaround as it has been this time it should then not be such a huge problem.