Unplayable lags on server for some players

[serca]

Today I started to get very big lags playing on server. Sometimes cannot even connect (getting infinite 'awaiting connection' message). The same reported by some other regular players.
My internet conenction is ok, and my server monitoring app shows stable typical ping to the server with no lags.
So, I guess the problem is on the server side.

[WhoAmI]

Admin, I'm reporting a problem. Players have large lags on the server. Not everyone but most players. Many people also cannot join the game. I also have the same problem. I thought it was my internet's fault but many people have the same problem.

Is it because of adding a new map or some tests?

[WhoAmI]

It is true. We added the topic at the same time Confirm.

[serca]

The funny thing is that we all having lags in different times. I.e. when you have it, I don't have it.

[WhoAmI]

Exactly. It surprised me too, because I thought that only I got lag.

[...jutuli...]

I have got the lags too, it started yesterday evening, after few games at srv. 2, then I quit. Today on srv. 1 one game OK, then complete stuck on conn. interrupted. Checked wifi, seemed OK. But when I started play thru LTE, it was nearly OK for few games. But yes, other players complaining too. Maybe some routing problems at some nodes, not at server, but dunno.

[adminless]

I'm still investigating more but it was almost with total certainty some short of network mass attack or something like that.

[adminless]

I took some initial steps on whatever came out, I don't know if the quality of service is restored? initially server health check up returned no issues so it's gonna be a network issue (either of the card itself or very likely external). I'll keep investigating.

[adminless]

ok, it seems, it's still happening as I see it now, it's not related with the server itself, server seems fine now as far as I can tell (both health as well as net). as I dig it seems like probably the hosting switch may be saturating/broken/dying. I'm opening complain now at the company. thanks for report.

[serca]

Yep, it's still happening. Getting infinite 'Awaiting Gamestate' message and cannot connect to the server right now.
The server monitoring app (that sends statusResponse packet on the same UDP port) has no problem at all while monitoring the server.

Thank you for support.

[adminless]

well issue is already escalated at the company, checked thoroughly the server and found no issues, so waiting for them to dig their end and we'll see. I'll issue tomorrow a hard reboot in the morning (not gonna change anything but anyways) and I'll wait and see how this evolves if on the next days it doesn't get shorted then unfortunately I believe that there will be no option but to move the servers somewhere else. fortunately there's already another new backup/spare server I was working on the background lately to replace the current web server as well as the low latency routes to save up costs and better efficiency but unfortunately it's still to soon to deploy this there and it's not really intended as a main time replacement for the game servers. anyways for today I can't do more, sorry for the inconvenience and I'll try to short this out as soon as possible.

[adminless]

ok, server hard cycled now (not that I expect that to change anything but anyways) and I'm waiting for the company check up (that I also expect not to really bring anything but anyways). if this miserably fails today all that is left prior move this somewhere else as soon as possible (which obviously takes quite some time) it would be request a hardware change (which is gonna be the only possibility left) for tomorrow or so at these hours. well, try for today and let me know how it's doing.

[adminless]

another possibility could be some bizarre bug as hardware server is gonna be fine I believe. I'm also digging that now.

[adminless]

well we're still pending to further deeper investigation but it seems like the hard cycling the server and escalate the issue did somehow improve the situation. as I see it now after a couple of hours of the server at mid/high loads this morning things seem back to normal so let me know how it does for you today. in the long run like I mentioned I'm gonna be looking forward add a backup server (which will replace some of the actual servers, not the game servers) in the event something like this happens as temporal solution.

[adminless]

btw as I see it surprisingly there was no more network shortages today (other than just the usual lagged clients, occasional spikes etc etc), please confirm me and I'll report back to the company.