About every month, even if active, it stops processing the checks it receives.

[Fantu]

Describe the bug
I don't know if this could be a problem caused by nsclient++, it's probably unrelated, but I'll try to explain the problem I've been encountering recently and can't seem to find the exact cause of.
About once a month, the last time being last night, nsca stops processing the checks sent by nsclient even though it remains active. Today I tried checking the server logs but found no errors or anything helpful. I tried to do a manual check in /var/run/icinga2/cmd/icinga2.cmd and it worked (icinga has most of the checks of two other types that have no problems, the nrpe and push api ones). I checked with tcpdump and nscp data arrives from clients on the port on which nsca listens.
If I restart nsca, it starts working again, but remains “stuck” for a long time at restart.
On the days when this happens, I haven't made any updates, reboots, or changes to the server, icinga, or nsca.
I've only had these problems in the last few months. The only significant difference is that I started using more up-to-date nscp, versions 0.9 and 0.10 (after using the same old version for 3-4 years).
Can nscp send data that causes nsca to freeze (not crash)?
Looking at the nsca git, I didn't find any bugs or commits that could be related to this problem. In fact, it seems to me that development has almost died out in recent years.
Has anyone else had the same problem and/or can advise me on how to solve it?

Client with nscp

• OS: Windows
• OS various, mainly Windows 2019
• Version: on all actually NSClient++ 0.10.9

Server with nsca

• OS: Debian 12
• nsca 2.10.2-2

[mickem]

Hmm...

Just thinking on the scenario one might wanna look into if there are lots of open TCP connections?

ss -t state established
ss -t state time-wait
ss -t state close-wait

NSCA connection should terminate pretty much "instantly" so if there a a lot of connections steadly growing due to NSClient not closing them maybe?

Which crypto do you use?
How many checks do you send "roughly" per minute/hour/*?
How many machines are there "roughly" targeting the NSCA instance?
Whats the check intevall?

[Fantu]

Thanks for reply.
There are only 7 Windows hosts monitored via nscp with passive nsca checks.
Of those, I only have 5 check services every 10 minutes.
The number of connections therefore does not seem huge to me, while with the other 2 types of checks (which do not use nsca) there are another 32 hosts monitored.
However, since the nsca service remains active even if it does not work and if you try to restart it, it takes a long time (perhaps reaching the timeout when shutting down), I suppose it goes into a loop or stalls due to some unforeseen event.
Currently, there don't seem to be many connections, and almost all of them appear to be unrelated to nsca. I should take a closer look when it happens again while the problem is ongoing.
During today's debugging, I didn't check the current connections, but only whether nsca was listening with ss -tulpn | grep nsca, and it was.
Regarding the type of encryption on nsca, it has actually changed. During the years when there were no problems, it was xor. The first time there was a problem, it was blowfish, and then after you documented it, I set RIJNDAEL-128 (14) on the nsca server side and aes256 on nsclient++ (which is still the one active today).