NRPEListener stops accepting connections after 30 minutes if NSCAgent is enabled

[srunschke]

Following scenario:

Windows 2003 R2 SP2 German NSClient++ 0.3.6 MSI install NRPE 2.12

Server has been running 0.3.0 for ages, I just upgraded to 0.3.6 for checking out NSCAgent capabilities, since I plan to do massive performance counter checks on that machine.

I'm running 5 NRPE checks against the machine - all fine. Now I enabled 2 NSCA checks to report to our NSCA server - all fine.

But suddenly after around 30 minutes, the NRPEListener stops accepting connections and check_nrpe just reports: "CHECK_NRPE: Error - Could not complete SSL handshake." or "CHECK_NRPE: Socket timeout after 10 seconds." I enabled debug=1, but sadly that shed no light. I see nsclient++ logging the checks and nsca submits for 30 minutes after a restart. Then suddenly the checks stop, but the nsca submits just keep working fine. At this point you cannot even stop nsclient++ anymore. "net stop nsclientpp" just keeps hanging and the service crashes into an unrecoverable state and I need to reboot the server to get it working again.

When nsclient++ stops responding to NRPE requests, the systemlog shows the following:

Anwendungspopup: Microsoft Visual C++ Runtime Library: Assertion failed!

Program: C:\Programme\NSClient++\NSClient++.exe File: .\trace\trace.cpp Line: 924

Expression: b

Anything I can do to help fix the issue? NSCAgent capability would be much, much appreciated! I'll attach a comment stripped down version of my nsc.ini.

[srunschke]

stripped down version of nsc.ini

[mickem]

Try the nightly build, it has a fix for the NSCA "hang" issue...

0.3.7 will hopefully be out soon, I am ill at the moment so I dont have time to fix it but hopefully I will feel better next week.

MickeM

[srunschke]

Tried - but the nightly build doesn't even install :-/ It seems it tries to modify the firewall and fails somehow, gonna attach the screenshot of the error.

[srunschke]

Errorbox during install of nightly-build

[srunschke]

Ok, I found the problem. The 0.3.7 nightly wants to fiddle with the windows firewall api, but the firewall service is completely deactivated on this system. nsc pukes on that and the install fails. Maybe you should check if the firewall is present at all before trying to modify it. :)

[srunschke]

Tested the nightly build - same problem, different errormessage in the systemlog:

Anwendungspopup: Microsoft Visual C++ Runtime Library: Assertion failed!

Program: C:\Programme\NSClient++\NSClient++.exe File: .\util\locale.cpp Line: 190

Expression: b

When I connect to the admin console, I can see the popup window. Clicking on "Retry" nsclient++ crashes. I'd be happy to provide more info or tests, but I currently don't know how...

If you want me to try anything else, just gimme an hint.

[mickem]

about the fireall... yes I know will be fixed (when I am well again). You can install the 0.3.6 version and replace the binaries to get around it.

and crashes?!?!

That is really bad. Strange about asserts as I dont really use them any longer so I am a bit doubtfull. which version are you using?

MickeM

[srunschke]

This happened with 0.3.6 and 0.3.7 nightly build from 19.07.2009 (disregard the comment about the 17.07. NB I mentioned early - it was a typo).

I am going to try and reproduce it with a fresh installed machine, maybe it's a glitch in windows on the particular machine.

nsclientpp crashes as soon as I hit "Retry" on the popup from windows. The one that shows you "error in application blablabla. Ignore - Retry - Cancel". I haven't yet tried what happens if you hit ignore. For now I just deactivated nrpelistener.dll and switched all checks on that machine to NSCA and it runs fine without errors so far.

Gonna come back after testing.

[bohboh]

I got the same issue with the latest nightyl build 12-19-2009.

[mickem]

Presumably fixed in 0.4.0 (as this has been rewritten)