LHC@home

Every CMS task I received since 22 Oct 2022, 20:21:30 UTC (10 of them until I stopped asking for them) has ended in a computation error after only about 2 min 30 sec. Their stderr logs all show this:

2022-10-22 21:03:43 (1968): Guest Log: Ncat: Version 7.50 ( https://nmap.org/ncat )
2022-10-22 21:03:43 (1968): Guest Log: Ncat: Connection timed out.
2022-10-22 21:03:43 (1968): Guest Log: run 2
2022-10-22 21:03:43 (1968): Guest Log: Ncat: Version 7.50 ( https://nmap.org/ncat )
2022-10-22 21:03:43 (1968): Guest Log: Ncat: Connection timed out.
2022-10-22 21:03:43 (1968): Guest Log: run 3
2022-10-22 21:03:43 (1968): Guest Log: Ncat: Version 7.50 ( https://nmap.org/ncat )
2022-10-22 21:03:43 (1968): Guest Log: NCAT DEBUG: Using system default trusted CA certificates and those in /usr/share/ncat/ca-bundle.crt.
2022-10-22 21:03:43 (1968): Guest Log: NCAT DEBUG: Unable to load trusted CA certificates from /usr/share/ncat/ca-bundle.crt: error:02001002:system library:fopen:No such file or directory
2022-10-22 21:03:43 (1968): Guest Log: libnsock nsi_new2(): nsi_new (IOD #1)
2022-10-22 21:03:43 (1968): Guest Log: libnsock nsock_connect_tcp(): TCP connection requested to 137.138.156.85:9618 (IOD #1) EID 8
2022-10-22 21:03:43 (1968): Guest Log: libnsock nsock_trace_handler_callback(): Callback: CONNECT TIMEOUT for EID 8 [137.138.156.85:9618]
2022-10-22 21:03:43 (1968): Guest Log: Ncat: Connection timed out.
2022-10-22 21:03:43 (1968): Guest Log: [ERROR] Could not connect to vocms0840.cern.ch on port 9618
2022-10-22 21:03:43 (1968): Guest Log: [INFO] Testing connection to WMAgent
2022-10-22 21:03:44 (1968): Guest Log: [INFO] Testing connection to EOSCMS
2022-10-22 21:03:44 (1968): Guest Log: [INFO] Testing connection to CMS-Frontier
2022-10-22 21:03:45 (1968): Guest Log: [INFO] Testing connection to Frontier
2022-10-22 21:03:45 (1968): Guest Log: [DEBUG] Check your firewall and your network load
2022-10-22 21:03:45 (1968): Guest Log: [ERROR] Could not connect to all required network services
2022-10-22 21:03:45 (1968): Guest Log: [DEBUG] Volunteer: hadron (806228)
2022-10-22 21:03:45 (1968): Guest Log: [INFO] Shutting Down.
2022-10-22 21:04:14 (1968): VM Completion File Detected.
2022-10-22 21:04:14 (1968): VM Completion Message: Could not connect to all required network services

I am quite sure there are no network or firewall issues on my end, since ATLAS and Theory tasks do not have any problems. It may be some obscure setting I need to make that I never found documented anywhere. However, since this has never occurred previously, I'm inclined to believe it is an error that has unexpectedly occurred at vocms0840.cern.ch

it isn't on your end and is happening here and at -dev
Suspend your CMS tasks.

VM Completion Message: Could not connect to all required network services

Looks like server side issues are solved and fresh CMS tasks are working fine again.

no kidding

Looks like server side issues are solved and fresh CMS tasks are working fine again.

Yes, I don't know what the problem was (I was asleep!) but someone seems to have fixed it. The VM with connection problems - vocms0840 - is our HTCondor server.

---