LHC@home

https://lhcathome.cern.ch/lhcathome/result.php?resultid=419896635

2025-02-28 20:37:00 (114248): Guest Log: Ncat: Version 7.50 ( https://nmap.org/ncat )
2025-02-28 20:37:00 (114248): Guest Log: Ncat: Could not resolve hostname "vocms0267.cern.ch": Name or service not known. QUITTING.
2025-02-28 20:37:00 (114248): Guest Log: run 2
2025-02-28 20:37:00 (114248): Guest Log: Ncat: Version 7.50 ( https://nmap.org/ncat )
2025-02-28 20:37:00 (114248): Guest Log: Ncat: Could not resolve hostname "vocms0267.cern.ch": Name or service not known. QUITTING.
2025-02-28 20:37:00 (114248): Guest Log: run 3
2025-02-28 20:37:00 (114248): Guest Log: Ncat: Version 7.50 ( https://nmap.org/ncat )
2025-02-28 20:37:00 (114248): Guest Log: Ncat: Could not resolve hostname "vocms0267.cern.ch": Name or service not known. QUITTING.
2025-02-28 20:37:00 (114248): Guest Log: [ERROR] Could not connect to vocms0267.cern.ch on port 4080


root@pm108:~# nslookup
> server 8.8.8.8
Default server: 8.8.8.8
Address: 8.8.8.8#53
> cern.ch
Server: 8.8.8.8
Address: 8.8.8.8#53

Non-authoritative answer:
Name: cern.ch
Address: 188.184.77.250
Name: cern.ch
Address: 2001:1458:d00:3c::100:2f9
>
> vocms0267.cern.ch
Server: 8.8.8.8
Address: 8.8.8.8#53

** server can't find vocms0267.cern.ch: NXDOMAIN
>

Did you allow all required ports in your firewall?
These must be allowed for CMS:
80
8000
8080
443
9618
4080
1094

In addition vocms0267.cern.ch doesn't exist any more and should automatically be redirected to vocms267.cern.ch.

if the server can't look up the ip address from the most common DNS server 8.8.8.8 it really doesn't matter which port the application is trying to use. Anyway I have no blocks on any outgoing port. I've tried several other DNS servers but they all say the same.

This is servers I have in my home in Sweden, just to test I tried from a server I have in my home in Thailand and the result is the same vocms0267.cern.ch does not exist.

vocms267.cern.ch works perfect from both sites so it seems that the DNS redirect vocms0267.cern.ch to vocms267.cern.ch isn't working?

[Edit]
I did an ugly fix so the CMS app look-up the correct ip local for vocms0267.cern.ch. Now it seems to be working, it has been running more than 10 minutes. Before it crashed in less than a minute.
At least it will be working until you change the ip address :)

Everything was crunching flawlessly 24/7 for days on Atlas and CMS until we ran out of work yesterday.
All these new CMS tasks are aborting around the 2 min mark on all my computers.
I REALLY doubt all of a sudden volunteers need to go fidget with ports now.
I'll just go back on other projects til somebody figures this out like back in december with the empty 30min CMS tasks.


2025-02-28 17:25:45 (10704): Guest Log: Ncat: Could not resolve hostname "vocms0267.cern.ch": Name or service not known. QUITTING.
2025-02-28 17:25:45 (10704): Guest Log: [ERROR] Could not connect to vocms0267.cern.ch on port 4080

In addition vocms0267.cern.ch doesn't exist any more and should automatically be redirected to vocms267.cern.ch.

It seems like this is the problem. The redirect is not working.

I've slowly ramped up how many CMS jobs my server accept and is now running 4 jobs without problems. So my temporary ugly fix seems to solve the problem for now but it would be better if it was fixed on the cern side.

I’ve just upgraded VirtualBOX from 7.0.6 to 7.1.6 to revolve the 'Error while computing' after the powering on of the CMS VM.

@computezrmle
Today the issue is inside the CMS Virtual Machine regarding WMAgent, missing trusded CA Certificate and the “written-wrong” vocms0267.cern.ch server.

2025-03-02 04:03:51 (24108): Guest Log: [INFO] Testing connection to WMAgent
2025-03-02 04:03:52 (24108): Guest Log: [DEBUG] Status run 1 of up to 3: 1
2025-03-02 04:03:58 (24108): Guest Log: [DEBUG] Status run 2 of up to 3: 1
2025-03-02 04:04:10 (24108): Guest Log: [DEBUG] Status run 3 of up to 3: 1
2025-03-02 04:04:10 (24108): Guest Log: [DEBUG] run 1
2025-03-02 04:04:10 (24108): Guest Log: Ncat: Version 7.50 ( https://nmap.org/ncat )
2025-03-02 04:04:10 (24108): Guest Log: Ncat: Connection refused.
2025-03-02 04:04:10 (24108): Guest Log: run 2
2025-03-02 04:04:10 (24108): Guest Log: Ncat: Version 7.50 ( https://nmap.org/ncat )
2025-03-02 04:04:10 (24108): Guest Log: Ncat: Connection refused.
2025-03-02 04:04:10 (24108): Guest Log: run 3
2025-03-02 04:04:10 (24108): Guest Log: Ncat: Version 7.50 ( https://nmap.org/ncat )
2025-03-02 04:04:10 (24108): Guest Log: NCAT DEBUG: Using system default trusted CA certificates and those in /usr/share/ncat/ca-bundle.crt.
2025-03-02 04:04:10 (24108): Guest Log: NCAT DEBUG: Unable to load trusted CA certificates from /usr/share/ncat/ca-bundle.crt: error:02001002:system library:fopen:No such file or directory
2025-03-02 04:04:10 (24108): Guest Log: libnsock nsi_new2(): nsi_new (IOD #1)
2025-03-02 04:04:10 (24108): Guest Log: libnsock nsock_connect_tcp(): TCP connection requested to 127.0.0.2:4080 (IOD #1) EID 8
2025-03-02 04:04:10 (24108): Guest Log: libnsock nsock_trace_handler_callback(): Callback: CONNECT ERROR [Connection refused (111)] for EID 8 [127.0.0.2:4080]
2025-03-02 04:04:10 (24108): Guest Log: Ncat: Connection refused.
2025-03-02 04:04:11 (24108): Guest Log: [ERROR] Could not connect to vocms0267.cern.ch on port 4080

https://lhcathome.cern.ch/lhcathome/result.php?resultid=419942573
https://lhcathome.cern.ch/lhcathome/results.php?hostid=10865276

The cert issue is a red herring and can be ignored.
The real issue is a failing remapping of vocms0267.cern.ch (wrong) to vocms267.cern.ch (right).

If you use a local DNS service you may try to fix it until it is solved at CERN.
Otherwise you will have to wait until they fixed it there.

If you are using pihole or similar or have a local DNS server it's easy to do a temporary fix.

You just have to add a local DNS entry (I did it in my firewall) if your router/firewall doesn't have that possibility maybe you have Pihole - click local DNS records (not CNAME), add vocms0267.cern.ch and ip 188.185.64.105 click save. From now on when CMS ask for vocms0267.cern.ch your Pihole will return the correct ip. When Cern had time to fix the issue just delete the DNS record.

It' works.
It was easy to add the suggested DNS entry in my home router.
Thanks

It' works.
It was easy to add the suggested DNS entry in my home router.
Thanks

you lucky one; my router unfortunately did NOT offer me this possibility :-(
So let's wait and see when the problem will be fixed server-side.

Sorry I hadn't picked up on this earlier, I'm having a complicated private life at the moment. I thought the lack of jobs was due to other problems (e.g. my main machine seems to be having VirtualBox problems at the moment, and I hadn't looked beyond that over the weekend.
Laurence has fixed the naming bug and we seem to be getting more jobs running again now.
Again, apologies for not having looked into this earlier.

---

The vocms267 issue is solved since around 13:12 UTC.
No need to modify a local DNS service any more.

I confirm cms tasks no longer failing on my machines. Thanks to the cern team for the fix. Let's smash!

Thanks Ivan