LHC@home

I have had numerous ATLAS native tasks fail (status = validate error) on a new host I am trying to setup, https://lhcathome.cern.ch/lhcathome/results.php?hostid=10557024. The error is as in the Subject then followed by "Error: unable to setup default version value for rcsetup. See https://lhcathome.cern.ch/lhcathome/result.php?resultid=203906053 for example.

It does Sixtrack and ATLAS Vbox no problem. I've tried reducing the number of cores and concurrent running tasks and increasing the RAM via app_config.xml but it doesn't help. Firewall is off. Even a single 2-core task allocated 4GB RAM in app_config.xml running all by itself gets the above errors.

I see this in the log:

head: cannot open `/cvmfs/atlas.cern.ch/repo/sw/ChangeLog' for reading: Input/output error

which makes me think there is a problem with cvmfs. Can you check "cvmfs_config chksetup" and try restarting the cvmfs daemons?

~$ sudo cvmfs_config chksetup
Warning: failed to access http://s1cern-cvmfs.openhtc.io/cvmfs/atlas.cern.ch/.cvmfspublished through proxy http://206.167.181.94:3128
Warning: failed to use Geo-API with s1cern-cvmfs.openhtc.io
Warning: failed to access http://s1cern-cvmfs.openhtc.io/cvmfs/atlas.cern.ch/.cvmfspublished through proxy http://ca-proxy.cern.ch:3128
Warning: failed to access http://s1cern-cvmfs.openhtc.io/cvmfs/atlas.cern.ch/.cvmfspublished through proxy http://kraken01.westgrid.ca:3128
Warning: failed to access http://s1cern-cvmfs.openhtc.io/cvmfs/atlas.cern.ch/.cvmfspublished through proxy http://atlascaq4.triumf.ca:3128
Warning: failed to access http://s1fnal-cvmfs.openhtc.io/cvmfs/atlas.cern.ch/.cvmfspublished through proxy http://206.167.181.94:3128
Warning: failed to use Geo-API with s1fnal-cvmfs.openhtc.io

Those are just the first few lines. There must be 20 more but they are are similar to the above just different URLs.

Is "service autofs restart" the correct command for cvmfs daemon restart? I ran that, got no output so I assume the daemons restarted, followed by "cvmfs_config chksetup" again, got the same warnings as above.

Also did...
sudo service autofs status
● autofs.service - Automounts filesystems on demand
Loaded: loaded (/lib/systemd/system/autofs.service; enabled; vendor preset: enabled)
Active: active (running) since Wed 2018-08-08 10:48:31 MDT; 37min ago
Process: 4070 ExecStart=/usr/sbin/automount $OPTIONS --pid-file /var/run/autofs.pid (code=exited, status=0/SUCCESS)
Main PID: 4071 (automount)
Tasks: 4 (limit: 9830)
CGroup: /system.slice/autofs.service
└─4071 /usr/sbin/automount --pid-file /var/run/autofs.pid

Aug 08 10:48:31 brad-ProLiant-ML350p-Gen8 systemd[1]: Starting Automounts filesystems on demand...
Aug 08 10:48:31 brad-ProLiant-ML350p-Gen8 systemd[1]: Started Automounts filesystems on demand.

If it is a CVMFS issue I would clone /etc/cvmfs from a machine where it works.
Then run
1. "cvmfs_config showconfig -s" to check if the new configuration will be accepted
2. "cvmfs_config reload" to activate the new configuration
3. "cvmfs_config probe" should report "OK" for all configured repositories

If "cvmfs_config probe" reports errors you may reboot and run (3) again.
Still errors? Then post the output of "cvmfs_config showconfig -s" here.

<edit>
After a reboot "cvmfs_config wipecache" before (3) should be run.
</edit>

Thank you, David and computezrmie, for your help with this problem but I have to put it on the backburner for a while due to other concerns. I'll get back to it when I can and keep you posted. In the meantime there are lots of Sixtrack tasks for that rig to crunch.

If it is a CVMFS issue I would clone /etc/cvmfs from a machine where it works.
Then run
1. "cvmfs_config showconfig -s" to check if the new configuration will be accepted
2. "cvmfs_config reload" to activate the new configuration
3. "cvmfs_config probe" should report "OK" for all configured repositories

That fixed the problem, whatever it was. Thank you:)

If "cvmfs_config probe" reports errors you may reboot and run (3) again.
Still errors? Then post the output of "cvmfs_config showconfig -s" here.

<edit>
After a reboot "cvmfs_config wipecache" before (3) should be run.
</edit>

There were no errors, I didn't reboot, I just did the 3 steps above.