LHC@home

Hi,
I'm getting a "failed to unshare remaining namespaces" error on some native tasks but not all of them.
Here is one that failed: https://lhcathome.cern.ch/lhcathome/result.php?resultid=414634007
Here is one that succeeded: https://lhcathome.cern.ch/lhcathome/result.php?resultid=414604581
I'm not sure to understand what's happening, and i've tried everything i had in mind for fixing this. Including the "kernel.unprivileged_userns_clone = 1" kernel setting.

The machine is running ubuntu 24.04

Thanks for your help

Basically a runc message.

Please post your boinc-client service unit file to check if it contains settings that could cause problems like this.

  GNU nano 7.2                                                                                                                                                    /usr/lib/systemd/system/boinc-client.service
[Unit]
Description=Berkeley Open Infrastructure Network Computing Client
Documentation=man:boinc(1)
Wants=vboxdrv.service
After=vboxdrv.service network-online.target

[Service]
Type=simple
ProtectHome=true
ProtectSystem=full
ProtectControlGroups=true
ReadWritePaths=-/var/lib/boinc -/etc/boinc-client
Nice=10
User=boinc
WorkingDirectory=/var/lib/boinc
ExecStart=/usr/bin/boinc
ExecStop=/usr/bin/boinccmd --quit
ExecReload=/usr/bin/boinccmd --read_cc_config
ExecStopPost=/bin/rm -f lockfile
IOSchedulingClass=idle
# The following options prevent setuid root as they imply NoNewPrivileges=true
# Since Atlas requires setuid root, they break Atlas
# In order to improve security, if you're not using Atlas,
# Add these options to the [Service] section of an override file using
# sudo systemctl edit boinc-client.service
#NoNewPrivileges=true
#ProtectKernelModules=true
#ProtectKernelTunables=true
#RestrictRealtime=true
#RestrictAddressFamilies=AF_INET AF_INET6 AF_UNIX
#RestrictNamespaces=true
#PrivateUsers=true
#CapabilityBoundingSet=
#MemoryDenyWriteExecute=true
#PrivateTmp=true  #Block X11 idle detection

[Install]
WantedBy=multi-user.target

Here is the boinc-client service

This log entry is from your last successful task
https://lhcathome.cern.ch/lhcathome/result.php?resultid=414604581

18:52:23 UTC +00:00 2024-09-29: cranky-0.1.4: [INFO] Found a local runc version 1.1.12-0ubuntu3.1.

This log entry is from your first failing task
https://lhcathome.cern.ch/lhcathome/result.php?resultid=414607041

20:45:19 UTC +00:00 2024-09-29: cranky-0.1.4: [INFO] Found a local runc version 1.1.14.

So, you obviously installed a runc version that does not work correctly on your Ubuntu version.
Do you remember what else you changed?



Nonetheless, you may try the following modifications in the boinc service first (leave other settings untouched):
#ProtectHome=true
ProtectSystem=strict
#ProtectControlGroups=true
ReadWritePaths=-/var/lib/boinc -/etc/boinc-client # leave this enabled
ReadWritePaths=-/tmp # add this line to make virtualbox tasks work

Use an override file to implement the changes, otherwise they may get lost when the main unit file gets updated.

I already had issues on the 28th : https://lhcathome.cern.ch/lhcathome/result.php?resultid=414588095
That's why I don't understand why it started to work and then to not work again.

I will try what you are proposing tonight. In the event it does not work are you saying I should try to install runc 1.12? The only thing I did to runc was using apt update. So if Ubuntu's repo have bad versions... rip

In the event it does not work are you saying I should try to install runc 1.12? The only thing I did to runc was using apt update. So if Ubuntu's repo have bad versions... rip

BTW: Ubuntu latest runc version is 1.1.12-0ubuntu3.1, AFAIK. And it works well. I don't know where does 1.1.14 come from... maybe a side-repo?

However, maybe you could simply uninstall runc. You don't need a local runc, as for LHC@home apps.
--
Bye.

Well I've found out what's happening.
I also have docker on that machine. Which is using, I assume, a custom version of runc that, for whatever reason, does not work with LHC apps.
Is there a way to force LHC to NOT use the local runc?

No.
LHC respects YOUR decision to use a locally installed runc.
OTOH this makes YOU responsible to solve local issues.

I also have docker on that machine. Which is using, I assume, a custom version of runc that, for whatever reason, does not work with LHC apps.

I don't use docker, but it seems to be quite flexible in the choice of the container runtime:

https://docs.docker.com/engine/daemon/alternative-runtimes/

See if you can get rid of runc 1.1.14 and use something else for docker; or uninstall runc, reinstall it manually out of $PATH, register it in the demon configuration and use the --runtime flag...
Just some hints, please forgive me if I'm wrong.
--
Bye

You are thinking about apptainer and singularity.
When I was configuring cvmfs and boinc tasks were failing untill I have installed runc.

You are thinking about apptainer and singularity.
When I was configuring cvmfs and boinc tasks were failing untill I have installed runc.

No, actually there's no need to have runc installed locally:

lem@biggy:/$ which runc
lem@biggy:/$

In stderr:

Found 'runc version spec: 1.0.2-dev' at '/cvmfs/grid.cern.ch/vc/containers/runc.new'.

I've got it installed on another pc, though:

lem@oldie:~$ which runc
/usr/sbin/runc
lem@oldie:~$ runc -v
runc version 1.1.12-0ubuntu3.1
spec: 1.0.2-dev
go: go1.22.2
libseccomp: 2.5.5
lem@oldie:~$

In stderr:

[INFO] Found a local runc version 1.1.12-0ubuntu3.1.

Both crunch fine. :-)
--
Bye, Lem