Questions and Answers : Unix/Linux : Amost all tasks fail on my computers
Message board moderation

To post messages, you must log in.

AuthorMessage
Anchun

Send message
Joined: 20 Feb 25
Posts: 4
Credit: 14,909
RAC: 10
Message 52072 - Posted: 8 Aug 2025, 19:32:44 UTC

Hello, everyone. It seems that I meet some trouble with LHC@home.
Tasks always fail on all computers of mine.
For example:
Computer A has 1TB RECC RAM, running fedora42, without VirtualBox, all tasks fail.
Computer B has 16GB RECC RAM, running fedora42, without VirtualBox, all tasks fail.
Computer C is a Virtual Machine running on Computer A by QEMU-KVM, Windows10, had already installed the newest VirtualBox with Extension Pack, only succeeded 4 tasks, and the others failed.The 4 successful task are [CMS Simulation v70.91 (vbox64_mt_mcore_cms) windows_x86_64]. However, another [CMS Simulation v70.91 (vbox64_mt_mcore_cms) windows_x86_64] all have failed.
The computer A and B had gotten a lot of [ATLAS Simulation v3.01 (native_mt) x86_64-pc-linux-gnu], all failed.

I don't know is there any useful logs of the task, so I don't know what the problem is. I can only see [Computation error] on BOINC manager. And some useless log on "Event Log"(Ctrl+Shift+E), example:

Fri 08 Aug 2025 11:53:53 AM PDT | LHC@home | Computation for task uJaMDmG4I37nsSi4ap6QjLDmwznN0nGgGQJmUXzaDmb6ZLDmlTN0Wo_3 finished
Fri 08 Aug 2025 11:53:53 AM PDT | LHC@home | Output file uJaMDmG4I37nsSi4ap6QjLDmwznN0nGgGQJmUXzaDmb6ZLDmlTN0Wo_3_r136231064_ATLAS_result for task uJaMDmG4I37nsSi4ap6QjLDmwznN0nGgGQJmUXzaDmb6ZLDmlTN0Wo_3 absent

But in fact it was not executed successfully.

some info of LHC@home preferences:
Run only the selected applications (all applications)
If no work for selected applications is available, accept work from other applications? ENABLE
Run native if available? (Not recommended for Windows) ENABLE

BOINC version:8.2.4 (for all computers). On A and B, it was installed by flatpak. On C, installed by boinc_8.2.4_windows_x86_64.exe.
None of the computers have ever encountered errors in other projects.(Such as WCG, Milkyway@home, Einstein@Home, PrimeGird, Rosetta@home, Asteroids@home and so on.)

Could anyone can help me? Thanks a lot!
ID: 52072 · Report as offensive     Reply Quote
computezrmle
Volunteer moderator
Volunteer developer
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 15 Jun 08
Posts: 2681
Credit: 286,838,543
RAC: 64,716
Message 52073 - Posted: 8 Aug 2025, 20:07:02 UTC - in response to Message 52072.  

Please make your computers visible for others at the prefs page.

So far:
- Don't use Flatpak. Replace it with the package for your OS.
- Disable "Run native if available?" for your Windows computer (requires a separate venue)
ID: 52073 · Report as offensive     Reply Quote
Anchun

Send message
Joined: 20 Feb 25
Posts: 4
Credit: 14,909
RAC: 10
Message 52074 - Posted: 8 Aug 2025, 21:38:38 UTC - in response to Message 52073.  
Last modified: 8 Aug 2025, 21:39:15 UTC

Thanks. The computers are visible now.
I add a preferences for Windows, disable "Run native if available?".
And for the reason why I use flatpak, that's because the package from my os (dnf install boinc-client boinc-manager) always have some errors. I had already tried a lot of times on fedora/openSUSE on QEMU-KVM, reinstall the system on VM(Virtual Machine). However, I meet a error that cause the manager cannot connect to the client. After I cost a lot of time and finally found a way to solve it, a new problem appear, has continued until now. The problem is there are only 3 projects available to add. LHC@home and so many project disappeared. I tried to solve the new problem on VM so many times, but the problem still exists. What is worse, when I reinstall the VM's system, use flatpak to install BOINC on the VM, the problem still exists!
Oh, it seems that pictures can't be uploaded here? I want to show you the UI. I'll describe it in words first.
Start manager-->click "Tools"-->click "Add project..."-->
Only have:
DENIS@home
RNA World
SIDock@home

No more. That's very strange!
I think my network is normal, or I cannot download it on VM successfully. And the BOINC previously installed on the host and Windoes VM is working fine, I can see all of the projects and add a project.
So I don't dare to switch BOINC to the os version on hosts now.

It is my first time to use the "Community" and my mother language isn't English, so my description might not be very precise.Sorry.
ID: 52074 · Report as offensive     Reply Quote
computezrmle
Volunteer moderator
Volunteer developer
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 15 Jun 08
Posts: 2681
Credit: 286,838,543
RAC: 64,716
Message 52075 - Posted: 9 Aug 2025, 7:44:46 UTC - in response to Message 52074.  

ATLAS/CMS/Theory either require a local CVMFS client (native app) or VirtualBox.
Neither works as long as you use BOINC's Flatpak version.

Your logs show typical messages like:
[2025-08-08 16:55:55] Checking for CVMFS
[2025-08-08 16:55:55] No cvmfs_config command found, will try listing directly
[2025-08-08 16:55:55] ls: cannot access '/cvmfs/atlas.cern.ch/repo/sw': No such file or directory
[2025-08-08 16:55:55] Failed to list /cvmfs/atlas.cern.ch/repo/sw
[2025-08-08 16:55:55] ** It looks like CVMFS is not installed on this host.
[2025-08-08 16:55:55] ** CVMFS is required to run ATLAS native tasks and can be installed following https://cvmfs.readthedocs.io/en/stable/cpt-quickstart.html
[2025-08-08 16:55:55] ** and setting 'CVMFS_REPOSITORIES=atlas.cern.ch,atlas-condb.cern.ch' in /etc/cvmfs/default.local



As for Windows

Some tasks get validated but show this:
2025-08-09 07:13:41 (11352): Guest Log: No HITS file was produced


This seems to be caused by not enough RAM configured for the VM:
2025-08-09 06:23:24 (11352): Setting Memory Size for VM. (4800MB)
2025-08-09 06:23:34 (11352): Setting CPU Count for VM. (8)


Best would be to follow an old advice and configure "(3000 + n * 900)MB" per VM with "n" being the number of virtual cores.
In this case (3000 + 8 * 900)MB = 10200MB.
The value can be set via a line like this in an app_config.xml
<cmdline>--memory_size_mb 10200</cmdline>


More details:
https://boinc.berkeley.edu/wiki/Client_configuration#Project-level_configuration
ID: 52075 · Report as offensive     Reply Quote
Anchun

Send message
Joined: 20 Feb 25
Posts: 4
Credit: 14,909
RAC: 10
Message 52128 - Posted: 28 Aug 2025, 7:42:12 UTC - in response to Message 52075.  
Last modified: 28 Aug 2025, 8:03:02 UTC

Thanks so much! Sorry to have kept you waiting. I successfully installed the native version of BOINC and solved the strange 'only 3 tasks available' problem.
Then, I configured CVMFS. I followed the steps from these websites:
https://cvmfs.readthedocs.io/en/stable/cpt-quickstart.html
https://lhcathome.cern.ch/lhcathome/forum_thread.php?id=5594
"cvmfs_config stat" and "cvmfs_config probe" return OK. And the tasks finally progress more than 10%. However, I still get "compute error".

For example:
<core_client_version>8.0.2</core_client_version>
<![CDATA[
<message>
process exited with code 195 (0xc3, -61)</message>
<stderr_txt>
12:34:06 (22086): wrapper (7.7.26015): starting
12:34:06 (22086): wrapper: running run_atlas (--nthreads 12)
[2025-08-27 12:34:06] Arguments: --nthreads 12
[2025-08-27 12:34:06] Threads: 12
[2025-08-27 12:34:06] Checking for CVMFS
[2025-08-27 12:34:06] Probing /cvmfs/atlas.cern.ch... OK
[2025-08-27 12:34:06] Probing /cvmfs/atlas-condb.cern.ch... OK
[2025-08-27 12:34:06] Running cvmfs_config stat atlas.cern.ch
[2025-08-27 12:34:07] VERSION PID UPTIME(M) MEM(K) REVISION EXPIRES(M) NOCATALOGS CACHEUSE(K) CACHEMAX(K) NOFDUSE NOFDMAX NOIOERR NOOPEN HITRATE(%) RX(K) SPEED(K/S) HOST PROXY ONLINE
[2025-08-27 12:34:07] 2.13.2.0 19304 2 26552 149982 1 1 45120 6144001 3 130560 0 4 0.000 1292 282 http://s1ihep-cvmfs.openhtc.io:8080/cvmfs/atlas.cern.ch DIRECT 1
[2025-08-27 12:34:07] CVMFS is ok
[2025-08-27 12:34:07] Efficiency of ATLAS tasks can be improved by the following measure(s):
[2025-08-27 12:34:07] Small home clusters do not require a local http proxy but it is suggested if
[2025-08-27 12:34:07] more than 10 cores throughout the same LAN segment are regularly running ATLAS like tasks.
[2025-08-27 12:34:07] Further information can be found at the LHC@home message board.
[2025-08-27 12:34:07] Using apptainer image /cvmfs/atlas.cern.ch/repo/containers/fs/singularity/x86_64-centos7
[2025-08-27 12:34:07] Checking for apptainer binary...
[2025-08-27 12:34:07] which: no apptainer in (/home/fedoravm/.local/bin:/home/fedoravm/bin:/usr/local/bin:/usr/bin)
[2025-08-27 12:34:07] apptainer is not installed, using version from CVMFS
[2025-08-27 12:34:07] Checking apptainer works with /cvmfs/atlas.cern.ch/repo/containers/sw/apptainer/x86_64-el7/current/bin/apptainer exec -B /cvmfs /cvmfs/atlas.cern.ch/repo/containers/fs/singularity/x86_64-centos7 hostname
12:34:20 (22657): wrapper (7.7.26015): starting
12:34:20 (22657): wrapper: running run_atlas (--nthreads 12)
[2025-08-27 12:34:20] Arguments: --nthreads 12
[2025-08-27 12:34:20] Threads: 12
[2025-08-27 12:34:20] Checking for CVMFS
[2025-08-27 12:34:20] Probing /cvmfs/atlas.cern.ch... OK
[2025-08-27 12:34:20] Probing /cvmfs/atlas-condb.cern.ch... OK
[2025-08-27 12:34:20] Running cvmfs_config stat atlas.cern.ch
[2025-08-27 12:34:21] VERSION PID UPTIME(M) MEM(K) REVISION EXPIRES(M) NOCATALOGS CACHEUSE(K) CACHEMAX(K) NOFDUSE NOFDMAX NOIOERR NOOPEN HITRATE(%) RX(K) SPEED(K/S) HOST PROXY ONLINE
[2025-08-27 12:34:21] 2.13.2.0 19304 2 27620 149982 1 1 75266 6144001 0 130560 0 6 83.333 11882 694 http://s1ihep-cvmfs.openhtc.io:8080/cvmfs/atlas.cern.ch DIRECT 1
[2025-08-27 12:34:21] CVMFS is ok
[2025-08-27 12:34:21] Efficiency of ATLAS tasks can be improved by the following measure(s):
[2025-08-27 12:34:21] Small home clusters do not require a local http proxy but it is suggested if
[2025-08-27 12:34:21] more than 10 cores throughout the same LAN segment are regularly running ATLAS like tasks.
[2025-08-27 12:34:21] Further information can be found at the LHC@home message board.
[2025-08-27 12:34:21] Using apptainer image /cvmfs/atlas.cern.ch/repo/containers/fs/singularity/x86_64-centos7
[2025-08-27 12:34:21] Checking for apptainer binary...
[2025-08-27 12:34:21] which: no apptainer in (/home/fedoravm/.local/bin:/home/fedoravm/bin:/usr/local/bin:/usr/bin)
[2025-08-27 12:34:21] apptainer is not installed, using version from CVMFS
[2025-08-27 12:34:21] Checking apptainer works with /cvmfs/atlas.cern.ch/repo/containers/sw/apptainer/x86_64-el7/current/bin/apptainer exec -B /cvmfs /cvmfs/atlas.cern.ch/repo/containers/fs/singularity/x86_64-centos7 hostname
[2025-08-27 13:02:07] INFO: fuse-overlayfs mount took an unexpectedly long time: 2.225s fedora
[2025-08-27 13:02:07] apptainer works
[2025-08-27 13:02:07] Set ATHENA_PROC_NUMBER=12
[2025-08-27 13:02:07] Set ATHENA_CORE_NUMBER=12
[2025-08-27 13:02:07] Starting ATLAS job with PandaID=6785752112
[2025-08-27 13:02:07] Running command: /cvmfs/atlas.cern.ch/repo/containers/sw/apptainer/x86_64-el7/current/bin/apptainer exec -B /cvmfs,/home/fedoravm/slots/5 /cvmfs/atlas.cern.ch/repo/containers/fs/singularity/x86_64-centos7 sh start_atlas.sh
13:57:35 (158024): wrapper (7.7.26015): starting
13:57:35 (158024): wrapper: running run_atlas (--nthreads 12)
[2025-08-27 13:57:35] Arguments: --nthreads 12
[2025-08-27 13:57:35] Threads: 12
[2025-08-27 13:57:35] This job has been restarted, cleaning up previous attempt
[2025-08-27 13:57:35] Checking for CVMFS
[2025-08-27 13:57:35] Probing /cvmfs/atlas.cern.ch... OK
[2025-08-27 13:57:35] Probing /cvmfs/atlas-condb.cern.ch... OK
[2025-08-27 13:57:35] Running cvmfs_config stat atlas.cern.ch
[2025-08-27 13:57:36] VERSION PID UPTIME(M) MEM(K) REVISION EXPIRES(M) NOCATALOGS CACHEUSE(K) CACHEMAX(K) NOFDUSE NOFDMAX NOIOERR NOOPEN HITRATE(%) RX(K) SPEED(K/S) HOST PROXY ONLINE
[2025-08-27 13:57:36] 2.13.2.0 19304 85 46020 149984 2 184 1781264 6144000 0 130560 0 349760 97.857 666974 152 http://s1cern-cvmfs.openhtc.io/cvmfs/atlas.cern.ch DIRECT 1
[2025-08-27 13:57:36] CVMFS is ok
[2025-08-27 13:57:36] Efficiency of ATLAS tasks can be improved by the following measure(s):
[2025-08-27 13:57:36] Small home clusters do not require a local http proxy but it is suggested if
[2025-08-27 13:57:36] more than 10 cores throughout the same LAN segment are regularly running ATLAS like tasks.
[2025-08-27 13:57:36] Further information can be found at the LHC@home message board.
[2025-08-27 13:57:36] Using apptainer image /cvmfs/atlas.cern.ch/repo/containers/fs/singularity/x86_64-centos7
[2025-08-27 13:57:36] Checking for apptainer binary...
[2025-08-27 13:57:36] which: no apptainer in (/home/fedoravm/.local/bin:/home/fedoravm/bin:/usr/local/bin:/usr/bin)
[2025-08-27 13:57:36] apptainer is not installed, using version from CVMFS
[2025-08-27 13:57:36] Checking apptainer works with /cvmfs/atlas.cern.ch/repo/containers/sw/apptainer/x86_64-el7/current/bin/apptainer exec -B /cvmfs /cvmfs/atlas.cern.ch/repo/containers/fs/singularity/x86_64-centos7 hostname
[2025-08-27 13:57:39] fedora
[2025-08-27 13:57:39] apptainer works
[2025-08-27 13:57:39] Set ATHENA_PROC_NUMBER=12
[2025-08-27 13:57:39] Set ATHENA_CORE_NUMBER=12
[2025-08-27 13:57:39] Starting ATLAS job with PandaID=6785752112
[2025-08-27 13:57:39] Running command: /cvmfs/atlas.cern.ch/repo/containers/sw/apptainer/x86_64-el7/current/bin/apptainer exec -B /cvmfs,/home/fedoravm/slots/5 /cvmfs/atlas.cern.ch/repo/containers/fs/singularity/x86_64-centos7 sh start_atlas.sh
14:02:47 (266068): wrapper (7.7.26015): starting
14:02:47 (266068): wrapper: running run_atlas (--nthreads 12)
[2025-08-27 14:02:47] Arguments: --nthreads 12
[2025-08-27 14:02:47] Threads: 12
[2025-08-27 14:02:47] This job has been restarted, cleaning up previous attempt
[2025-08-27 14:02:47] Checking for CVMFS
[2025-08-27 14:02:47] Probing /cvmfs/atlas.cern.ch... OK
[2025-08-27 14:02:47] Probing /cvmfs/atlas-condb.cern.ch... OK
[2025-08-27 14:02:47] Running cvmfs_config stat atlas.cern.ch
[2025-08-27 14:02:48] VERSION PID UPTIME(M) MEM(K) REVISION EXPIRES(M) NOCATALOGS CACHEUSE(K) CACHEMAX(K) NOFDUSE NOFDMAX NOIOERR NOOPEN HITRATE(%) RX(K) SPEED(K/S) HOST PROXY ONLINE
[2025-08-27 14:02:48] 2.13.2.0 19304 90 40904 149984 3 1 1784324 10240001 0 130560 0 4 100.000 0 0 http://s1ihep-cvmfs.openhtc.io:8080/cvmfs/atlas.cern.ch DIRECT 1
[2025-08-27 14:02:48] CVMFS is ok
[2025-08-27 14:02:48] Efficiency of ATLAS tasks can be improved by the following measure(s):
[2025-08-27 14:02:48] Small home clusters do not require a local http proxy but it is suggested if
[2025-08-27 14:02:48] more than 10 cores throughout the same LAN segment are regularly running ATLAS like tasks.
[2025-08-27 14:02:48] Further information can be found at the LHC@home message board.
[2025-08-27 14:02:48] Using apptainer image /cvmfs/atlas.cern.ch/repo/containers/fs/singularity/x86_64-centos7
[2025-08-27 14:02:48] Checking for apptainer binary...
[2025-08-27 14:02:48] which: no apptainer in (/home/fedoravm/.local/bin:/home/fedoravm/bin:/usr/local/bin:/usr/bin)
[2025-08-27 14:02:48] apptainer is not installed, using version from CVMFS
[2025-08-27 14:02:48] Checking apptainer works with /cvmfs/atlas.cern.ch/repo/containers/sw/apptainer/x86_64-el7/current/bin/apptainer exec -B /cvmfs /cvmfs/atlas.cern.ch/repo/containers/fs/singularity/x86_64-centos7 hostname
[2025-08-27 14:02:51] fedora
[2025-08-27 14:02:51] apptainer works
[2025-08-27 14:02:51] Set ATHENA_PROC_NUMBER=12
[2025-08-27 14:02:51] Set ATHENA_CORE_NUMBER=12
[2025-08-27 14:02:51] Starting ATLAS job with PandaID=6785752112
[2025-08-27 14:02:51] Running command: /cvmfs/atlas.cern.ch/repo/containers/sw/apptainer/x86_64-el7/current/bin/apptainer exec -B /cvmfs,/home/fedoravm/slots/5 /cvmfs/atlas.cern.ch/repo/containers/fs/singularity/x86_64-centos7 sh start_atlas.sh
14:17:26 (376997): wrapper (7.7.26015): starting
14:17:26 (376997): wrapper: running run_atlas (--nthreads 12)
[2025-08-27 14:17:26] Arguments: --nthreads 12
[2025-08-27 14:17:26] Threads: 12
[2025-08-27 14:17:26] This job has been restarted, cleaning up previous attempt
[2025-08-27 14:17:26] Checking for CVMFS
[2025-08-27 14:17:26] Probing /cvmfs/atlas.cern.ch... OK
[2025-08-27 14:17:26] Probing /cvmfs/atlas-condb.cern.ch... OK
[2025-08-27 14:17:26] Running cvmfs_config stat atlas.cern.ch
[2025-08-27 14:17:27] VERSION PID UPTIME(M) MEM(K) REVISION EXPIRES(M) NOCATALOGS CACHEUSE(K) CACHEMAX(K) NOFDUSE NOFDMAX NOIOERR NOOPEN HITRATE(%) RX(K) SPEED(K/S) HOST PROXY ONLINE
[2025-08-27 14:17:27] 2.13.2.0 19304 105 52004 149986 1 11 1890376 10240000 10 130560 0 337663 99.869 100982 269 http://s1ihep-cvmfs.openhtc.io:8080/cvmfs/atlas.cern.ch DIRECT 1
[2025-08-27 14:17:27] CVMFS is ok
[2025-08-27 14:17:27] Efficiency of ATLAS tasks can be improved by the following measure(s):
[2025-08-27 14:17:27] Small home clusters do not require a local http proxy but it is suggested if
[2025-08-27 14:17:27] more than 10 cores throughout the same LAN segment are regularly running ATLAS like tasks.
[2025-08-27 14:17:27] Further information can be found at the LHC@home message board.
[2025-08-27 14:17:27] Using apptainer image /cvmfs/atlas.cern.ch/repo/containers/fs/singularity/x86_64-centos7
[2025-08-27 14:17:27] Checking for apptainer binary...
[2025-08-27 14:17:27] which: no apptainer in (/home/fedoravm/.local/bin:/home/fedoravm/bin:/usr/local/bin:/usr/bin)
[2025-08-27 14:17:27] apptainer is not installed, using version from CVMFS
[2025-08-27 14:17:27] Checking apptainer works with /cvmfs/atlas.cern.ch/repo/containers/sw/apptainer/x86_64-el7/current/bin/apptainer exec -B /cvmfs /cvmfs/atlas.cern.ch/repo/containers/fs/singularity/x86_64-centos7 hostname
[2025-08-27 14:17:28] fedora
[2025-08-27 14:17:28] apptainer works
[2025-08-27 14:17:28] Set ATHENA_PROC_NUMBER=12
[2025-08-27 14:17:28] Set ATHENA_CORE_NUMBER=12
[2025-08-27 14:17:28] Starting ATLAS job with PandaID=6785752112
[2025-08-27 14:17:28] Running command: /cvmfs/atlas.cern.ch/repo/containers/sw/apptainer/x86_64-el7/current/bin/apptainer exec -B /cvmfs,/home/fedoravm/slots/5 /cvmfs/atlas.cern.ch/repo/containers/fs/singularity/x86_64-centos7 sh start_atlas.sh
23:56:14 (586469): wrapper (7.7.26015): starting
23:56:14 (586469): wrapper: running run_atlas (--nthreads 12)
[2025-08-27 23:56:14] Arguments: --nthreads 12
[2025-08-27 23:56:14] Threads: 12
[2025-08-27 23:56:14] This job has been restarted, cleaning up previous attempt
[2025-08-27 23:56:14] Checking for CVMFS
[2025-08-27 23:56:18] Probing /cvmfs/atlas.cern.ch... OK
[2025-08-27 23:56:22] Probing /cvmfs/atlas-condb.cern.ch... OK
[2025-08-27 23:56:22] Running cvmfs_config stat atlas.cern.ch
[2025-08-27 23:56:22] VERSION PID UPTIME(M) MEM(K) REVISION EXPIRES(M) NOCATALOGS CACHEUSE(K) CACHEMAX(K) NOFDUSE NOFDMAX NOIOERR NOOPEN HITRATE(%) RX(K) SPEED(K/S) HOST PROXY ONLINE
[2025-08-27 23:56:22] 2.13.2.0 586830 0 25732 150000 3 1 1926245 10240001 0 130560 0 0 100.000 0 0 http://s1ihep-cvmfs.openhtc.io:8080/cvmfs/atlas.cern.ch DIRECT 1
[2025-08-27 23:56:22] CVMFS is ok
[2025-08-27 23:56:23] Efficiency of ATLAS tasks can be improved by the following measure(s):
[2025-08-27 23:56:23] Small home clusters do not require a local http proxy but it is suggested if
[2025-08-27 23:56:23] more than 10 cores throughout the same LAN segment are regularly running ATLAS like tasks.
[2025-08-27 23:56:23] Further information can be found at the LHC@home message board.
[2025-08-27 23:56:23] Using apptainer image /cvmfs/atlas.cern.ch/repo/containers/fs/singularity/x86_64-centos7
[2025-08-27 23:56:23] Checking for apptainer binary...
[2025-08-27 23:56:23] which: no apptainer in (/home/fedoravm/.local/bin:/home/fedoravm/bin:/usr/local/bin:/usr/bin)
[2025-08-27 23:56:23] apptainer is not installed, using version from CVMFS
[2025-08-27 23:56:23] Checking apptainer works with /cvmfs/atlas.cern.ch/repo/containers/sw/apptainer/x86_64-el7/current/bin/apptainer exec -B /cvmfs /cvmfs/atlas.cern.ch/repo/containers/fs/singularity/x86_64-centos7 hostname
[2025-08-27 23:56:27] fedora
[2025-08-27 23:56:27] apptainer works
[2025-08-27 23:56:27] Set ATHENA_PROC_NUMBER=12
[2025-08-27 23:56:27] Set ATHENA_CORE_NUMBER=12
[2025-08-27 23:56:27] Starting ATLAS job with PandaID=6785752112
[2025-08-27 23:56:27] Running command: /cvmfs/atlas.cern.ch/repo/containers/sw/apptainer/x86_64-el7/current/bin/apptainer exec -B /cvmfs,/home/fedoravm/slots/5 /cvmfs/atlas.cern.ch/repo/containers/fs/singularity/x86_64-centos7 sh start_atlas.sh
[2025-08-28 00:00:01] Job failed
[2025-08-28 00:00:01] ++ pwd
[2025-08-28 00:00:01] + export RUNTIME_CONFIG_DIR=/home/fedoravm/slots/5/
[2025-08-28 00:00:01] + RUNTIME_CONFIG_DIR=/home/fedoravm/slots/5/
[2025-08-28 00:00:01] + mv ATLAS.root_0 EVNT.46070566._000769.pool.root.1
[2025-08-28 00:00:01] mv: &#226;&#128;&#152;ATLAS.root_0&#226;&#128;&#153; and &#226;&#128;&#152;EVNT.46070566._000769.pool.root.1&#226;&#128;&#153; are the same file
[2025-08-28 00:00:01] + tar --strip-components=5 -xvf input.tar.gz
[2025-08-28 00:00:01] + umask 077
[2025-08-28 00:00:01] ++ date +%s
[2025-08-28 00:00:01] + ACCOUNTING_STARTTIME=1756364190
[2025-08-28 00:00:01] + echo 'Detecting resource accounting method available for the job.'
[2025-08-28 00:00:01] Detecting resource accounting method available for the job.
[2025-08-28 00:00:01] + JOB_ACCOUNTING=
[2025-08-28 00:00:01] + command -v arc-job-cgroup
[2025-08-28 00:00:01] + '[' -z '' ']'
[2025-08-28 00:00:01] + GNU_TIME=/usr/bin/time
[2025-08-28 00:00:01] + echo 'Looking for /usr/bin/time tool for accounting measurements'
[2025-08-28 00:00:01] Looking for /usr/bin/time tool for accounting measurements
[2025-08-28 00:00:01] + '[' '!' -z /usr/bin/time ']'
[2025-08-28 00:00:01] + /usr/bin/time --version
[2025-08-28 00:00:01] + echo 'GNU time found and will be used for job accounting.'
[2025-08-28 00:00:01] GNU time found and will be used for job accounting.
[2025-08-28 00:00:01] + JOB_ACCOUNTING=gnutime
[2025-08-28 00:00:01] + '[' -z gnutime ']'
[2025-08-28 00:00:01] + export PANDA_JSID=harvester-CERN_central_ACTA
[2025-08-28 00:00:01] + PANDA_JSID=harvester-CERN_central_ACTA
[2025-08-28 00:00:01] + export GTAG=http://aipanda404.cern.ch/data/jobs/2025-08-27/BOINC_MCORE/6785752112.out
[2025-08-28 00:00:01] + GTAG=http://aipanda404.cern.ch/data/jobs/2025-08-27/BOINC_MCORE/6785752112.out
[2025-08-28 00:00:01] + export PILOT_NOKILL=YES
[2025-08-28 00:00:01] + PILOT_NOKILL=YES
[2025-08-28 00:00:01] + export GRID_GLOBAL_JOBID=u2ONDml2SB8n9Rq4apOajLDm4fhM0noT9bVof3QYDmh3eLDmcxjYim
[2025-08-28 00:00:01] + GRID_GLOBAL_JOBID=u2ONDml2SB8n9Rq4apOajLDm4fhM0noT9bVof3QYDmh3eLDmcxjYim
[2025-08-28 00:00:01] + export GRID_GLOBAL_JOBURL=https://arc-boinc-03.cern.ch:443/arex/u2ONDml2SB8n9Rq4apOajLDm4fhM0noT9bVof3QYDmh3eLDmcxjYim
[2025-08-28 00:00:01] + GRID_GLOBAL_JOBURL=https://arc-boinc-03.cern.ch:443/arex/u2ONDml2SB8n9Rq4apOajLDm4fhM0noT9bVof3QYDmh3eLDmcxjYim
[2025-08-28 00:00:01] + export GRID_GLOBAL_JOBINTERFACE=org.nordugrid.arcrest
[2025-08-28 00:00:01] + GRID_GLOBAL_JOBINTERFACE=org.nordugrid.arcrest
[2025-08-28 00:00:01] + export GRID_GLOBAL_JOBHOST=arc-boinc-03.cern.ch
[2025-08-28 00:00:01] + GRID_GLOBAL_JOBHOST=arc-boinc-03.cern.ch
[2025-08-28 00:00:01] ++ pwd
[2025-08-28 00:00:01] + RUNTIME_JOB_DIR=/home/fedoravm/slots/5
[2025-08-28 00:00:01] + RUNTIME_JOB_STDIN=/dev/null
[2025-08-28 00:00:01] ++ pwd
[2025-08-28 00:00:01] + RUNTIME_JOB_STDOUT=/home/fedoravm/slots/5/log.46070568._065220.job.log.1
[2025-08-28 00:00:01] ++ pwd
[2025-08-28 00:00:01] + RUNTIME_JOB_STDERR=/home/fedoravm/slots/5/log.46070568._065220.job.log.1
[2025-08-28 00:00:01] ++ pwd
[2025-08-28 00:00:01] + RUNTIME_JOB_DIAG=/home/fedoravm/slots/5.diag
[2025-08-28 00:00:01] + '[' '!' -z '' ']'
[2025-08-28 00:00:01] + RUNTIME_LOCAL_SCRATCH_DIR=
[2025-08-28 00:00:01] + RUNTIME_LOCAL_SCRATCH_MOVE_TOOL=mv
[2025-08-28 00:00:01] + RUNTIME_FRONTEND_SEES_NODE=
[2025-08-28 00:00:01] + RUNTIME_NODE_SEES_FRONTEND=yes
[2025-08-28 00:00:01] + '[' '!' -z '' ']'
[2025-08-28 00:00:01] + '[' -z yes ']'
[2025-08-28 00:00:01] ++ pwd
[2025-08-28 00:00:01] + RUNTIME_JOB_DIAG=/home/fedoravm/slots/5/u2ONDml2SB8n9Rq4apOajLDm4fhM0noT9bVof3QYDmh3eLDmcxjYim.diag
[2025-08-28 00:00:01] + runtimeenvironments=
[2025-08-28 00:00:01] + runtimeenvironments='APPS/HEP/ATLAS-SITE;'
[2025-08-28 00:00:01] + RTE_function_0 1
[2025-08-28 00:00:01] + export PROJECT_ROOT=/boincdata/boinc/project/lhcathome
[2025-08-28 00:00:01] + PROJECT_ROOT=/boincdata/boinc/project/lhcathome
[2025-08-28 00:00:01] + export BOINC_APP=ATLAS
[2025-08-28 00:00:01] + BOINC_APP=ATLAS
[2025-08-28 00:00:01] + export WU_TEMPLATE=templates/ATLAS_IN_DYNAMIC
[2025-08-28 00:00:01] + WU_TEMPLATE=templates/ATLAS_IN_DYNAMIC
[2025-08-28 00:00:01] + export RESULT_TEMPLATE=templates/ATLAS_OUT_2
[2025-08-28 00:00:01] + RESULT_TEMPLATE=templates/ATLAS_OUT_2
[2025-08-28 00:00:01] + '[' -z 1 ']'
[2025-08-28 00:00:01] + '[' 1 == 1 ']'
[2025-08-28 00:00:01] + shift
[2025-08-28 00:00:01] + export ATLAS_LOCAL_AREA=/home/fedoravm/slots/5/
[2025-08-28 00:00:01] + ATLAS_LOCAL_AREA=/home/fedoravm/slots/5/
[2025-08-28 00:00:01] + '[' 0 -ne 0 ']'
[2025-08-28 00:00:01] + echo 'runtimeenvironments=APPS/HEP/ATLAS-SITE;'
[2025-08-28 00:00:01] + '[' -f /etc/os-release ']'
[2025-08-28 00:00:01] +++ cat /etc/os-release
[2025-08-28 00:00:01] ++ eval 'NAME="CentOS' 'Linux"' 'VERSION="7' '(Core)"' 'ID="centos"' 'ID_LIKE="rhel' 'fedora"' 'VERSION_ID="7"' 'PRETTY_NAME="CentOS' Linux 7 '(Core)"' 'ANSI_COLOR="0;31"' 'CPE_NAME="cpe:/o:centos:centos:7"' 'HOME_URL="https://www.centos.org/"' 'BUG_REPORT_URL="https://bugs.centos.org/"' 'CENTOS_MANTISBT_PROJECT="CentOS-7"' 'CENTOS_MANTISBT_PROJECT_VERSION="7"' 'REDHAT_SUPPORT_PRODUCT="centos"' 'REDHAT_SUPPORT_PRODUCT_VERSION="7"'
[2025-08-28 00:00:01] +++ NAME='CentOS Linux'
[2025-08-28 00:00:01] +++ VERSION='7 (Core)'
[2025-08-28 00:00:01] +++ ID=centos
[2025-08-28 00:00:01] +++ ID_LIKE='rhel fedora'
[2025-08-28 00:00:01] +++ VERSION_ID=7
[2025-08-28 00:00:01] +++ PRETTY_NAME='CentOS Linux 7 (Core)'
[2025-08-28 00:00:01] +++ ANSI_COLOR='0;31'
[2025-08-28 00:00:01] +++ CPE_NAME=cpe:/o:centos:centos:7
[2025-08-28 00:00:01] +++ HOME_URL=https://www.centos.org/
[2025-08-28 00:00:01] +++ BUG_REPORT_URL=https://bugs.centos.org/
[2025-08-28 00:00:01] +++ CENTOS_MANTISBT_PROJECT=CentOS-7
[2025-08-28 00:00:01] +++ CENTOS_MANTISBT_PROJECT_VERSION=7
[2025-08-28 00:00:01] +++ REDHAT_SUPPORT_PRODUCT=centos
[2025-08-28 00:00:01] +++ REDHAT_SUPPORT_PRODUCT_VERSION=7
[2025-08-28 00:00:01] ++ echo 'CentOS Linux 7 (Core)'
[2025-08-28 00:00:01] + SYSTEM_SOFTWARE='CentOS Linux 7 (Core)'
[2025-08-28 00:00:01] + '[' -n 'CentOS Linux 7 (Core)' ']'
[2025-08-28 00:00:01] + echo 'systemsoftware=CentOS Linux 7 (Core)'
[2025-08-28 00:00:01] + HOME=/home/fedoravm/slots/5
[2025-08-28 00:00:01] + export HOME
[2025-08-28 00:00:01] + cd /home/fedoravm/slots/5
[2025-08-28 00:00:01] + '[' '!' -z '' ']'
[2025-08-28 00:00:01] + '[' -z '' ']'
[2025-08-28 00:00:01] ++ /bin/hostname -f
[2025-08-28 00:00:01] + nodename=fedora
[2025-08-28 00:00:01] + echo nodename=fedora
[2025-08-28 00:00:01] + echo Processors=2
[2025-08-28 00:00:01] + echo Nodecount=1
[2025-08-28 00:00:01] + echo Benchmark=HEPSPEC:1.0
[2025-08-28 00:00:01] + executable=./runpilot2-wrapper.sh
[2025-08-28 00:00:01] + '[' '!' -f ./runpilot2-wrapper.sh ']'
[2025-08-28 00:00:01] ++ dd if=./runpilot2-wrapper.sh count=1
[2025-08-28 00:00:01] ++ head -n 1
[2025-08-28 00:00:01] ++ tr -d '\0'
[2025-08-28 00:00:01] + line1='#!/bin/bash'
[2025-08-28 00:00:01] ++ echo '#!/bin/bash'
[2025-08-28 00:00:01] ++ sed -n 's/^#! *//p'
[2025-08-28 00:00:01] + shebang=/bin/bash
[2025-08-28 00:00:01] ++ echo /bin/bash
[2025-08-28 00:00:01] ++ awk '{print $1}'
[2025-08-28 00:00:01] + interpreter=/bin/bash
[2025-08-28 00:00:01] + '[' /bin/bash = /usr/bin/env ']'
[2025-08-28 00:00:01] + '[' x/bin/bash = x ']'
[2025-08-28 00:00:01] + type /bin/bash
[2025-08-28 00:00:01] + '[' xgnutime = xgnutime ']'
[2025-08-28 00:00:01] + /usr/bin/time -o /home/fedoravm/slots/5/u2ONDml2SB8n9Rq4apOajLDm4fhM0noT9bVof3QYDmh3eLDmcxjYim.diag -a -f 'WallTime=%es\nKernelTime=%Ss\nUserTime=%Us\nCPUUsage=%P\nMaxResidentMemory=%MkB\nAverageResidentMemory=%tkB\nAverageTotalMemory=%KkB\nAverageUnsharedMemory=%DkB\nAverageUnsharedStack=%pkB\nAverageSharedMemory=%XkB\nPageSize=%ZB\nMajorPageFaults=%F\nMinorPageFaults=%R\nSwaps=%W\nForcedSwitches=%c\nWaitSwitches=%w\nInputs=%I\nOutputs=%O\nSocketReceived=%r\nSocketSent=%s\nSignals=%k\n' ./runpilot2-wrapper.sh -q BOINC_MCORE -j managed --pilot-user ATLAS --harvester-submit-mode PUSH -w generic --job-type managed --resource-type MCORE --pilotversion 3.10.5.57 -z -t --piloturl local --mute --container
[2025-08-28 00:00:01] + RESULT=2
[2025-08-28 00:00:01] + sed -i -e /nodename=/d /home/fedoravm/slots/5/u2ONDml2SB8n9Rq4apOajLDm4fhM0noT9bVof3QYDmh3eLDmcxjYim.diag
[2025-08-28 00:00:01] ++ awk -F '>' '{print $2}'
[2025-08-28 00:00:01] ++ grep domain_name init_data.xml
[2025-08-28 00:00:01] ++ awk -F '<' '{print $1}'
[2025-08-28 00:00:01] ++ sed -e 's# #_#g'
[2025-08-28 00:00:01] + hostname=fedora
[2025-08-28 00:00:01] ++ grep user_name init_data.xml
[2025-08-28 00:00:01] ++ awk -F '>' '{print $2}'
[2025-08-28 00:00:01] ++ awk -F '<' '{print $1}'
[2025-08-28 00:00:01] ++ sed -e 's# #_#g'
[2025-08-28 00:00:01] + username=Anchun
[2025-08-28 00:00:01] + nodename=Anchun@fedora
[2025-08-28 00:00:01] + echo nodename=Anchun@fedora
[2025-08-28 00:00:01] + '[' -n 12 ']'
[2025-08-28 00:00:01] + sed -i -e s/Processors=1/Processors=12/ /home/fedoravm/slots/5/u2ONDml2SB8n9Rq4apOajLDm4fhM0noT9bVof3QYDmh3eLDmcxjYim.diag
[2025-08-28 00:00:01] + echo exitcode=2
[2025-08-28 00:00:01] + runtimeenvironments=
[2025-08-28 00:00:01] + runtimeenvironments='APPS/HEP/ATLAS-SITE;'
[2025-08-28 00:00:01] + RTE_function_0 2
[2025-08-28 00:00:01] + export PROJECT_ROOT=/boincdata/boinc/project/lhcathome
[2025-08-28 00:00:01] + PROJECT_ROOT=/boincdata/boinc/project/lhcathome
[2025-08-28 00:00:01] + export BOINC_APP=ATLAS
[2025-08-28 00:00:01] + BOINC_APP=ATLAS
[2025-08-28 00:00:01] + export WU_TEMPLATE=templates/ATLAS_IN_DYNAMIC
[2025-08-28 00:00:01] + WU_TEMPLATE=templates/ATLAS_IN_DYNAMIC
[2025-08-28 00:00:01] + export RESULT_TEMPLATE=templates/ATLAS_OUT_2
[2025-08-28 00:00:01] + RESULT_TEMPLATE=templates/ATLAS_OUT_2
[2025-08-28 00:00:01] + '[' -z 2 ']'
[2025-08-28 00:00:01] + '[' 2 == 1 ']'
[2025-08-28 00:00:01] + '[' 0 -ne 0 ']'
[2025-08-28 00:00:01] + echo 'zip all output files'
[2025-08-28 00:00:01] + flist='*.diag '
[2025-08-28 00:00:01] + for f in ./heartbeat.json ./output.list ./log.46070568._065220.job.log.1 ./log.46070568._065220.job.log.1 ./gmlog/
[2025-08-28 00:00:01] + '[' -e ./heartbeat.json ']'
[2025-08-28 00:00:01] + flist='*.diag  ./heartbeat.json'
[2025-08-28 00:00:01] + for f in ./heartbeat.json ./output.list ./log.46070568._065220.job.log.1 ./log.46070568._065220.job.log.1 ./gmlog/
[2025-08-28 00:00:01] + '[' -e ./output.list ']'
[2025-08-28 00:00:01] + for f in ./heartbeat.json ./output.list ./log.46070568._065220.job.log.1 ./log.46070568._065220.job.log.1 ./gmlog/
[2025-08-28 00:00:01] + '[' -e ./log.46070568._065220.job.log.1 ']'
[2025-08-28 00:00:01] + flist='*.diag  ./heartbeat.json ./log.46070568._065220.job.log.1'
[2025-08-28 00:00:01] + for f in ./heartbeat.json ./output.list ./log.46070568._065220.job.log.1 ./log.46070568._065220.job.log.1 ./gmlog/
[2025-08-28 00:00:01] + '[' -e ./log.46070568._065220.job.log.1 ']'
[2025-08-28 00:00:01] + flist='*.diag  ./heartbeat.json ./log.46070568._065220.job.log.1 ./log.46070568._065220.job.log.1'
[2025-08-28 00:00:01] + for f in ./heartbeat.json ./output.list ./log.46070568._065220.job.log.1 ./log.46070568._065220.job.log.1 ./gmlog/
[2025-08-28 00:00:01] + '[' -e ./gmlog/ ']'
[2025-08-28 00:00:01] + '[' -f output.list ']'
[2025-08-28 00:00:01] + tar cvf result.tar.gz u2ONDml2SB8n9Rq4apOajLDm4fhM0noT9bVof3QYDmh3eLDmcxjYim.diag ./heartbeat.json ./log.46070568._065220.job.log.1 ./log.46070568._065220.job.log.1
[2025-08-28 00:00:01] + '[' xgnutime = xcgroup ']'
[2025-08-28 00:00:01] + '[' -n '' ']'
[2025-08-28 00:00:01] + '[' -n '' ']'
[2025-08-28 00:00:01] ++ date +%s
[2025-08-28 00:00:01] + ACCOUNTING_ENDTIME=1756364401
[2025-08-28 00:00:01] ++ date -d '1970-01-01 UTC 1756364190 seconds' +%Y%m%d%H%M%SZ
[2025-08-28 00:00:01] + echo LRMSStartTime=20250827235630Z
[2025-08-28 00:00:01] ++ date -d '1970-01-01 UTC 1756364401 seconds' +%Y%m%d%H%M%SZ
[2025-08-28 00:00:01] + echo LRMSEndTime=20250828000001Z
[2025-08-28 00:00:01] + echo walltime=211
[2025-08-28 00:00:01] + echo exitcode=2
[2025-08-28 00:00:01] + exit 2
[2025-08-28 00:00:01] ./runtime_log
[2025-08-28 00:00:01] ./runtime_log.err
[2025-08-28 00:00:01] ./log.46070568._065220.job.log.1
[2025-08-28 00:00:01] ./pilot3/pilot/util/__pycache__/loggingsupport.cpython-39.pyc
[2025-08-28 00:00:01] ./pilot3/pilot/util/__pycache__/realtimelogger.cpython-39.pyc
[2025-08-28 00:00:01] ./pilot3/pilot/util/__pycache__/lokirealtimelogger.cpython-39.pyc
[2025-08-28 00:00:01] ./pilot3/pilot/util/loggingsupport.py
[2025-08-28 00:00:01] ./pilot3/pilot/util/lokirealtimelogger.py
[2025-08-28 00:00:01] ./pilot3/pilot/util/realtimelogger.py
[2025-08-28 00:00:01] ./pilotlog.txt
[2025-08-28 00:00:01] ./PanDA_Pilot-6785752112/pilotlog.txt
[2025-08-28 00:00:01] ./PanDA_Pilot-6785752112/PoolFileCatalog.xml
[2025-08-28 00:00:01] ./PanDA_Pilot-6785752112/prmon.log
[2025-08-28 00:00:01] ./PanDA_Pilot-6785752112/log.EVNTtoHITS
00:10:01 (586469): run_atlas exited; CPU time 21.024090
00:10:01 (586469): app exit status: 0x1
00:10:01 (586469): called boinc_finish(195)

</stderr_txt>
]]>


What's wrong?

Note: I gave up configuring it on VM_Windows10, because I found that the overhead of nested virtualization is too high. (But in fact, I gave 131072MB to VM_Windows10.)
ID: 52128 · Report as offensive     Reply Quote
computezrmle
Volunteer moderator
Volunteer developer
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 15 Jun 08
Posts: 2681
Credit: 286,838,543
RAC: 64,716
Message 52129 - Posted: 28 Aug 2025, 8:26:49 UTC - in response to Message 52128.  

Your computer list shows ARM entries.
Remove this project from your ARM computers since it doesn't have any apps running on ARM.

This computer gets ATLAS tasks:
https://lhcathome.cern.ch/lhcathome/show_host_detail.php?hostid=10894282

The logs show apptainer is not installed locally.
Instead you use an apptainer version from CVMFS that may be too old for your OS.
Consider to install a recent apptainer package for your OS.

The logs show that you restart each task a couple of times.
ATLAS does not support checkpoints, hence in case of a restart it always starts from scratch.
Ensure the tasks run continuously to the end once they have been started.
ID: 52129 · Report as offensive     Reply Quote
Anchun

Send message
Joined: 20 Feb 25
Posts: 4
Credit: 14,909
RAC: 10
Message 52130 - Posted: 28 Aug 2025, 10:04:51 UTC - in response to Message 52129.  

I installed apptainer by running command 'sudo dnf install apptainer'. I also removed the ARM devices.
Now everything is working! Computations are running successfully!
Thank you for your help.
ID: 52130 · Report as offensive     Reply Quote

Questions and Answers : Unix/Linux : Amost all tasks fail on my computers


©2025 CERN