Name PQFODmd8I07nsSi4ap6QjLDmwznN0nGgGQJmUXzaDmknVLDm2pnbGn_2
Workunit 233835285
Created 27 Jul 2025, 21:39:21 UTC
Sent 28 Jul 2025, 0:14:45 UTC
Report deadline 5 Aug 2025, 0:14:45 UTC
Received 28 Jul 2025, 2:29:05 UTC
Server state Over
Outcome Success
Client state Done
Exit status 0 (0x00000000)
Computer ID 10877984
Run time 2 hours 3 min 16 sec
CPU time 11 hours 51 min 47 sec
Validate state Valid
Credit 308.74
Device peak FLOPS 12.00 GFLOPS
Application version ATLAS Simulation v3.01 (native_mt)
x86_64-pc-linux-gnu
Peak working set size 2.36 GB
Peak swap size 3.02 GB
Peak disk usage 496.85 MB

Stderr output

<core_client_version>8.1.0</core_client_version>
<![CDATA[
<stderr_txt>
20:14:57 (3183849): wrapper (7.7.26015): starting
20:14:57 (3183849): wrapper: running run_atlas (--nthreads 8)
[2025-07-27 20:14:57] Arguments: --nthreads 8
[2025-07-27 20:14:57] Threads: 8
[2025-07-27 20:14:57] Checking for CVMFS
[2025-07-27 20:14:57] Probing /cvmfs/atlas.cern.ch... OK
[2025-07-27 20:14:57] Probing /cvmfs/atlas-condb.cern.ch... OK
[2025-07-27 20:14:57] Running cvmfs_config stat atlas.cern.ch
[2025-07-27 20:14:57] VERSION PID UPTIME(M) MEM(K) REVISION EXPIRES(M) NOCATALOGS CACHEUSE(K) CACHEMAX(K) NOFDUSE NOFDMAX NOIOERR NOOPEN HITRATE(%) RX(K) SPEED(K/S) HOST PROXY ONLINE
[2025-07-27 20:14:57] 2.13.1.0 641173 14877 247212 148828 0 396 37902193 40960000 14892 16776704 0 52092516 99.809 51365545 40373 http://cvmfs-s1fnal.opensciencegrid.org:8000/cvmfs/atlas.cern.ch http://192.41.237.109:6081 1
[2025-07-27 20:14:57] CVMFS is ok
[2025-07-27 20:14:57] Efficiency of ATLAS tasks can be improved by the following measure(s):
[2025-07-27 20:14:57] The CVMFS client on this computer should be configured to use Cloudflare's openhtc.io.
[2025-07-27 20:14:57] Further information can be found at the LHC@home message board.
[2025-07-27 20:14:57] Using apptainer image /cvmfs/atlas.cern.ch/repo/containers/fs/singularity/x86_64-centos7
[2025-07-27 20:14:57] Checking for apptainer binary...
[2025-07-27 20:14:57] Using apptainer found in PATH at /usr/bin/apptainer
[2025-07-27 20:14:57] Running /usr/bin/apptainer --version
[2025-07-27 20:14:57] apptainer version 1.4.1-1.el9
[2025-07-27 20:14:57] Checking apptainer works with /usr/bin/apptainer exec -B /cvmfs /cvmfs/atlas.cern.ch/repo/containers/fs/singularity/x86_64-centos7 hostname
[2025-07-27 20:14:58] c-212-8.aglt2.org
[2025-07-27 20:14:58] apptainer works
[2025-07-27 20:14:58] Set ATHENA_PROC_NUMBER=8
[2025-07-27 20:14:58] Set ATHENA_CORE_NUMBER=8
[2025-07-27 20:14:58] Starting ATLAS job with PandaID=6747525616
[2025-07-27 20:14:58] Running command: /usr/bin/apptainer exec -B /cvmfs,/tmp/boinchome/slots/2 /cvmfs/atlas.cern.ch/repo/containers/fs/singularity/x86_64-centos7 sh start_atlas.sh
[2025-07-27 22:19:30]  *** The last 200 lines of the pilot log: ***
[2025-07-27 22:19:30] 2025-07-28 02:18:57,624 | WARNING  | format EVNTtoHITS has no such key: dbTime
[2025-07-27 22:19:30] 2025-07-28 02:18:57,626 | INFO     | fitting pss+swap vs Time
[2025-07-27 22:19:30] 2025-07-28 02:18:57,626 | INFO     | sum of square deviations: 549290726.9917359
[2025-07-27 22:19:30] 2025-07-28 02:18:57,627 | INFO     | sum of deviations: 17053206901.289253
[2025-07-27 22:19:30] 2025-07-28 02:18:57,627 | INFO     | mean x: 1753665412.0082645
[2025-07-27 22:19:30] 2025-07-28 02:18:57,627 | INFO     | mean y: 2502289.7107438017
[2025-07-27 22:19:30] 2025-07-28 02:18:57,627 | INFO     | -- intersect: -54441561003.31755
[2025-07-27 22:19:30] 2025-07-28 02:18:57,627 | INFO     | intersect: -54441561003.31755
[2025-07-27 22:19:30] 2025-07-28 02:18:57,627 | INFO     | chi2: 1.580796547801314
[2025-07-27 22:19:30] 2025-07-28 02:18:57,628 | INFO     | sum of square deviations: 483968876.9913803
[2025-07-27 22:19:30] 2025-07-28 02:18:57,628 | INFO     | sum of deviations: 16491753599.34482
[2025-07-27 22:19:30] 2025-07-28 02:18:57,628 | INFO     | mean x: 1753665259.5086207
[2025-07-27 22:19:30] 2025-07-28 02:18:57,628 | INFO     | mean y: 2500977.6551724137
[2025-07-27 22:19:30] 2025-07-28 02:18:57,628 | INFO     | -- intersect: -59755505643.2176
[2025-07-27 22:19:30] 2025-07-28 02:18:57,628 | INFO     | intersect: -59755505643.2176
[2025-07-27 22:19:30] 2025-07-28 02:18:57,629 | INFO     | chi2: 1.5824857000891348
[2025-07-27 22:19:30] 2025-07-28 02:18:57,629 | INFO     | current chi2=1.5824857000891348 (change=-0.10685450257151971 %)
[2025-07-27 22:19:30] 2025-07-28 02:18:57,630 | INFO     | right removable region: 115
[2025-07-27 22:19:30] 2025-07-28 02:18:57,630 | INFO     | sum of square deviations: 483968266.9913803
[2025-07-27 22:19:30] 2025-07-28 02:18:57,630 | INFO     | sum of deviations: -2477873678.9568973
[2025-07-27 22:19:30] 2025-07-28 02:18:57,630 | INFO     | mean x: 1753665564.5086207
[2025-07-27 22:19:30] 2025-07-28 02:18:57,630 | INFO     | mean y: 2547146.456896552
[2025-07-27 22:19:30] 2025-07-28 02:18:57,631 | INFO     | -- intersect: 8981155952.778475
[2025-07-27 22:19:30] 2025-07-28 02:18:57,631 | INFO     | intersect: 8981155952.778475
[2025-07-27 22:19:30] 2025-07-28 02:18:57,631 | INFO     | chi2: 0.07485311174021474
[2025-07-27 22:19:30] 2025-07-28 02:18:57,631 | INFO     | current chi2=0.07485311174021474 (change=95.26484848133519 %)
[2025-07-27 22:19:30] 2025-07-28 02:18:57,631 | INFO     | sum of square deviations: 424041256.9909904
[2025-07-27 22:19:30] 2025-07-28 02:18:57,632 | INFO     | sum of deviations: -6007364902.162161
[2025-07-27 22:19:30] 2025-07-28 02:18:57,632 | INFO     | mean x: 1753665717.0090091
[2025-07-27 22:19:30] 2025-07-28 02:18:57,632 | INFO     | mean y: 2555952.1621621624
[2025-07-27 22:19:30] 2025-07-28 02:18:57,632 | INFO     | -- intersect: 24846624081.855488
[2025-07-27 22:19:30] 2025-07-28 02:18:57,632 | INFO     | intersect: 24846624081.855488
[2025-07-27 22:19:30] 2025-07-28 02:18:57,632 | INFO     | chi2: 0.009156543906746058
[2025-07-27 22:19:30] 2025-07-28 02:18:57,632 | INFO     | current chi2=0.009156543906746058 (change=87.76731695734338 %)
[2025-07-27 22:19:30] 2025-07-28 02:18:57,632 | INFO     | sum of square deviations: 369277134.4905652
[2025-07-27 22:19:30] 2025-07-28 02:18:57,633 | INFO     | sum of deviations: -5880604267.792454
[2025-07-27 22:19:30] 2025-07-28 02:18:57,633 | INFO     | mean x: 1753665869.509434
[2025-07-27 22:19:30] 2025-07-28 02:18:57,633 | INFO     | mean y: 2555574.7924528304
[2025-07-27 22:19:30] 2025-07-28 02:18:57,633 | INFO     | -- intersect: 27929047722.067886
[2025-07-27 22:19:30] 2025-07-28 02:18:57,633 | INFO     | intersect: 27929047722.067886
[2025-07-27 22:19:30] 2025-07-28 02:18:57,633 | INFO     | chi2: 0.0075606085301527645
[2025-07-27 22:19:30] 2025-07-28 02:18:57,634 | INFO     | current chi2=0.0075606085301527645 (change=17.429451470412243 %)
[2025-07-27 22:19:30] 2025-07-28 02:18:57,634 | INFO     | left removable region: 30
[2025-07-27 22:19:30] 2025-07-28 02:18:57,634 | INFO     | sum of square deviations: 190398812.98823565
[2025-07-27 22:19:30] 2025-07-28 02:18:57,634 | INFO     | sum of deviations: -2955036701.458823
[2025-07-27 22:19:30] 2025-07-28 02:18:57,634 | INFO     | mean x: 1753666144.0117648
[2025-07-27 22:19:30] 2025-07-28 02:18:57,634 | INFO     | mean y: 2547337.4588235295
[2025-07-27 22:19:30] 2025-07-28 02:18:57,634 | INFO     | -- intersect: 27219879926.505653
[2025-07-27 22:19:30] 2025-07-28 02:18:57,635 | INFO     | intersect: 27219879926.505653
[2025-07-27 22:19:30] 2025-07-28 02:18:57,635 | INFO     | chi2: 0.006367666839507775
[2025-07-27 22:19:30] 2025-07-28 02:18:57,635 | INFO     | -- intersect: 27219879926.505653
[2025-07-27 22:19:30] 2025-07-28 02:18:57,635 | INFO     | current memory leak: -15.52 B/s (using 85 data points, chi2=0.01)
[2025-07-27 22:19:30] 2025-07-28 02:18:57,635 | INFO     | could have reported an average CPU frequency of 2984 MHz (8 samples)
[2025-07-27 22:19:30] 2025-07-28 02:18:57,636 | INFO     | ..............................
[2025-07-27 22:19:30] 2025-07-28 02:18:57,636 | INFO     | . Timing measurements:
[2025-07-27 22:19:30] 2025-07-28 02:18:57,636 | INFO     | . get job = 0 s
[2025-07-27 22:19:30] 2025-07-28 02:18:57,636 | INFO     | . initial setup = 2 s
[2025-07-27 22:19:30] 2025-07-28 02:18:57,636 | INFO     | . payload setup = 12 s
[2025-07-27 22:19:30] 2025-07-28 02:18:57,636 | INFO     | . stage-in = 0 s
[2025-07-27 22:19:30] 2025-07-28 02:18:57,636 | INFO     | . payload execution = 7384 s
[2025-07-27 22:19:30] 2025-07-28 02:18:57,636 | INFO     | . stage-out = 0 s
[2025-07-27 22:19:30] 2025-07-28 02:18:57,636 | INFO     | . log creation = 0 s
[2025-07-27 22:19:30] 2025-07-28 02:18:57,636 | INFO     | ..............................
[2025-07-27 22:19:30] 2025-07-28 02:18:57,714 | INFO     | 
[2025-07-27 22:19:30] 2025-07-28 02:18:57,714 | INFO     | job summary report
[2025-07-27 22:19:30] 2025-07-28 02:18:57,714 | INFO     | --------------------------------------------------
[2025-07-27 22:19:30] 2025-07-28 02:18:57,714 | INFO     | PanDA job id: 6747525616
[2025-07-27 22:19:30] 2025-07-28 02:18:57,714 | INFO     | task id: 45743349
[2025-07-27 22:19:30] 2025-07-28 02:18:57,715 | INFO     | errors: (none)
[2025-07-27 22:19:30] 2025-07-28 02:18:57,715 | INFO     | status: LOG_TRANSFER = DONE 
[2025-07-27 22:19:30] 2025-07-28 02:18:57,715 | INFO     | pilot state: finished 
[2025-07-27 22:19:30] 2025-07-28 02:18:57,715 | INFO     | transexitcode: 0
[2025-07-27 22:19:30] 2025-07-28 02:18:57,715 | INFO     | exeerrorcode: 0
[2025-07-27 22:19:30] 2025-07-28 02:18:57,715 | INFO     | exeerrordiag: 
[2025-07-27 22:19:30] 2025-07-28 02:18:57,715 | INFO     | exitcode: 0
[2025-07-27 22:19:30] 2025-07-28 02:18:57,715 | INFO     | exitmsg: OK
[2025-07-27 22:19:30] 2025-07-28 02:18:57,716 | INFO     | cpuconsumptiontime: 42129 s
[2025-07-27 22:19:30] 2025-07-28 02:18:57,716 | INFO     | nevents: 400
[2025-07-27 22:19:30] 2025-07-28 02:18:57,716 | INFO     | neventsw: 0
[2025-07-27 22:19:30] 2025-07-28 02:18:57,719 | INFO     | pid: 3196900
[2025-07-27 22:19:30] 2025-07-28 02:18:57,720 | INFO     | pgrp: 3196900
[2025-07-27 22:19:30] 2025-07-28 02:18:57,720 | INFO     | corecount: 8
[2025-07-27 22:19:30] 2025-07-28 02:18:57,720 | INFO     | event service: False
[2025-07-27 22:19:30] 2025-07-28 02:18:57,720 | INFO     | sizes: {0: 2397475, 7: 2397737, 11: 2397765, 7400: 2425161, 7401: 2434159, 7402: 2434215, 7408: 2434565}
[2025-07-27 22:19:30] 2025-07-28 02:18:57,720 | INFO     | --------------------------------------------------
[2025-07-27 22:19:30] 2025-07-28 02:18:57,720 | INFO     | 
[2025-07-27 22:19:30] 2025-07-28 02:18:57,720 | INFO     | executing command: ls -lF /tmp/boinchome/slots/2
[2025-07-27 22:19:30] 2025-07-28 02:18:57,742 | INFO     | queue jobs had 0 job(s) [purged]
[2025-07-27 22:19:30] 2025-07-28 02:18:57,743 | INFO     | queue payloads had 0 job(s) [purged]
[2025-07-27 22:19:30] 2025-07-28 02:18:57,743 | INFO     | queue data_in had 0 job(s) [purged]
[2025-07-27 22:19:30] 2025-07-28 02:18:57,743 | INFO     | queue data_out had 0 job(s) [purged]
[2025-07-27 22:19:30] 2025-07-28 02:18:57,743 | INFO     | queue current_data_in had 0 job(s) [purged]
[2025-07-27 22:19:30] 2025-07-28 02:18:57,743 | INFO     | queue validated_jobs had 0 job(s) [purged]
[2025-07-27 22:19:30] 2025-07-28 02:18:57,744 | INFO     | queue validated_payloads had 0 job(s) [purged]
[2025-07-27 22:19:30] 2025-07-28 02:18:57,744 | INFO     | queue monitored_payloads had 0 job(s) [purged]
[2025-07-27 22:19:30] 2025-07-28 02:18:57,744 | INFO     | queue finished_jobs had 0 job(s) [purged]
[2025-07-27 22:19:30] 2025-07-28 02:18:57,744 | INFO     | queue finished_payloads had 0 job(s) [purged]
[2025-07-27 22:19:30] 2025-07-28 02:18:57,744 | INFO     | queue finished_data_in had 1 job(s) [purged]
[2025-07-27 22:19:30] 2025-07-28 02:18:57,744 | INFO     | queue finished_data_out had 0 job(s) [purged]
[2025-07-27 22:19:30] 2025-07-28 02:18:57,744 | INFO     | queue failed_jobs had 0 job(s) [purged]
[2025-07-27 22:19:30] 2025-07-28 02:18:57,745 | INFO     | queue failed_payloads had 0 job(s) [purged]
[2025-07-27 22:19:30] 2025-07-28 02:18:57,745 | INFO     | queue failed_data_in had 0 job(s) [purged]
[2025-07-27 22:19:30] 2025-07-28 02:18:57,745 | INFO     | queue failed_data_out had 0 job(s) [purged]
[2025-07-27 22:19:30] 2025-07-28 02:18:57,745 | INFO     | queue completed_jobs had 0 job(s) [purged]
[2025-07-27 22:19:30] 2025-07-28 02:18:57,745 | INFO     | queue completed_jobids has 1 job(s)
[2025-07-27 22:19:30] 2025-07-28 02:18:57,745 | INFO     | queue realtimelog_payloads had 0 job(s) [purged]
[2025-07-27 22:19:30] 2025-07-28 02:18:57,745 | INFO     | queue messages had 0 job(s) [purged]
[2025-07-27 22:19:30] 2025-07-28 02:18:57,745 | INFO     | job 6747525616 has completed (purged errors)
[2025-07-27 22:19:30] 2025-07-28 02:18:57,745 | INFO     | overall cleanup function is called
[2025-07-27 22:19:30] 2025-07-28 02:18:58,111 | INFO     | number of running child processes to parent process 3196900: 1
[2025-07-27 22:19:30] 2025-07-28 02:18:58,111 | INFO     | maximum number of monitored processes: 6
[2025-07-27 22:19:30] 2025-07-28 02:18:58,111 | INFO     | aborting job monitoring since job object (job id=6747525616) has expired
[2025-07-27 22:19:30] 2025-07-28 02:18:58,757 | INFO     | --- collectZombieJob: --- 10, [3196900]
[2025-07-27 22:19:30] 2025-07-28 02:18:58,758 | INFO     | zombie collector waiting for pid 3196900
[2025-07-27 22:19:30] 2025-07-28 02:18:58,758 | INFO     | harmless exception when collecting zombies: [Errno 10] No child processes
[2025-07-27 22:19:30] 2025-07-28 02:18:58,758 | INFO     | collected zombie processes
[2025-07-27 22:19:30] 2025-07-28 02:18:58,758 | INFO     | will attempt to kill all subprocesses of pid=3196900
[2025-07-27 22:19:30] 2025-07-28 02:18:59,163 | INFO     | process IDs to be killed: [3196900] (in reverse order)
[2025-07-27 22:19:30] 2025-07-28 02:18:59,290 | WARNING  | found no corresponding commands to process id(s)
[2025-07-27 22:19:30] 2025-07-28 02:18:59,290 | INFO     | Do not look for orphan processes in BOINC jobs
[2025-07-27 22:19:30] 2025-07-28 02:18:59,307 | INFO     | did not find any defunct processes belonging to 3196900
[2025-07-27 22:19:30] 2025-07-28 02:18:59,324 | INFO     | did not find any defunct processes belonging to 3196900
[2025-07-27 22:19:30] 2025-07-28 02:18:59,324 | INFO     | ready for new job
[2025-07-27 22:19:30] 2025-07-28 02:18:59,324 | INFO     | pilot has finished with previous job - re-establishing logging
[2025-07-27 22:19:30] 2025-07-28 02:18:59,326 | INFO     | ***************************************
[2025-07-27 22:19:30] 2025-07-28 02:18:59,326 | INFO     | ***  PanDA Pilot version 3.10.4.12  ***
[2025-07-27 22:19:30] 2025-07-28 02:18:59,326 | INFO     | ***************************************
[2025-07-27 22:19:30] 2025-07-28 02:18:59,326 | INFO     | 
[2025-07-27 22:19:30] 2025-07-28 02:18:59,331 | INFO     | architecture information:
[2025-07-27 22:19:30] 2025-07-28 02:18:59,332 | INFO     | executing command: cat /etc/os-release
[2025-07-27 22:19:30] 2025-07-28 02:18:59,348 | INFO     | cat /etc/os-release:
[2025-07-27 22:19:30] NAME="CentOS Linux"
[2025-07-27 22:19:30] VERSION="7 (Core)"
[2025-07-27 22:19:30] ID="centos"
[2025-07-27 22:19:30] ID_LIKE="rhel fedora"
[2025-07-27 22:19:30] VERSION_ID="7"
[2025-07-27 22:19:30] PRETTY_NAME="CentOS Linux 7 (Core)"
[2025-07-27 22:19:30] ANSI_COLOR="0;31"
[2025-07-27 22:19:30] CPE_NAME="cpe:/o:centos:centos:7"
[2025-07-27 22:19:30] HOME_URL="https://www.centos.org/"
[2025-07-27 22:19:30] BUG_REPORT_URL="https://bugs.centos.org/"
[2025-07-27 22:19:30] 
[2025-07-27 22:19:30] CENTOS_MANTISBT_PROJECT="CentOS-7"
[2025-07-27 22:19:30] CENTOS_MANTISBT_PROJECT_VERSION="7"
[2025-07-27 22:19:30] REDHAT_SUPPORT_PRODUCT="centos"
[2025-07-27 22:19:30] REDHAT_SUPPORT_PRODUCT_VERSION="7"
[2025-07-27 22:19:30] 
[2025-07-27 22:19:30] 2025-07-28 02:18:59,349 | INFO     | ***************************************
[2025-07-27 22:19:30] 2025-07-28 02:18:59,852 | INFO     | executing command: df -mP /tmp/boinchome/slots/2
[2025-07-27 22:19:30] 2025-07-28 02:18:59,871 | INFO     | sufficient remaining disk space (129103822848 B)
[2025-07-27 22:19:30] 2025-07-28 02:18:59,871 | WARNING  | since timefloor is set to 0, pilot was only allowed to run one job
[2025-07-27 22:19:30] 2025-07-28 02:18:59,871 | INFO     | current server update state: UPDATING_FINAL
[2025-07-27 22:19:30] 2025-07-28 02:18:59,871 | INFO     | update_server=False
[2025-07-27 22:19:30] 2025-07-28 02:18:59,871 | WARNING  | setting graceful_stop since proceed_with_getjob() returned False (pilot will end)
[2025-07-27 22:19:30] 2025-07-28 02:18:59,871 | WARNING  | job:job_monitor:received graceful stop - abort after this iteration
[2025-07-27 22:19:30] 2025-07-28 02:18:59,871 | INFO     | aborting loop
[2025-07-27 22:19:30] 2025-07-28 02:19:00,127 | INFO     | all job control threads have been joined
[2025-07-27 22:19:30] 2025-07-28 02:19:00,211 | INFO     | all payload control threads have been joined
[2025-07-27 22:19:30] 2025-07-28 02:19:00,422 | INFO     | all data control threads have been joined
[2025-07-27 22:19:30] 2025-07-28 02:19:00,877 | INFO     | [job] retrieve thread has finished
[2025-07-27 22:19:30] 2025-07-28 02:19:00,877 | INFO     | [job] job monitor thread has finished
[2025-07-27 22:19:30] 2025-07-28 02:19:01,132 | INFO     | [job] control thread has finished
[2025-07-27 22:19:30] 2025-07-28 02:19:01,154 | INFO     | [job] validate thread has finished
[2025-07-27 22:19:30] 2025-07-28 02:19:01,217 | INFO     | [payload] control thread has finished
[2025-07-27 22:19:30] 2025-07-28 02:19:01,368 | WARNING  | data:copytool_out:received graceful stop - abort after this iteration
[2025-07-27 22:19:30] 2025-07-28 02:19:01,428 | INFO     | [data] control thread has finished
[2025-07-27 22:19:30] 2025-07-28 02:19:01,523 | INFO     | [payload] execute_payloads thread has finished
[2025-07-27 22:19:30] 2025-07-28 02:19:01,624 | INFO     | [job] create_data_payload thread has finished
[2025-07-27 22:19:30] 2025-07-28 02:19:01,865 | INFO     | [data] copytool_in thread has finished
[2025-07-27 22:19:30] 2025-07-28 02:19:01,904 | INFO     | [payload] run_realtimelog thread has finished
[2025-07-27 22:19:30] 2025-07-28 02:19:02,147 | INFO     | [payload] validate_pre thread has finished
[2025-07-27 22:19:30] 2025-07-28 02:19:02,212 | WARNING  | data:queue_monitoring:received graceful stop - abort after this iteration
[2025-07-27 22:19:30] 2025-07-28 02:19:02,315 | INFO     | [payload] validate_post thread has finished
[2025-07-27 22:19:30] 2025-07-28 02:19:02,393 | INFO     | [payload] failed_post thread has finished
[2025-07-27 22:19:30] 2025-07-28 02:19:02,723 | WARNING  | job:queue_monitor:received graceful stop - abort after this iteration
[2025-07-27 22:19:30] 2025-07-28 02:19:03,374 | INFO     | [data] copytool_out thread has finished
[2025-07-27 22:19:30] 2025-07-28 02:19:03,729 | INFO     | [job] queue monitor thread has finished
[2025-07-27 22:19:30] 2025-07-28 02:19:06,218 | INFO     | [data] queue_monitor thread has finished
[2025-07-27 22:19:30] 2025-07-28 02:19:06,435 | INFO     | only monitor.control thread still running - safe to abort: ['<_MainThread(MainThread, started 140507715102528)>', '<ExcThread(monitor, started 140507061659392)>']
[2025-07-27 22:19:30] 2025-07-28 02:19:11,460 | INFO     | all workflow threads have been joined
[2025-07-27 22:19:30] 2025-07-28 02:19:11,461 | INFO     | end of generic workflow (traces error code: 0)
[2025-07-27 22:19:30] 2025-07-28 02:19:11,462 | INFO     | traces error code: 0
[2025-07-27 22:19:30] 2025-07-28 02:19:11,462 | INFO     | pilot has finished (exit code=0, shell exit code=0)
[2025-07-27 22:19:30] 2025-07-28 02:19:30,294 | INFO     | PID=3188208 has CPU usage=5.6% CMD=/cvmfs/atlas.cern.ch/repo/ATLASLocalRootBase/x86_64/python/3.9.20-x86_64-centos7/bin/python3 pilot3/pilot.py -q BOINC_MCORE -i P
[2025-07-27 22:19:30] 2025-07-28 02:19:30,294 | INFO     | .. there are 16 such processes running
[2025-07-27 22:19:30] 2025-07-28 02:19:30,295 | INFO     | found 0 job(s) in 20 queues
[2025-07-27 22:19:30] 2025-07-28 02:19:30,295 | WARNING  | pilot monitor received instruction that args.graceful_stop has been set
[2025-07-27 22:19:30] 2025-07-28 02:19:30,295 | WARNING  | will wait for a maximum of 300 s for threads to finish
[2025-07-27 22:19:30] 2025-07-28 02:19:30,295 | WARNING  | job_aborted has been set - aborting pilot monitoring
[2025-07-27 22:19:30] 2025-07-28 02:19:30,295 | INFO     | [monitor] control thread has ended
[2025-07-27 22:19:30] 2025-07-28 02:19:30,433 [wrapper] ==== pilot stdout END ====
[2025-07-27 22:19:30] 2025-07-28 02:19:30,436 [wrapper] ==== wrapper stdout RESUME ====
[2025-07-27 22:19:30] 2025-07-28 02:19:30,438 [wrapper] pilotpid: 3188208
[2025-07-27 22:19:30] 2025-07-28 02:19:30,440 [wrapper] Pilot exit status: 0
[2025-07-27 22:19:30] 2025-07-28 02:19:30,451 [wrapper] pandaids: 6747525616
[2025-07-27 22:19:30] 2025-07-28 02:19:30,540 [wrapper] cleanup supervisor_pilot 3722160 3188209
[2025-07-27 22:19:30] 2025-07-28 02:19:30,544 [wrapper] Test setup, not cleaning
[2025-07-27 22:19:30] 2025-07-28 02:19:30,548 [wrapper] apfmon messages muted
[2025-07-27 22:19:30] 2025-07-28 02:19:30,553 [wrapper] ==== wrapper stdout END ====
[2025-07-27 22:19:30] 2025-07-28 02:19:30,559 [wrapper] ==== wrapper stderr END ====
[2025-07-27 22:19:30]  *** Error codes and diagnostics ***
[2025-07-27 22:19:30]     "exeErrorCode": 0,
[2025-07-27 22:19:30]     "exeErrorDiag": "",
[2025-07-27 22:19:30]     "pilotErrorCode": 0,
[2025-07-27 22:19:30]     "pilotErrorDiag": "",
[2025-07-27 22:19:30]  *** Listing of results directory ***
[2025-07-27 22:19:30] total 359904
[2025-07-27 22:19:30] -rw-r--r--. 1 boincer umatlas    530466 Jul 27 12:02 pilot3.tar.gz
[2025-07-27 22:19:30] -rw-r--r--. 1 boincer umatlas      5112 Jul 27 12:09 queuedata.json
[2025-07-27 22:19:30] -rwx------. 1 boincer umatlas     36292 Jul 27 12:10 runpilot2-wrapper.sh
[2025-07-27 22:19:30] -rw-r--r--. 1 boincer umatlas       100 Jul 27 20:14 wrapper_26015_x86_64-pc-linux-gnu
[2025-07-27 22:19:30] -rwxr-xr-x. 1 boincer umatlas      7986 Jul 27 20:14 run_atlas
[2025-07-27 22:19:30] -rw-r--r--. 1 boincer umatlas       105 Jul 27 20:14 job.xml
[2025-07-27 22:19:30] -rw-r--r--. 2 boincer umatlas 148220471 Jul 27 20:14 EVNT.45743347._002871.pool.root.1
[2025-07-27 22:19:30] -rw-r--r--. 2 boincer umatlas     15095 Jul 27 20:14 start_atlas.sh
[2025-07-27 22:19:30] drwxrwx--x. 2 boincer umatlas      4096 Jul 27 20:14 shared
[2025-07-27 22:19:30] -rw-r--r--. 2 boincer umatlas    542898 Jul 27 20:14 input.tar.gz
[2025-07-27 22:19:30] -rw-r--r--. 1 boincer umatlas         0 Jul 27 20:14 boinc_lockfile
[2025-07-27 22:19:30] -rw-r--r--. 1 boincer umatlas      2563 Jul 27 20:14 pandaJob.out
[2025-07-27 22:19:30] -rw-------. 1 boincer umatlas    980125 Jul 27 20:15 agis_schedconf.cvmfs.json
[2025-07-27 22:19:30] -rw-------. 1 boincer umatlas   1593052 Jul 27 20:15 agis_ddmendpoints.agis.ALL.json
[2025-07-27 22:19:30] drwx------. 4 boincer umatlas      4096 Jul 27 20:15 pilot3
[2025-07-27 22:19:30] -rw-r--r--. 1 boincer umatlas       529 Jul 27 22:18 boinc_task_state.xml
[2025-07-27 22:19:30] -rw-------. 1 boincer umatlas 214377456 Jul 27 22:18 HITS.45743349._052163.pool.root.1
[2025-07-27 22:19:30] -rw-r--r--. 1 boincer umatlas      6943 Jul 27 22:18 init_data.xml
[2025-07-27 22:19:30] -rw-------. 1 boincer umatlas      1032 Jul 27 22:18 memory_monitor_summary.json
[2025-07-27 22:19:30] -rw-------. 1 boincer umatlas    334545 Jul 27 22:18 log.45743349._052163.job.log.tgz.1
[2025-07-27 22:19:30] -rw-------. 1 boincer umatlas      7766 Jul 27 22:18 heartbeat.json
[2025-07-27 22:19:30] -rw-r--r--. 1 boincer umatlas      8192 Jul 27 22:19 boinc_mmap_file
[2025-07-27 22:19:30] -rw-r--r--. 1 boincer umatlas        27 Jul 27 22:19 wrapper_checkpoint.txt
[2025-07-27 22:19:30] -rw-------. 1 boincer umatlas       823 Jul 27 22:19 pilotlog.txt
[2025-07-27 22:19:30] -rw-------. 1 boincer umatlas        95 Jul 27 22:19 pilot_heartbeat.json
[2025-07-27 22:19:30] -rw-------. 1 boincer umatlas    699469 Jul 27 22:19 log.45743349._052163.job.log.1
[2025-07-27 22:19:30] -rw-------. 1 boincer umatlas       357 Jul 27 22:19 output.list
[2025-07-27 22:19:30] -rw-r--r--. 1 boincer umatlas       620 Jul 27 22:19 runtime_log
[2025-07-27 22:19:30] -rw-------. 1 boincer umatlas   1054720 Jul 27 22:19 result.tar.gz
[2025-07-27 22:19:30] -rw-r--r--. 1 boincer umatlas      8655 Jul 27 22:19 runtime_log.err
[2025-07-27 22:19:30] -rw-------. 1 boincer umatlas       653 Jul 27 22:19 PQFODmd8I07nsSi4ap6QjLDmwznN0nGgGQJmUXzaDmknVLDm2pnbGn.diag
[2025-07-27 22:19:30] -rw-r--r--. 1 boincer umatlas     21043 Jul 27 22:19 stderr.txt
[2025-07-27 22:19:30] HITS file was successfully produced:
[2025-07-27 22:19:30] -rw-------. 1 boincer umatlas 214377456 Jul 27 22:18 shared/HITS.pool.root.1
[2025-07-27 22:19:30]  *** Contents of shared directory: ***
[2025-07-27 22:19:30] total 355688
[2025-07-27 22:19:30] -rw-r--r--. 2 boincer umatlas 148220471 Jul 27 20:14 ATLAS.root_0
[2025-07-27 22:19:30] -rw-r--r--. 2 boincer umatlas     15095 Jul 27 20:14 start_atlas.sh
[2025-07-27 22:19:30] -rw-r--r--. 2 boincer umatlas    542898 Jul 27 20:14 input.tar.gz
[2025-07-27 22:19:30] -rw-------. 1 boincer umatlas 214377456 Jul 27 22:18 HITS.pool.root.1
[2025-07-27 22:19:30] -rw-------. 1 boincer umatlas   1054720 Jul 27 22:19 result.tar.gz
22:19:32 (3183849): run_atlas exited; CPU time 42344.801747
22:19:32 (3183849): called boinc_finish(0)

</stderr_txt>
]]>


©2025 CERN