| Name | 7VXMDm6dcW8n9Rq4apOajLDm4fhM0noT9bVof3QYDmtwlMDmNguVOm_1 |
| Workunit | 237065304 |
| Created | 24 Oct 2025, 21:59:52 UTC |
| Sent | 24 Oct 2025, 23:52:58 UTC |
| Report deadline | 1 Nov 2025, 23:52:58 UTC |
| Received | 25 Oct 2025, 14:47:18 UTC |
| Server state | Over |
| Outcome | Success |
| Client state | Done |
| Exit status | 0 (0x00000000) |
| Computer ID | 10878280 |
| Run time | 14 hours 36 min 1 sec |
| CPU time | 1 days 7 hours 6 min 59 sec |
| Validate state | Valid |
| Credit | 1,801.71 |
| Device peak FLOPS | 24.75 GFLOPS |
| Application version | ATLAS Simulation v3.01 (native_mt) x86_64-pc-linux-gnu |
| Peak working set size | 2.74 GB |
| Peak swap size | 31.77 GB |
| Peak disk usage | 1.03 GB |
Stderr output
<core_client_version>8.1.0</core_client_version>
<![CDATA[
<stderr_txt>
19:53:14 (3867790): wrapper (7.7.26015): starting
19:53:14 (3867790): wrapper: running run_atlas (--nthreads 10)
[2025-10-24 19:53:14] Arguments: --nthreads 10
[2025-10-24 19:53:14] Threads: 10
[2025-10-24 19:53:14] Checking for CVMFS
[2025-10-24 19:53:14] Probing /cvmfs/atlas.cern.ch... OK
[2025-10-24 19:53:14] Probing /cvmfs/atlas-condb.cern.ch... OK
[2025-10-24 19:53:14] Running cvmfs_config stat atlas.cern.ch
[2025-10-24 19:53:15] VERSION PID UPTIME(M) MEM(K) REVISION EXPIRES(M) NOCATALOGS CACHEUSE(K) CACHEMAX(K) NOFDUSE NOFDMAX NOIOERR NOOPEN HITRATE(%) RX(K) SPEED(K/S) HOST PROXY ONLINE
[2025-10-24 19:53:15] 2.13.2.0 128802 135254 143680 152099 0 150 38868269 39288832 11122 16776704 0 103725082 99.666 89131624 32539 http://cvmfs-s1fnal.opensciencegrid.org:8000/cvmfs/atlas.cern.ch http://192.41.237.109:6081 1
[2025-10-24 19:53:15] CVMFS is ok
[2025-10-24 19:53:15] Efficiency of ATLAS tasks can be improved by the following measure(s):
[2025-10-24 19:53:15] The CVMFS client on this computer should be configured to use Cloudflare's openhtc.io.
[2025-10-24 19:53:15] Further information can be found at the LHC@home message board.
[2025-10-24 19:53:15] Using apptainer image /cvmfs/atlas.cern.ch/repo/containers/fs/singularity/x86_64-centos7
[2025-10-24 19:53:15] Checking for apptainer binary...
[2025-10-24 19:53:15] Using apptainer found in PATH at /usr/bin/apptainer
[2025-10-24 19:53:15] Running /usr/bin/apptainer --version
[2025-10-24 19:53:15] apptainer version 1.4.3-1.el9
[2025-10-24 19:53:15] Checking apptainer works with /usr/bin/apptainer exec -B /cvmfs /cvmfs/atlas.cern.ch/repo/containers/fs/singularity/x86_64-centos7 hostname
[2025-10-24 19:53:15] c-210-7.aglt2.org
[2025-10-24 19:53:15] apptainer works
[2025-10-24 19:53:15] Set ATHENA_PROC_NUMBER=10
[2025-10-24 19:53:15] Set ATHENA_CORE_NUMBER=10
[2025-10-24 19:53:15] Starting ATLAS job with PandaID=6855925172
[2025-10-24 19:53:15] Running command: /usr/bin/apptainer exec -B /cvmfs,/tmp/boinchome/slots/1 /cvmfs/atlas.cern.ch/repo/containers/fs/singularity/x86_64-centos7 sh start_atlas.sh
[2025-10-24 23:40:01] *** The last 200 lines of the pilot log: ***
[2025-10-24 23:40:01] 2025-10-25 03:38:55,505 | WARNING | format EVNTtoHITS has no such key: dbTime
[2025-10-24 23:40:01] 2025-10-25 03:38:55,517 | INFO | fitting pss+swap vs Time
[2025-10-24 23:40:01] 2025-10-25 03:38:55,518 | INFO | sum of square deviations: 3301265048.1090918
[2025-10-24 23:40:01] 2025-10-25 03:38:55,520 | INFO | sum of deviations: -28895960640.36364
[2025-10-24 23:40:01] 2025-10-25 03:38:55,520 | INFO | mean x: 1761356801.4363637
[2025-10-24 23:40:01] 2025-10-25 03:38:55,520 | INFO | mean y: 2573780.0454545454
[2025-10-24 23:40:01] 2025-10-25 03:38:55,520 | INFO | -- intersect: 15419723286.745735
[2025-10-24 23:40:01] 2025-10-25 03:38:55,528 | INFO | intersect: 15419723286.745735
[2025-10-24 23:40:01] 2025-10-25 03:38:55,529 | INFO | chi2: 4.486110018184544
[2025-10-24 23:40:01] 2025-10-25 03:38:55,529 | INFO | sum of square deviations: 3081273363.534897
[2025-10-24 23:40:01] 2025-10-25 03:38:55,531 | INFO | sum of deviations: 15470876610.837196
[2025-10-24 23:40:01] 2025-10-25 03:38:55,531 | INFO | mean x: 1761356648.9534883
[2025-10-24 23:40:01] 2025-10-25 03:38:55,531 | INFO | mean y: 2604259.88372093
[2025-10-24 23:40:01] 2025-10-25 03:38:55,531 | INFO | -- intersect: -8841054892.894495
[2025-10-24 23:40:01] 2025-10-25 03:38:55,532 | INFO | intersect: -8841054892.894495
[2025-10-24 23:40:01] 2025-10-25 03:38:55,532 | INFO | chi2: 2.2563817385678364
[2025-10-24 23:40:01] 2025-10-25 03:38:55,532 | INFO | current chi2=2.2563817385678364 (change=49.702933512072946 %)
[2025-10-24 23:40:01] 2025-10-25 03:38:55,541 | INFO | sum of square deviations: 2871281378.328571
[2025-10-24 23:40:01] 2025-10-25 03:38:55,542 | INFO | sum of deviations: 14993958274.557137
[2025-10-24 23:40:01] 2025-10-25 03:38:55,543 | INFO | mean x: 1761356496.4714286
[2025-10-24 23:40:01] 2025-10-25 03:38:55,543 | INFO | mean y: 2603913.5095238094
[2025-10-24 23:40:01] 2025-10-25 03:38:55,543 | INFO | -- intersect: -9195277566.878983
[2025-10-24 23:40:01] 2025-10-25 03:38:55,543 | INFO | intersect: -9195277566.878983
[2025-10-24 23:40:01] 2025-10-25 03:38:55,543 | INFO | chi2: 2.2575353530777806
[2025-10-24 23:40:01] 2025-10-25 03:38:55,544 | INFO | current chi2=2.2575353530777806 (change=-0.05112674376971521 %)
[2025-10-24 23:40:01] 2025-10-25 03:38:55,544 | INFO | right removable region: 209
[2025-10-24 23:40:01] 2025-10-25 03:38:55,544 | INFO | sum of square deviations: 3081160878.623267
[2025-10-24 23:40:01] 2025-10-25 03:38:55,554 | INFO | sum of deviations: -73707561257.83252
[2025-10-24 23:40:01] 2025-10-25 03:38:55,554 | INFO | mean x: 1761356953.9581394
[2025-10-24 23:40:01] 2025-10-25 03:38:55,554 | INFO | mean y: 2604600.018604651
[2025-10-24 23:40:01] 2025-10-25 03:38:55,554 | INFO | -- intersect: 42137803213.46711
[2025-10-24 23:40:01] 2025-10-25 03:38:55,554 | INFO | intersect: 42137803213.46711
[2025-10-24 23:40:01] 2025-10-25 03:38:55,555 | INFO | chi2: 2.579318558468318
[2025-10-24 23:40:01] 2025-10-25 03:38:55,555 | INFO | current chi2=2.579318558468318 (change=42.50434010728683 %)
[2025-10-24 23:40:01] 2025-10-25 03:38:55,555 | INFO | sum of square deviations: 2871083494.328571
[2025-10-24 23:40:01] 2025-10-25 03:38:55,565 | INFO | sum of deviations: -75308760057.45718
[2025-10-24 23:40:01] 2025-10-25 03:38:55,565 | INFO | mean x: 1761357106.4714286
[2025-10-24 23:40:01] 2025-10-25 03:38:55,565 | INFO | mean y: 2605713.5904761907
[2025-10-24 23:40:01] 2025-10-25 03:38:55,566 | INFO | -- intersect: 46203149852.685356
[2025-10-24 23:40:01] 2025-10-25 03:38:55,566 | INFO | intersect: 46203149852.685356
[2025-10-24 23:40:01] 2025-10-25 03:38:55,566 | INFO | chi2: 2.5461350377842615
[2025-10-24 23:40:01] 2025-10-25 03:38:55,566 | INFO | current chi2=2.5461350377842615 (change=1.286522774595238 %)
[2025-10-24 23:40:01] 2025-10-25 03:38:55,566 | INFO | left removable region: 20
[2025-10-24 23:40:01] 2025-10-25 03:38:55,567 | INFO | sum of square deviations: 2092949878.8677197
[2025-10-24 23:40:01] 2025-10-25 03:38:55,568 | INFO | sum of deviations: -11038250223.666668
[2025-10-24 23:40:01] 2025-10-25 03:38:55,568 | INFO | mean x: 1761357076.026455
[2025-10-24 23:40:01] 2025-10-25 03:38:55,577 | INFO | mean y: 2621456.3333333335
[2025-10-24 23:40:01] 2025-10-25 03:38:55,577 | INFO | -- intersect: 9292046078.830322
[2025-10-24 23:40:01] 2025-10-25 03:38:55,577 | INFO | intersect: 9292046078.830322
[2025-10-24 23:40:01] 2025-10-25 03:38:55,577 | INFO | chi2: 0.12313454469230783
[2025-10-24 23:40:01] 2025-10-25 03:38:55,578 | INFO | -- intersect: 9292046078.830322
[2025-10-24 23:40:01] 2025-10-25 03:38:55,578 | INFO | current memory leak: -5.27 B/s (using 189 data points, chi2=0.12)
[2025-10-24 23:40:01] 2025-10-25 03:38:55,578 | INFO | could have reported an average CPU frequency of 2599 MHz (11 samples)
[2025-10-24 23:40:01] 2025-10-25 03:38:55,579 | INFO | ..............................
[2025-10-24 23:40:01] 2025-10-25 03:38:55,579 | INFO | . Timing measurements:
[2025-10-24 23:40:01] 2025-10-25 03:38:55,579 | INFO | . get job = 0 s
[2025-10-24 23:40:01] 2025-10-25 03:38:55,579 | INFO | . initial setup = 4 s
[2025-10-24 23:40:01] 2025-10-25 03:38:55,579 | INFO | . payload setup = 38 s
[2025-10-24 23:40:01] 2025-10-25 03:38:55,579 | INFO | . stage-in = 0 s
[2025-10-24 23:40:01] 2025-10-25 03:38:55,579 | INFO | . payload execution = 13393 s
[2025-10-24 23:40:01] 2025-10-25 03:38:55,579 | INFO | . stage-out = 4 s
[2025-10-24 23:40:01] 2025-10-25 03:38:55,579 | INFO | . log creation = 0 s
[2025-10-24 23:40:01] 2025-10-25 03:38:55,580 | INFO | ..............................
[2025-10-24 23:40:01] 2025-10-25 03:38:55,914 | INFO |
[2025-10-24 23:40:01] 2025-10-25 03:38:55,914 | INFO | job summary report
[2025-10-24 23:40:01] 2025-10-25 03:38:55,914 | INFO | --------------------------------------------------
[2025-10-24 23:40:01] 2025-10-25 03:38:55,914 | INFO | PanDA job id: 6855925172
[2025-10-24 23:40:01] 2025-10-25 03:38:55,915 | INFO | task id: 46771637
[2025-10-24 23:40:01] 2025-10-25 03:38:55,915 | INFO | errors: (none)
[2025-10-24 23:40:01] 2025-10-25 03:38:55,915 | INFO | status: LOG_TRANSFER = DONE
[2025-10-24 23:40:01] 2025-10-25 03:38:55,915 | INFO | pilot state: finished
[2025-10-24 23:40:01] 2025-10-25 03:38:55,915 | INFO | transexitcode: 0
[2025-10-24 23:40:01] 2025-10-25 03:38:55,915 | INFO | exeerrorcode: 0
[2025-10-24 23:40:01] 2025-10-25 03:38:55,915 | INFO | exeerrordiag:
[2025-10-24 23:40:01] 2025-10-25 03:38:55,915 | INFO | exitcode: 0
[2025-10-24 23:40:01] 2025-10-25 03:38:55,915 | INFO | exitmsg: OK
[2025-10-24 23:40:01] 2025-10-25 03:38:55,916 | INFO | cpuconsumptiontime: 54642 s
[2025-10-24 23:40:01] 2025-10-25 03:38:55,916 | INFO | nevents: 400
[2025-10-24 23:40:01] 2025-10-25 03:38:55,916 | INFO | neventsw: 0
[2025-10-24 23:40:01] 2025-10-25 03:38:55,916 | INFO | pid: 3888457
[2025-10-24 23:40:01] 2025-10-25 03:38:55,916 | INFO | pgrp: 3888457
[2025-10-24 23:40:01] 2025-10-25 03:38:55,916 | INFO | corecount: 10
[2025-10-24 23:40:01] 2025-10-25 03:38:55,916 | INFO | event service: False
[2025-10-24 23:40:01] 2025-10-25 03:38:55,916 | INFO | sizes: {0: 2348107, 1: 2348963, 7: 2349225, 12: 2349253, 13436: 2379628, 13442: 2388626, 13445: 2388810, 13479: 2389004}
[2025-10-24 23:40:01] 2025-10-25 03:38:55,916 | INFO | --------------------------------------------------
[2025-10-24 23:40:01] 2025-10-25 03:38:55,916 | INFO |
[2025-10-24 23:40:01] 2025-10-25 03:38:55,917 | INFO | executing command: ls -lF /tmp/boinchome/slots/1
[2025-10-24 23:40:01] 2025-10-25 03:38:55,973 | INFO | queue jobs had 0 job(s) [purged]
[2025-10-24 23:40:01] 2025-10-25 03:38:55,973 | INFO | queue payloads had 0 job(s) [purged]
[2025-10-24 23:40:01] 2025-10-25 03:38:55,973 | INFO | queue data_in had 0 job(s) [purged]
[2025-10-24 23:40:01] 2025-10-25 03:38:55,973 | INFO | queue data_out had 0 job(s) [purged]
[2025-10-24 23:40:01] 2025-10-25 03:38:55,974 | INFO | queue current_data_in had 0 job(s) [purged]
[2025-10-24 23:40:01] 2025-10-25 03:38:55,974 | INFO | queue validated_jobs had 0 job(s) [purged]
[2025-10-24 23:40:01] 2025-10-25 03:38:55,974 | INFO | queue validated_payloads had 0 job(s) [purged]
[2025-10-24 23:40:01] 2025-10-25 03:38:55,974 | INFO | queue monitored_payloads had 0 job(s) [purged]
[2025-10-24 23:40:01] 2025-10-25 03:38:55,974 | INFO | queue finished_jobs had 0 job(s) [purged]
[2025-10-24 23:40:01] 2025-10-25 03:38:55,974 | INFO | queue finished_payloads had 0 job(s) [purged]
[2025-10-24 23:40:01] 2025-10-25 03:38:55,974 | INFO | queue finished_data_in had 1 job(s) [purged]
[2025-10-24 23:40:01] 2025-10-25 03:38:55,974 | INFO | queue finished_data_out had 0 job(s) [purged]
[2025-10-24 23:40:01] 2025-10-25 03:38:55,974 | INFO | queue failed_jobs had 0 job(s) [purged]
[2025-10-24 23:40:01] 2025-10-25 03:38:55,974 | INFO | queue failed_payloads had 0 job(s) [purged]
[2025-10-24 23:40:01] 2025-10-25 03:38:55,975 | INFO | queue failed_data_in had 0 job(s) [purged]
[2025-10-24 23:40:01] 2025-10-25 03:38:55,975 | INFO | queue failed_data_out had 0 job(s) [purged]
[2025-10-24 23:40:01] 2025-10-25 03:38:55,975 | INFO | queue completed_jobs had 0 job(s) [purged]
[2025-10-24 23:40:01] 2025-10-25 03:38:55,975 | INFO | queue completed_jobids has 1 job(s)
[2025-10-24 23:40:01] 2025-10-25 03:38:55,975 | INFO | queue realtimelog_payloads had 0 job(s) [purged]
[2025-10-24 23:40:01] 2025-10-25 03:38:55,975 | INFO | queue messages had 0 job(s) [purged]
[2025-10-24 23:40:01] 2025-10-25 03:38:55,975 | INFO | job 6855925172 has completed (purged errors)
[2025-10-24 23:40:01] 2025-10-25 03:38:55,975 | INFO | overall cleanup function is called
[2025-10-24 23:40:01] 2025-10-25 03:38:56,991 | INFO | --- collectZombieJob: --- 10, [3888457]
[2025-10-24 23:40:01] 2025-10-25 03:38:56,992 | INFO | zombie collector waiting for pid 3888457
[2025-10-24 23:40:01] 2025-10-25 03:38:56,992 | INFO | harmless exception when collecting zombies: [Errno 10] No child processes
[2025-10-24 23:40:01] 2025-10-25 03:38:56,992 | INFO | collected zombie processes
[2025-10-24 23:40:01] 2025-10-25 03:38:56,992 | INFO | will attempt to kill all subprocesses of pid=3888457
[2025-10-24 23:40:01] 2025-10-25 03:38:57,883 | INFO | process IDs to be killed: [3888457] (in reverse order)
[2025-10-24 23:40:01] 2025-10-25 03:38:58,031 | WARNING | found no corresponding commands to process id(s)
[2025-10-24 23:40:01] 2025-10-25 03:38:58,031 | INFO | Do not look for orphan processes in BOINC jobs
[2025-10-24 23:40:01] 2025-10-25 03:38:58,046 | INFO | did not find any defunct processes belonging to 3888457
[2025-10-24 23:40:01] 2025-10-25 03:38:58,081 | INFO | did not find any defunct processes belonging to 3888457
[2025-10-24 23:40:01] 2025-10-25 03:38:58,081 | INFO | ready for new job
[2025-10-24 23:40:01] 2025-10-25 03:38:58,082 | INFO | pilot has finished with previous job - re-establishing logging
[2025-10-24 23:40:01] 2025-10-25 03:38:58,107 | INFO | ***************************************
[2025-10-24 23:40:01] 2025-10-25 03:38:58,107 | INFO | *** PanDA Pilot version 3.11.0.29 ***
[2025-10-24 23:40:01] 2025-10-25 03:38:58,107 | INFO | ***************************************
[2025-10-24 23:40:01] 2025-10-25 03:38:58,107 | INFO |
[2025-10-24 23:40:01] 2025-10-25 03:38:58,122 | INFO | architecture information:
[2025-10-24 23:40:01] 2025-10-25 03:38:58,130 | INFO | executing command: cat /etc/os-release
[2025-10-24 23:40:01] 2025-10-25 03:38:58,190 | INFO | cat /etc/os-release:
[2025-10-24 23:40:01] NAME="CentOS Linux"
[2025-10-24 23:40:01] VERSION="7 (Core)"
[2025-10-24 23:40:01] ID="centos"
[2025-10-24 23:40:01] ID_LIKE="rhel fedora"
[2025-10-24 23:40:01] VERSION_ID="7"
[2025-10-24 23:40:01] PRETTY_NAME="CentOS Linux 7 (Core)"
[2025-10-24 23:40:01] ANSI_COLOR="0;31"
[2025-10-24 23:40:01] CPE_NAME="cpe:/o:centos:centos:7"
[2025-10-24 23:40:01] HOME_URL="https://www.centos.org/"
[2025-10-24 23:40:01] BUG_REPORT_URL="https://bugs.centos.org/"
[2025-10-24 23:40:01]
[2025-10-24 23:40:01] CENTOS_MANTISBT_PROJECT="CentOS-7"
[2025-10-24 23:40:01] CENTOS_MANTISBT_PROJECT_VERSION="7"
[2025-10-24 23:40:01] REDHAT_SUPPORT_PRODUCT="centos"
[2025-10-24 23:40:01] REDHAT_SUPPORT_PRODUCT_VERSION="7"
[2025-10-24 23:40:01]
[2025-10-24 23:40:01] 2025-10-25 03:38:58,190 | INFO | ***************************************
[2025-10-24 23:40:01] 2025-10-25 03:38:58,694 | INFO | executing command: df -mP /tmp/boinchome/slots/1
[2025-10-24 23:40:01] 2025-10-25 03:38:58,822 | INFO | sufficient remaining disk space (52976156672 B)
[2025-10-24 23:40:01] 2025-10-25 03:38:58,822 | WARNING | since timefloor is set to 0, pilot was only allowed to run one job
[2025-10-24 23:40:01] 2025-10-25 03:38:58,822 | INFO | current server update state: UPDATING_FINAL
[2025-10-24 23:40:01] 2025-10-25 03:38:58,822 | INFO | update_server=False
[2025-10-24 23:40:01] 2025-10-25 03:38:58,823 | WARNING | setting graceful_stop since proceed_with_getjob() returned False (pilot will end)
[2025-10-24 23:40:01] 2025-10-25 03:38:58,825 | WARNING | data:copytool_out:received graceful stop - abort after this iteration
[2025-10-24 23:40:01] 2025-10-25 03:38:58,966 | WARNING | job monitor detected an abort_job request (signal=args.signal)
[2025-10-24 23:40:01] 2025-10-25 03:38:58,971 | WARNING | cannot recover job monitoring - aborting pilot
[2025-10-24 23:40:01] 2025-10-25 03:38:58,972 | WARNING | job:job_monitor:received graceful stop - abort after this iteration
[2025-10-24 23:40:01] 2025-10-25 03:38:58,972 | INFO | will abort loop
[2025-10-24 23:40:01] 2025-10-25 03:38:59,235 | INFO | all data control threads have been joined
[2025-10-24 23:40:01] 2025-10-25 03:38:59,558 | WARNING | data:queue_monitoring:received graceful stop - abort after this iteration
[2025-10-24 23:40:01] 2025-10-25 03:38:59,640 | INFO | all payload control threads have been joined
[2025-10-24 23:40:01] 2025-10-25 03:38:59,836 | INFO | [job] retrieve thread has finished
[2025-10-24 23:40:01] 2025-10-25 03:38:59,863 | INFO | all job control threads have been joined
[2025-10-24 23:40:01] 2025-10-25 03:38:59,987 | INFO | [job] job monitor thread has finished
[2025-10-24 23:40:01] 2025-10-25 03:39:00,097 | INFO | [payload] run_realtimelog thread has finished
[2025-10-24 23:40:01] 2025-10-25 03:39:00,106 | INFO | [job] validate thread has finished
[2025-10-24 23:40:01] 2025-10-25 03:39:00,241 | INFO | [data] control thread has finished
[2025-10-24 23:40:01] 2025-10-25 03:39:00,527 | INFO | [payload] failed_post thread has finished
[2025-10-24 23:40:01] 2025-10-25 03:39:00,527 | INFO | [payload] validate_post thread has finished
[2025-10-24 23:40:01] 2025-10-25 03:39:00,646 | INFO | [payload] control thread has finished
[2025-10-24 23:40:01] 2025-10-25 03:39:00,832 | INFO | [data] copytool_out thread has finished
[2025-10-24 23:40:01] 2025-10-25 03:39:00,869 | INFO | [job] control thread has finished
[2025-10-24 23:40:01] 2025-10-25 03:39:00,878 | INFO | [data] copytool_in thread has finished
[2025-10-24 23:40:01] 2025-10-25 03:39:00,930 | WARNING | job:queue_monitor:received graceful stop - abort after this iteration
[2025-10-24 23:40:01] 2025-10-25 03:39:01,241 | INFO | [payload] validate_pre thread has finished
[2025-10-24 23:40:01] 2025-10-25 03:39:01,241 | INFO | [payload] execute_payloads thread has finished
[2025-10-24 23:40:01] 2025-10-25 03:39:01,242 | INFO | [job] create_data_payload thread has finished
[2025-10-24 23:40:01] 2025-10-25 03:39:01,951 | INFO | [job] queue monitor thread has finished
[2025-10-24 23:40:01] 2025-10-25 03:39:03,567 | INFO | [data] queue_monitor thread has finished
[2025-10-24 23:40:01] 2025-10-25 03:39:28,970 | INFO | PID=3871479 has CPU usage=2.6% CMD=/cvmfs/atlas.cern.ch/repo/ATLASLocalRootBase/x86_64/python/3.9.20-x86_64-centos7/bin/python3 pilot3/pilot.py -q BOINC_MCORE -i P
[2025-10-24 23:40:01] 2025-10-25 03:39:28,971 | INFO | .. there are 14 such processes running
[2025-10-24 23:40:01] 2025-10-25 03:39:28,972 | INFO | found 0 job(s) in 20 queues
[2025-10-24 23:40:01] 2025-10-25 03:39:28,972 | WARNING | pilot monitor received instruction that args.graceful_stop has been set
[2025-10-24 23:40:01] 2025-10-25 03:39:28,972 | WARNING | will wait for a maximum of 300 s for threads to finish
[2025-10-24 23:40:01] 2025-10-25 03:39:54,705 | INFO | [monitor] cgroup control has ended
[2025-10-24 23:40:01] 2025-10-25 03:39:55,714 | INFO | only monitor.control thread still running - safe to abort: ['<_MainThread(MainThread, started 139693507573568)>', '<ExcThread(monitor, started 139692645271296)>']
[2025-10-24 23:40:01] 2025-10-25 03:39:56,153 | WARNING | job_aborted has been set - aborting pilot monitoring
[2025-10-24 23:40:01] 2025-10-25 03:39:56,154 | INFO | [monitor] control thread has ended
[2025-10-24 23:40:01] 2025-10-25 03:40:00,741 | INFO | all workflow threads have been joined
[2025-10-24 23:40:01] 2025-10-25 03:40:00,742 | INFO | end of generic workflow (traces error code: 0)
[2025-10-24 23:40:01] 2025-10-25 03:40:00,742 | INFO | traces error code: 0
[2025-10-24 23:40:01] 2025-10-25 03:40:00,742 | INFO | pilot has finished (exit code=0, shell exit code=0)
[2025-10-24 23:40:01] 2025-10-25 03:40:01,001 [wrapper] ==== pilot stdout END ====
[2025-10-24 23:40:01] 2025-10-25 03:40:01,012 [wrapper] ==== wrapper stdout RESUME ====
[2025-10-24 23:40:01] 2025-10-25 03:40:01,028 [wrapper] pilotpid: 3871479
[2025-10-24 23:40:01] 2025-10-25 03:40:01,045 [wrapper] Pilot exit status: 0
[2025-10-24 23:40:01] 2025-10-25 03:40:01,154 [wrapper] pandaids: 6855925172
[2025-10-24 23:40:01] 2025-10-25 03:40:01,457 [wrapper] cleanup supervisor_pilot 201187 3871480
[2025-10-24 23:40:01] 2025-10-25 03:40:01,461 [wrapper] Test setup, not cleaning
[2025-10-24 23:40:01] 2025-10-25 03:40:01,483 [wrapper] apfmon messages muted
[2025-10-24 23:40:01] 2025-10-25 03:40:01,499 [wrapper] ==== wrapper stdout END ====
[2025-10-24 23:40:01] 2025-10-25 03:40:01,520 [wrapper] ==== wrapper stderr END ====
[2025-10-24 23:40:01] *** Error codes and diagnostics ***
[2025-10-24 23:40:01] "exeErrorCode": 0,
[2025-10-24 23:40:01] "exeErrorDiag": "",
[2025-10-24 23:40:01] "pilotErrorCode": 0,
[2025-10-24 23:40:01] "pilotErrorDiag": "",
[2025-10-24 23:40:01] *** Listing of results directory ***
[2025-10-24 23:40:01] total 426808
[2025-10-24 23:40:01] -rw-r--r--. 1 boincer umatlas 557409 Oct 24 14:40 pilot3.tar.gz
[2025-10-24 23:40:01] -rwx------. 1 boincer umatlas 36292 Oct 24 15:28 runpilot2-wrapper.sh
[2025-10-24 23:40:01] -rw-r--r--. 1 boincer umatlas 5112 Oct 24 15:28 queuedata.json
[2025-10-24 23:40:01] -rw-r--r--. 1 boincer umatlas 100 Oct 24 19:53 wrapper_26015_x86_64-pc-linux-gnu
[2025-10-24 23:40:01] -rwxr-xr-x. 1 boincer umatlas 7986 Oct 24 19:53 run_atlas
[2025-10-24 23:40:01] -rw-r--r--. 1 boincer umatlas 105 Oct 24 19:53 job.xml
[2025-10-24 23:40:01] -rw-r--r--. 2 boincer umatlas 227031331 Oct 24 19:53 EVNT.46771634._004956.pool.root.1
[2025-10-24 23:40:01] -rw-r--r--. 2 boincer umatlas 15120 Oct 24 19:53 start_atlas.sh
[2025-10-24 23:40:01] drwxrwx--x. 2 boincer umatlas 4096 Oct 24 19:53 shared
[2025-10-24 23:40:01] -rw-r--r--. 2 boincer umatlas 569687 Oct 24 19:53 input.tar.gz
[2025-10-24 23:40:01] -rw-r--r--. 1 boincer umatlas 0 Oct 24 19:53 boinc_lockfile
[2025-10-24 23:40:01] -rw-r--r--. 1 boincer umatlas 2562 Oct 24 19:53 pandaJob.out
[2025-10-24 23:40:01] -rw-------. 1 boincer umatlas 978597 Oct 24 19:53 agis_schedconf.cvmfs.json
[2025-10-24 23:40:01] drwx------. 5 boincer umatlas 4096 Oct 24 19:53 pilot3
[2025-10-24 23:40:01] -rw-------. 1 boincer umatlas 201453261 Oct 24 23:35 HITS.46771637._123459.pool.root.1
[2025-10-24 23:40:01] -rw-r--r--. 1 boincer umatlas 530 Oct 24 23:35 boinc_task_state.xml
[2025-10-24 23:40:01] -rw-r--r--. 1 boincer umatlas 6659 Oct 24 23:37 init_data.xml
[2025-10-24 23:40:01] -rw-------. 1 boincer umatlas 1046 Oct 24 23:38 memory_monitor_summary.json
[2025-10-24 23:40:01] -rw-------. 1 boincer umatlas 1560772 Oct 24 23:38 agis_ddmendpoints.agis.ALL.json
[2025-10-24 23:40:01] -rw-------. 1 boincer umatlas 490413 Oct 24 23:38 log.46771637._123459.job.log.tgz.1
[2025-10-24 23:40:01] -rw-------. 1 boincer umatlas 7835 Oct 24 23:38 heartbeat.json
[2025-10-24 23:40:01] -rw-------. 1 boincer umatlas 95 Oct 24 23:39 pilot_heartbeat.json
[2025-10-24 23:40:01] -rw-r--r--. 1 boincer umatlas 8192 Oct 24 23:39 boinc_mmap_file
[2025-10-24 23:40:01] -rw-r--r--. 1 boincer umatlas 28 Oct 24 23:39 wrapper_checkpoint.txt
[2025-10-24 23:40:01] -rw-------. 1 boincer umatlas 5010 Oct 24 23:40 pilotlog.txt
[2025-10-24 23:40:01] -rw-------. 1 boincer umatlas 1841776 Oct 24 23:40 log.46771637._123459.job.log.1
[2025-10-24 23:40:01] -rw-------. 1 boincer umatlas 357 Oct 24 23:40 output.list
[2025-10-24 23:40:01] -rw-r--r--. 1 boincer umatlas 620 Oct 24 23:40 runtime_log
[2025-10-24 23:40:01] -rw-------. 1 boincer umatlas 2355200 Oct 24 23:40 result.tar.gz
[2025-10-24 23:40:01] -rw-r--r--. 1 boincer umatlas 8701 Oct 24 23:40 runtime_log.err
[2025-10-24 23:40:01] -rw-------. 1 boincer umatlas 656 Oct 24 23:40 7VXMDm6dcW8n9Rq4apOajLDm4fhM0noT9bVof3QYDmtwlMDmNguVOm.diag
[2025-10-24 23:40:01] -rw-r--r--. 1 boincer umatlas 21022 Oct 24 23:40 stderr.txt
23:47:06 (213742): wrapper (7.7.26015): starting
23:47:06 (213742): wrapper: running run_atlas (--nthreads 10)
[2025-10-24 23:47:06] Arguments: --nthreads 10
[2025-10-24 23:47:06] Threads: 10
[2025-10-24 23:47:06] This job has been restarted, cleaning up previous attempt
[2025-10-24 23:47:06] Checking for CVMFS
[2025-10-24 23:47:06] Probing /cvmfs/atlas.cern.ch... OK
[2025-10-24 23:47:06] Probing /cvmfs/atlas-condb.cern.ch... OK
[2025-10-24 23:47:06] Running cvmfs_config stat atlas.cern.ch
[2025-10-24 23:47:06] VERSION PID UPTIME(M) MEM(K) REVISION EXPIRES(M) NOCATALOGS CACHEUSE(K) CACHEMAX(K) NOFDUSE NOFDMAX NOIOERR NOOPEN HITRATE(%) RX(K) SPEED(K/S) HOST PROXY ONLINE
[2025-10-24 23:47:06] 2.13.2.0 128802 135488 147032 152105 2 342 21086639 39288833 11919 16776704 0 104319387 99.666 89875266 32550 http://cvmfs-s1fnal.opensciencegrid.org:8000/cvmfs/atlas.cern.ch http://192.41.237.109:6081 1
[2025-10-24 23:47:06] CVMFS is ok
[2025-10-24 23:47:06] Efficiency of ATLAS tasks can be improved by the following measure(s):
[2025-10-24 23:47:06] The CVMFS client on this computer should be configured to use Cloudflare's openhtc.io.
[2025-10-24 23:47:06] Further information can be found at the LHC@home message board.
[2025-10-24 23:47:06] Using apptainer image /cvmfs/atlas.cern.ch/repo/containers/fs/singularity/x86_64-centos7
[2025-10-24 23:47:06] Checking for apptainer binary...
[2025-10-24 23:47:06] Using apptainer found in PATH at /usr/bin/apptainer
[2025-10-24 23:47:06] Running /usr/bin/apptainer --version
[2025-10-24 23:47:06] apptainer version 1.4.3-1.el9
[2025-10-24 23:47:06] Checking apptainer works with /usr/bin/apptainer exec -B /cvmfs /cvmfs/atlas.cern.ch/repo/containers/fs/singularity/x86_64-centos7 hostname
[2025-10-24 23:47:07] c-210-7.aglt2.org
[2025-10-24 23:47:07] apptainer works
[2025-10-24 23:47:07] Set ATHENA_PROC_NUMBER=10
[2025-10-24 23:47:07] Set ATHENA_CORE_NUMBER=10
[2025-10-24 23:47:07] Starting ATLAS job with PandaID=6855925172
[2025-10-24 23:47:07] Running command: /usr/bin/apptainer exec -B /cvmfs,/tmp/boinchome/slots/1 /cvmfs/atlas.cern.ch/repo/containers/fs/singularity/x86_64-centos7 sh start_atlas.sh
[2025-10-25 10:43:01] *** The last 200 lines of the pilot log: ***
[2025-10-25 10:43:01] 2025-10-25 14:41:41,948 | WARNING | format EVNTtoHITS has no such key: dbTime
[2025-10-25 10:43:01] 2025-10-25 14:41:41,960 | INFO | fitting pss+swap vs Time
[2025-10-25 10:43:01] 2025-10-25 14:41:41,961 | INFO | sum of square deviations: 82433617174.7776
[2025-10-25 10:43:01] 2025-10-25 14:41:41,972 | INFO | sum of deviations: 35662730573.10421
[2025-10-25 10:43:01] 2025-10-25 14:41:41,972 | INFO | mean x: 1761383649.332815
[2025-10-25 10:43:01] 2025-10-25 14:41:41,972 | INFO | mean y: 2594021.3125972007
[2025-10-25 10:43:01] 2025-10-25 14:41:41,972 | INFO | -- intersect: -759422164.254559
[2025-10-25 10:43:01] 2025-10-25 14:41:41,972 | INFO | intersect: -759422164.254559
[2025-10-25 10:43:01] 2025-10-25 14:41:41,973 | INFO | chi2: 2.115156076323103
[2025-10-25 10:43:01] 2025-10-25 14:41:41,974 | INFO | sum of square deviations: 80525623028.37157
[2025-10-25 10:43:01] 2025-10-25 14:41:41,993 | INFO | sum of deviations: 74754458171.86528
[2025-10-25 10:43:01] 2025-10-25 14:41:41,994 | INFO | mean x: 1761383496.838558
[2025-10-25 10:43:01] 2025-10-25 14:41:41,995 | INFO | mean y: 2597124.9404388713
[2025-10-25 10:43:01] 2025-10-25 14:41:41,995 | INFO | -- intersect: -1632550347.220566
[2025-10-25 10:43:01] 2025-10-25 14:41:41,995 | INFO | intersect: -1632550347.220566
[2025-10-25 10:43:01] 2025-10-25 14:41:41,996 | INFO | chi2: 1.4215749395159107
[2025-10-25 10:43:01] 2025-10-25 14:41:41,996 | INFO | current chi2=1.4215749395159107 (change=32.791014553067136 %)
[2025-10-25 10:43:01] 2025-10-25 14:41:41,997 | INFO | sum of square deviations: 78647225288.29404
[2025-10-25 10:43:01] 2025-10-25 14:41:42,009 | INFO | sum of deviations: 72943242332.87209
[2025-10-25 10:43:01] 2025-10-25 14:41:42,009 | INFO | mean x: 1761383344.3412323
[2025-10-25 10:43:01] 2025-10-25 14:41:42,009 | INFO | mean y: 2596977.9589257506
[2025-10-25 10:43:01] 2025-10-25 14:41:42,009 | INFO | -- intersect: -1631039957.818521
[2025-10-25 10:43:01] 2025-10-25 14:41:42,009 | INFO | intersect: -1631039957.818521
[2025-10-25 10:43:01] 2025-10-25 14:41:42,010 | INFO | chi2: 1.4215600807505302
[2025-10-25 10:43:01] 2025-10-25 14:41:42,011 | INFO | current chi2=1.4215600807505302 (change=0.0010452326477845102 %)
[2025-10-25 10:43:01] 2025-10-25 14:41:42,011 | INFO | right removable region: 632
[2025-10-25 10:43:01] 2025-10-25 14:41:42,012 | INFO | sum of square deviations: 80525217678.32597
[2025-10-25 10:43:01] 2025-10-25 14:41:42,025 | INFO | sum of deviations: -64816520990.87459
[2025-10-25 10:43:01] 2025-10-25 14:41:42,025 | INFO | mean x: 1761383801.8432603
[2025-10-25 10:43:01] 2025-10-25 14:41:42,025 | INFO | mean y: 2602023.9012539187
[2025-10-25 10:43:01] 2025-10-25 14:41:42,025 | INFO | -- intersect: 1420378634.2200496
[2025-10-25 10:43:01] 2025-10-25 14:41:42,025 | INFO | intersect: 1420378634.2200496
[2025-10-25 10:43:01] 2025-10-25 14:41:42,026 | INFO | chi2: 0.7705787533339067
[2025-10-25 10:43:01] 2025-10-25 14:41:42,027 | INFO | current chi2=0.7705787533339067 (change=63.56870483650323 %)
[2025-10-25 10:43:01] 2025-10-25 14:41:42,028 | INFO | sum of square deviations: 78646609177.84196
[2025-10-25 10:43:01] 2025-10-25 14:41:42,040 | INFO | sum of deviations: -52102820981.17221
[2025-10-25 10:43:01] 2025-10-25 14:41:42,040 | INFO | mean x: 1761383954.349131
[2025-10-25 10:43:01] 2025-10-25 14:41:42,040 | INFO | mean y: 2600991.0505529228
[2025-10-25 10:43:01] 2025-10-25 14:41:42,040 | INFO | -- intersect: 1169505372.7135615
[2025-10-25 10:43:01] 2025-10-25 14:41:42,040 | INFO | intersect: 1169505372.7135615
[2025-10-25 10:43:01] 2025-10-25 14:41:42,041 | INFO | chi2: 0.7594190751336132
[2025-10-25 10:43:01] 2025-10-25 14:41:42,042 | INFO | current chi2=0.7594190751336132 (change=1.4482203346525189 %)
[2025-10-25 10:43:01] 2025-10-25 14:41:42,042 | INFO | left removable region: 20
[2025-10-25 10:43:01] 2025-10-25 14:41:42,043 | INFO | sum of square deviations: 71075956133.74341
[2025-10-25 10:43:01] 2025-10-25 14:41:42,054 | INFO | sum of deviations: 14782838506.758173
[2025-10-25 10:43:01] 2025-10-25 14:41:42,055 | INFO | mean x: 1761383923.8643792
[2025-10-25 10:43:01] 2025-10-25 14:41:42,055 | INFO | mean y: 2601598.732026144
[2025-10-25 10:43:01] 2025-10-25 14:41:42,055 | INFO | -- intersect: -363742457.8416349
[2025-10-25 10:43:01] 2025-10-25 14:41:42,055 | INFO | intersect: -363742457.8416349
[2025-10-25 10:43:01] 2025-10-25 14:41:42,056 | INFO | chi2: 0.02399318103209396
[2025-10-25 10:43:01] 2025-10-25 14:41:42,056 | INFO | -- intersect: -363742457.8416349
[2025-10-25 10:43:01] 2025-10-25 14:41:42,057 | INFO | current memory leak: 0.21 B/s (using 612 data points, chi2=0.02)
[2025-10-25 10:43:01] 2025-10-25 14:41:42,058 | INFO | could have reported an average CPU frequency of 2599 MHz (25 samples)
[2025-10-25 10:43:01] 2025-10-25 14:41:42,058 | INFO | ..............................
[2025-10-25 10:43:01] 2025-10-25 14:41:42,058 | INFO | . Timing measurements:
[2025-10-25 10:43:01] 2025-10-25 14:41:42,058 | INFO | . get job = 0 s
[2025-10-25 10:43:01] 2025-10-25 14:41:42,058 | INFO | . initial setup = 1 s
[2025-10-25 10:43:01] 2025-10-25 14:41:42,058 | INFO | . payload setup = 6 s
[2025-10-25 10:43:01] 2025-10-25 14:41:42,058 | INFO | . stage-in = 0 s
[2025-10-25 10:43:01] 2025-10-25 14:41:42,059 | INFO | . payload execution = 39189 s
[2025-10-25 10:43:01] 2025-10-25 14:41:42,059 | INFO | . stage-out = 6 s
[2025-10-25 10:43:01] 2025-10-25 14:41:42,059 | INFO | . log creation = 1 s
[2025-10-25 10:43:01] 2025-10-25 14:41:42,059 | INFO | ..............................
[2025-10-25 10:43:01] 2025-10-25 14:41:42,180 | INFO |
[2025-10-25 10:43:01] 2025-10-25 14:41:42,180 | INFO | job summary report
[2025-10-25 10:43:01] 2025-10-25 14:41:42,180 | INFO | --------------------------------------------------
[2025-10-25 10:43:01] 2025-10-25 14:41:42,180 | INFO | PanDA job id: 6855925172
[2025-10-25 10:43:01] 2025-10-25 14:41:42,180 | INFO | task id: 46771637
[2025-10-25 10:43:01] 2025-10-25 14:41:42,180 | INFO | errors: (none)
[2025-10-25 10:43:01] 2025-10-25 14:41:42,180 | INFO | status: LOG_TRANSFER = DONE
[2025-10-25 10:43:01] 2025-10-25 14:41:42,181 | INFO | pilot state: finished
[2025-10-25 10:43:01] 2025-10-25 14:41:42,181 | INFO | transexitcode: 0
[2025-10-25 10:43:01] 2025-10-25 14:41:42,181 | INFO | exeerrorcode: 0
[2025-10-25 10:43:01] 2025-10-25 14:41:42,181 | INFO | exeerrordiag:
[2025-10-25 10:43:01] 2025-10-25 14:41:42,181 | INFO | exitcode: 0
[2025-10-25 10:43:01] 2025-10-25 14:41:42,181 | INFO | exitmsg: OK
[2025-10-25 10:43:01] 2025-10-25 14:41:42,181 | INFO | cpuconsumptiontime: 54729 s
[2025-10-25 10:43:01] 2025-10-25 14:41:42,181 | INFO | nevents: 400
[2025-10-25 10:43:01] 2025-10-25 14:41:42,181 | INFO | neventsw: 0
[2025-10-25 10:43:01] 2025-10-25 14:41:42,181 | INFO | pid: 235037
[2025-10-25 10:43:01] 2025-10-25 14:41:42,181 | INFO | pgrp: 235037
[2025-10-25 10:43:01] 2025-10-25 14:41:42,182 | INFO | corecount: 10
[2025-10-25 10:43:01] 2025-10-25 14:41:42,182 | INFO | event service: False
[2025-10-25 10:43:01] 2025-10-25 14:41:42,182 | INFO | sizes: {0: 2348764, 1: 2348963, 5: 2349225, 11: 2349281, 39200: 2391351, 39207: 2400349, 39210: 2400533, 39249: 2400791}
[2025-10-25 10:43:01] 2025-10-25 14:41:42,182 | INFO | --------------------------------------------------
[2025-10-25 10:43:01] 2025-10-25 14:41:42,182 | INFO |
[2025-10-25 10:43:01] 2025-10-25 14:41:42,182 | INFO | executing command: ls -lF /tmp/boinchome/slots/1
[2025-10-25 10:43:01] 2025-10-25 14:41:42,473 | INFO | queue jobs had 0 job(s) [purged]
[2025-10-25 10:43:01] 2025-10-25 14:41:42,473 | INFO | queue payloads had 0 job(s) [purged]
[2025-10-25 10:43:01] 2025-10-25 14:41:42,474 | INFO | queue data_in had 0 job(s) [purged]
[2025-10-25 10:43:01] 2025-10-25 14:41:42,474 | INFO | queue data_out had 0 job(s) [purged]
[2025-10-25 10:43:01] 2025-10-25 14:41:42,474 | INFO | queue current_data_in had 0 job(s) [purged]
[2025-10-25 10:43:01] 2025-10-25 14:41:42,474 | INFO | queue validated_jobs had 0 job(s) [purged]
[2025-10-25 10:43:01] 2025-10-25 14:41:42,474 | INFO | queue validated_payloads had 0 job(s) [purged]
[2025-10-25 10:43:01] 2025-10-25 14:41:42,474 | INFO | queue monitored_payloads had 0 job(s) [purged]
[2025-10-25 10:43:01] 2025-10-25 14:41:42,474 | INFO | queue finished_jobs had 0 job(s) [purged]
[2025-10-25 10:43:01] 2025-10-25 14:41:42,474 | INFO | queue finished_payloads had 0 job(s) [purged]
[2025-10-25 10:43:01] 2025-10-25 14:41:42,474 | INFO | queue finished_data_in had 1 job(s) [purged]
[2025-10-25 10:43:01] 2025-10-25 14:41:42,475 | INFO | queue finished_data_out had 0 job(s) [purged]
[2025-10-25 10:43:01] 2025-10-25 14:41:42,475 | INFO | queue failed_jobs had 0 job(s) [purged]
[2025-10-25 10:43:01] 2025-10-25 14:41:42,475 | INFO | queue failed_payloads had 0 job(s) [purged]
[2025-10-25 10:43:01] 2025-10-25 14:41:42,475 | INFO | queue failed_data_in had 0 job(s) [purged]
[2025-10-25 10:43:01] 2025-10-25 14:41:42,475 | INFO | queue failed_data_out had 0 job(s) [purged]
[2025-10-25 10:43:01] 2025-10-25 14:41:42,475 | INFO | queue completed_jobs had 0 job(s) [purged]
[2025-10-25 10:43:01] 2025-10-25 14:41:42,475 | INFO | queue completed_jobids has 1 job(s)
[2025-10-25 10:43:01] 2025-10-25 14:41:42,475 | INFO | queue realtimelog_payloads had 0 job(s) [purged]
[2025-10-25 10:43:01] 2025-10-25 14:41:42,475 | INFO | queue messages had 0 job(s) [purged]
[2025-10-25 10:43:01] 2025-10-25 14:41:42,476 | INFO | job 6855925172 has completed (purged errors)
[2025-10-25 10:43:01] 2025-10-25 14:41:42,476 | INFO | overall cleanup function is called
[2025-10-25 10:43:01] 2025-10-25 14:41:43,487 | INFO | --- collectZombieJob: --- 10, [235037]
[2025-10-25 10:43:01] 2025-10-25 14:41:43,488 | INFO | zombie collector waiting for pid 235037
[2025-10-25 10:43:01] 2025-10-25 14:41:43,488 | INFO | harmless exception when collecting zombies: [Errno 10] No child processes
[2025-10-25 10:43:01] 2025-10-25 14:41:43,488 | INFO | collected zombie processes
[2025-10-25 10:43:01] 2025-10-25 14:41:43,488 | INFO | will attempt to kill all subprocesses of pid=235037
[2025-10-25 10:43:01] 2025-10-25 14:41:44,061 | INFO | process IDs to be killed: [235037] (in reverse order)
[2025-10-25 10:43:01] 2025-10-25 14:41:44,274 | WARNING | found no corresponding commands to process id(s)
[2025-10-25 10:43:01] 2025-10-25 14:41:44,274 | INFO | Do not look for orphan processes in BOINC jobs
[2025-10-25 10:43:01] 2025-10-25 14:41:44,285 | INFO | did not find any defunct processes belonging to 235037
[2025-10-25 10:43:01] 2025-10-25 14:41:44,297 | INFO | did not find any defunct processes belonging to 235037
[2025-10-25 10:43:01] 2025-10-25 14:41:44,298 | INFO | ready for new job
[2025-10-25 10:43:01] 2025-10-25 14:41:44,298 | INFO | pilot has finished with previous job - re-establishing logging
[2025-10-25 10:43:01] 2025-10-25 14:41:44,306 | INFO | ***************************************
[2025-10-25 10:43:01] 2025-10-25 14:41:44,306 | INFO | *** PanDA Pilot version 3.11.0.29 ***
[2025-10-25 10:43:01] 2025-10-25 14:41:44,306 | INFO | ***************************************
[2025-10-25 10:43:01] 2025-10-25 14:41:44,306 | INFO |
[2025-10-25 10:43:01] 2025-10-25 14:41:44,308 | INFO | architecture information:
[2025-10-25 10:43:01] 2025-10-25 14:41:44,330 | INFO | executing command: cat /etc/os-release
[2025-10-25 10:43:01] 2025-10-25 14:41:44,430 | INFO | cat /etc/os-release:
[2025-10-25 10:43:01] NAME="CentOS Linux"
[2025-10-25 10:43:01] VERSION="7 (Core)"
[2025-10-25 10:43:01] ID="centos"
[2025-10-25 10:43:01] ID_LIKE="rhel fedora"
[2025-10-25 10:43:01] VERSION_ID="7"
[2025-10-25 10:43:01] PRETTY_NAME="CentOS Linux 7 (Core)"
[2025-10-25 10:43:01] ANSI_COLOR="0;31"
[2025-10-25 10:43:01] CPE_NAME="cpe:/o:centos:centos:7"
[2025-10-25 10:43:01] HOME_URL="https://www.centos.org/"
[2025-10-25 10:43:01] BUG_REPORT_URL="https://bugs.centos.org/"
[2025-10-25 10:43:01]
[2025-10-25 10:43:01] CENTOS_MANTISBT_PROJECT="CentOS-7"
[2025-10-25 10:43:01] CENTOS_MANTISBT_PROJECT_VERSION="7"
[2025-10-25 10:43:01] REDHAT_SUPPORT_PRODUCT="centos"
[2025-10-25 10:43:01] REDHAT_SUPPORT_PRODUCT_VERSION="7"
[2025-10-25 10:43:01]
[2025-10-25 10:43:01] 2025-10-25 14:41:44,430 | INFO | ***************************************
[2025-10-25 10:43:01] 2025-10-25 14:41:44,937 | INFO | executing command: df -mP /tmp/boinchome/slots/1
[2025-10-25 10:43:01] 2025-10-25 14:41:45,082 | INFO | sufficient remaining disk space (52218036224 B)
[2025-10-25 10:43:01] 2025-10-25 14:41:45,082 | WARNING | since timefloor is set to 0, pilot was only allowed to run one job
[2025-10-25 10:43:01] 2025-10-25 14:41:45,083 | INFO | current server update state: UPDATING_FINAL
[2025-10-25 10:43:01] 2025-10-25 14:41:45,083 | INFO | update_server=False
[2025-10-25 10:43:01] 2025-10-25 14:41:45,083 | WARNING | setting graceful_stop since proceed_with_getjob() returned False (pilot will end)
[2025-10-25 10:43:01] 2025-10-25 14:41:45,491 | WARNING | job monitor detected an abort_job request (signal=args.signal)
[2025-10-25 10:43:01] 2025-10-25 14:41:45,492 | WARNING | cannot recover job monitoring - aborting pilot
[2025-10-25 10:43:01] 2025-10-25 14:41:45,492 | WARNING | job:job_monitor:received graceful stop - abort after this iteration
[2025-10-25 10:43:01] 2025-10-25 14:41:45,492 | INFO | will abort loop
[2025-10-25 10:43:01] 2025-10-25 14:41:45,510 | INFO | all data control threads have been joined
[2025-10-25 10:43:01] 2025-10-25 14:41:45,726 | WARNING | data:queue_monitoring:received graceful stop - abort after this iteration
[2025-10-25 10:43:01] 2025-10-25 14:41:45,782 | INFO | all payload control threads have been joined
[2025-10-25 10:43:01] 2025-10-25 14:41:46,089 | INFO | [job] retrieve thread has finished
[2025-10-25 10:43:01] 2025-10-25 14:41:46,162 | WARNING | data:copytool_out:received graceful stop - abort after this iteration
[2025-10-25 10:43:01] 2025-10-25 14:41:46,497 | INFO | [job] job monitor thread has finished
[2025-10-25 10:43:01] 2025-10-25 14:41:46,515 | INFO | all job control threads have been joined
[2025-10-25 10:43:01] 2025-10-25 14:41:46,516 | INFO | [data] control thread has finished
[2025-10-25 10:43:01] 2025-10-25 14:41:46,788 | INFO | [payload] control thread has finished
[2025-10-25 10:43:01] 2025-10-25 14:41:46,851 | INFO | [payload] validate_post thread has finished
[2025-10-25 10:43:01] 2025-10-25 14:41:46,914 | INFO | [payload] run_realtimelog thread has finished
[2025-10-25 10:43:01] 2025-10-25 14:41:47,194 | WARNING | job:queue_monitor:received graceful stop - abort after this iteration
[2025-10-25 10:43:01] 2025-10-25 14:41:47,377 | INFO | [job] create_data_payload thread has finished
[2025-10-25 10:43:01] 2025-10-25 14:41:47,484 | INFO | [payload] validate_pre thread has finished
[2025-10-25 10:43:01] 2025-10-25 14:41:47,491 | INFO | [data] copytool_in thread has finished
[2025-10-25 10:43:01] 2025-10-25 14:41:47,512 | INFO | [payload] failed_post thread has finished
[2025-10-25 10:43:01] 2025-10-25 14:41:47,521 | INFO | [job] control thread has finished
[2025-10-25 10:43:01] 2025-10-25 14:41:47,547 | INFO | [job] validate thread has finished
[2025-10-25 10:43:01] 2025-10-25 14:41:47,583 | INFO | [payload] execute_payloads thread has finished
[2025-10-25 10:43:01] 2025-10-25 14:41:48,169 | INFO | [data] copytool_out thread has finished
[2025-10-25 10:43:01] 2025-10-25 14:41:48,200 | INFO | [job] queue monitor thread has finished
[2025-10-25 10:43:01] 2025-10-25 14:41:49,735 | INFO | [data] queue_monitor thread has finished
[2025-10-25 10:43:01] 2025-10-25 14:42:15,497 | INFO | PID=221095 has CPU usage=2.0% CMD=/cvmfs/atlas.cern.ch/repo/ATLASLocalRootBase/x86_64/python/3.9.20-x86_64-centos7/bin/python3 pilot3/pilot.py -q BOINC_MCORE -i PR
[2025-10-25 10:43:01] 2025-10-25 14:42:15,497 | INFO | .. there are 13 such processes running
[2025-10-25 10:43:01] 2025-10-25 14:42:15,498 | INFO | found 0 job(s) in 20 queues
[2025-10-25 10:43:01] 2025-10-25 14:42:15,498 | WARNING | pilot monitor received instruction that args.graceful_stop has been set
[2025-10-25 10:43:01] 2025-10-25 14:42:15,498 | WARNING | will wait for a maximum of 300 s for threads to finish
[2025-10-25 10:43:01] 2025-10-25 14:42:34,474 | INFO | [monitor] cgroup control has ended
[2025-10-25 10:43:01] 2025-10-25 14:42:35,535 | INFO | only monitor.control thread still running - safe to abort: ['<_MainThread(MainThread, started 140281009002304)>', '<ExcThread(monitor, started 140280384702208)>']
[2025-10-25 10:43:01] 2025-10-25 14:42:35,615 | WARNING | job_aborted has been set - aborting pilot monitoring
[2025-10-25 10:43:01] 2025-10-25 14:42:35,616 | INFO | [monitor] control thread has ended
[2025-10-25 10:43:01] 2025-10-25 14:42:40,561 | INFO | all workflow threads have been joined
[2025-10-25 10:43:01] 2025-10-25 14:42:40,561 | INFO | end of generic workflow (traces error code: 0)
[2025-10-25 10:43:01] 2025-10-25 14:42:40,562 | INFO | traces error code: 0
[2025-10-25 10:43:01] 2025-10-25 14:42:40,562 | INFO | pilot has finished (exit code=0, shell exit code=0)
[2025-10-25 10:43:01] 2025-10-25 14:42:40,746 [wrapper] ==== pilot stdout END ====
[2025-10-25 10:43:01] 2025-10-25 14:42:40,750 [wrapper] ==== wrapper stdout RESUME ====
[2025-10-25 10:43:01] 2025-10-25 14:42:40,753 [wrapper] pilotpid: 221095
[2025-10-25 10:43:01] 2025-10-25 14:42:40,757 [wrapper] Pilot exit status: 0
[2025-10-25 10:43:01] 2025-10-25 14:42:40,801 [wrapper] pandaids: 6855925172 6855925172
[2025-10-25 10:43:01] 2025-10-25 14:42:40,925 [wrapper] cleanup supervisor_pilot 1373308 221096
[2025-10-25 10:43:01] 2025-10-25 14:42:40,954 [wrapper] Test setup, not cleaning
[2025-10-25 10:43:01] 2025-10-25 14:42:40,967 [wrapper] apfmon messages muted
[2025-10-25 10:43:01] 2025-10-25 14:42:40,984 [wrapper] ==== wrapper stdout END ====
[2025-10-25 10:43:01] 2025-10-25 14:42:41,017 [wrapper] ==== wrapper stderr END ====
[2025-10-25 10:43:01] *** Error codes and diagnostics ***
[2025-10-25 10:43:01] "exeErrorCode": 0,
[2025-10-25 10:43:01] "exeErrorDiag": "",
[2025-10-25 10:43:01] "pilotErrorCode": 0,
[2025-10-25 10:43:01] "pilotErrorDiag": "",
[2025-10-25 10:43:01] *** Listing of results directory ***
[2025-10-25 10:43:01] total 654596
[2025-10-25 10:43:01] drwx------. 5 boincer umatlas 4096 Oct 7 03:00 pilot3
[2025-10-25 10:43:01] -rw-r--r--. 1 boincer umatlas 557409 Oct 24 14:40 pilot3.tar.gz
[2025-10-25 10:43:01] -rwx------. 1 boincer umatlas 36292 Oct 24 15:28 runpilot2-wrapper.sh
[2025-10-25 10:43:01] -rw-r--r--. 1 boincer umatlas 5112 Oct 24 15:28 queuedata.json
[2025-10-25 10:43:01] -rw-r--r--. 1 boincer umatlas 100 Oct 24 19:53 wrapper_26015_x86_64-pc-linux-gnu
[2025-10-25 10:43:01] -rwxr-xr-x. 1 boincer umatlas 7986 Oct 24 19:53 run_atlas
[2025-10-25 10:43:01] -rw-r--r--. 1 boincer umatlas 105 Oct 24 19:53 job.xml
[2025-10-25 10:43:01] -rw-r--r--. 3 boincer umatlas 227031331 Oct 24 19:53 EVNT.46771634._004956.pool.root.1
[2025-10-25 10:43:01] -rw-r--r--. 3 boincer umatlas 227031331 Oct 24 19:53 ATLAS.root_0
[2025-10-25 10:43:01] -rw-r--r--. 2 boincer umatlas 15120 Oct 24 19:53 start_atlas.sh
[2025-10-25 10:43:01] -rw-r--r--. 2 boincer umatlas 569687 Oct 24 19:53 input.tar.gz
[2025-10-25 10:43:01] -rw-r--r--. 1 boincer umatlas 0 Oct 24 19:53 boinc_lockfile
[2025-10-25 10:43:01] drwxrwx--x. 2 boincer umatlas 4096 Oct 24 23:40 shared
[2025-10-25 10:43:01] -rw-r--r--. 1 boincer umatlas 2562 Oct 24 23:47 pandaJob.out
[2025-10-25 10:43:01] -rw-------. 1 boincer umatlas 978597 Oct 24 23:47 agis_schedconf.cvmfs.json
[2025-10-25 10:43:01] -rw-r--r--. 1 boincer umatlas 6659 Oct 25 10:36 init_data.xml
[2025-10-25 10:43:01] -rw-------. 1 boincer umatlas 201459527 Oct 25 10:39 HITS.46771637._123459.pool.root.1
[2025-10-25 10:43:01] -rw-r--r--. 1 boincer umatlas 533 Oct 25 10:39 boinc_task_state.xml
[2025-10-25 10:43:01] -rw-------. 1 boincer umatlas 1053 Oct 25 10:40 memory_monitor_summary.json
[2025-10-25 10:43:01] -rw-------. 1 boincer umatlas 1560772 Oct 25 10:40 agis_ddmendpoints.agis.ALL.json
[2025-10-25 10:43:01] -rw-------. 1 boincer umatlas 787995 Oct 25 10:40 log.46771637._123459.job.log.tgz.1
[2025-10-25 10:43:01] -rw-------. 1 boincer umatlas 96 Oct 25 10:41 pilot_heartbeat.json
[2025-10-25 10:43:01] -rw-------. 1 boincer umatlas 7846 Oct 25 10:41 heartbeat.json
[2025-10-25 10:43:01] -rw-------. 1 boincer umatlas 5009 Oct 25 10:42 pilotlog.txt
[2025-10-25 10:43:01] -rw-------. 1 boincer umatlas 4639276 Oct 25 10:42 log.46771637._123459.job.log.1
[2025-10-25 10:43:01] -rw-------. 1 boincer umatlas 357 Oct 25 10:42 output.list
[2025-10-25 10:43:01] -rw-r--r--. 1 boincer umatlas 620 Oct 25 10:42 runtime_log
[2025-10-25 10:43:01] -rw-------. 2 boincer umatlas 5447680 Oct 25 10:42 result.tar.gz
[2025-10-25 10:43:01] -rw-r--r--. 1 boincer umatlas 8786 Oct 25 10:42 runtime_log.err
[2025-10-25 10:43:01] -rw-------. 1 boincer umatlas 1284 Oct 25 10:42 7VXMDm6dcW8n9Rq4apOajLDm4fhM0noT9bVof3QYDmtwlMDmNguVOm.diag
[2025-10-25 10:43:01] -rw-r--r--. 1 boincer umatlas 8192 Oct 25 10:43 boinc_mmap_file
[2025-10-25 10:43:01] -rw-r--r--. 1 boincer umatlas 29 Oct 25 10:43 wrapper_checkpoint.txt
[2025-10-25 10:43:01] -rw-r--r--. 1 boincer umatlas 45196 Oct 25 10:43 stderr.txt
[2025-10-25 10:43:01] HITS file was successfully produced:
[2025-10-25 10:43:01] -rw-------. 1 boincer umatlas 201459527 Oct 25 10:39 shared/HITS.pool.root.1
[2025-10-25 10:43:01] *** Contents of shared directory: ***
[2025-10-25 10:43:01] total 424352
[2025-10-25 10:43:01] -rw-r--r--. 3 boincer umatlas 227031331 Oct 24 19:53 ATLAS.root_0
[2025-10-25 10:43:01] -rw-r--r--. 2 boincer umatlas 15120 Oct 24 19:53 start_atlas.sh
[2025-10-25 10:43:01] -rw-r--r--. 2 boincer umatlas 569687 Oct 24 19:53 input.tar.gz
[2025-10-25 10:43:01] -rw-------. 1 boincer umatlas 201459527 Oct 25 10:39 HITS.pool.root.1
[2025-10-25 10:43:01] -rw-------. 2 boincer umatlas 5447680 Oct 25 10:42 result.tar.gz
10:43:03 (213742): run_atlas exited; CPU time 55521.116233
10:43:03 (213742): called boinc_finish(0)
</stderr_txt>
]]>
©2025 CERN