Name fRjNDm2NjK7nsSi4ap6QjLDmwznN0nGgGQJmkKkKDmLFILDmJpT0Uo_1
Workunit 231365561
Created 4 Apr 2025, 8:39:27 UTC
Sent 4 Apr 2025, 11:47:50 UTC
Report deadline 12 Apr 2025, 11:47:50 UTC
Received 4 Apr 2025, 16:31:05 UTC
Server state Over
Outcome Success
Client state Done
Exit status 0 (0x00000000)
Computer ID 10858622
Run time 2 hours 5 min 6 sec
CPU time 10 hours 29 min 34 sec
Validate state Valid
Credit 1,314.40
Device peak FLOPS 44.38 GFLOPS
Application version ATLAS Simulation v3.01 (native_mt)
x86_64-pc-linux-gnu
Peak working set size 2.36 GB
Peak swap size 2.98 GB
Peak disk usage 987.63 MB

Stderr output

<core_client_version>7.20.2</core_client_version>
<![CDATA[
<stderr_txt>
23:24:52 (3309475): wrapper (7.7.26015): starting
23:24:52 (3309475): wrapper: running run_atlas (--nthreads 6)
[2025-04-04 23:24:52] Arguments: --nthreads 6
[2025-04-04 23:24:52] Threads: 6
[2025-04-04 23:24:52] Checking for CVMFS
[2025-04-04 23:24:52] Probing /cvmfs/atlas.cern.ch... OK
[2025-04-04 23:24:52] Probing /cvmfs/atlas-condb.cern.ch... OK
[2025-04-04 23:24:52] Running cvmfs_config stat atlas.cern.ch
[2025-04-04 23:24:53] VERSION PID UPTIME(M) MEM(K) REVISION EXPIRES(M) NOCATALOGS CACHEUSE(K) CACHEMAX(K) NOFDUSE NOFDMAX NOIOERR NOOPEN HITRATE(%) RX(K) SPEED(K/S) HOST PROXY ONLINE
[2025-04-04 23:24:53] 2.11.5.0 2777923 5634 74496 144490 1 187 45410097 52428801 1858 130560 0 15617545 99.997 1407389 69 http://s1cern-cvmfs.openhtc.io/cvmfs/atlas.cern.ch http://128.142.35.143:3126 1
[2025-04-04 23:24:53] CVMFS is ok
[2025-04-04 23:24:53] Using apptainer image /cvmfs/atlas.cern.ch/repo/containers/fs/singularity/x86_64-centos7
[2025-04-04 23:24:53] Checking for apptainer binary...
[2025-04-04 23:24:53] Using apptainer found in PATH at /usr/bin/apptainer
[2025-04-04 23:24:53] Running /usr/bin/apptainer --version
[2025-04-04 23:24:53] apptainer version 1.3.5-1.el9
[2025-04-04 23:24:53] Checking apptainer works with /usr/bin/apptainer exec -B /cvmfs /cvmfs/atlas.cern.ch/repo/containers/fs/singularity/x86_64-centos7 hostname
[2025-04-04 23:24:53] testio008
[2025-04-04 23:24:53] apptainer works
[2025-04-04 23:24:53] Set ATHENA_PROC_NUMBER=6
[2025-04-04 23:24:53] Set ATHENA_CORE_NUMBER=6
[2025-04-04 23:24:53] Starting ATLAS job with PandaID=6586398931
[2025-04-04 23:24:53] Running command: /usr/bin/apptainer exec -B /cvmfs,/var/lib/boinc/slots/1 /cvmfs/atlas.cern.ch/repo/containers/fs/singularity/x86_64-centos7 sh start_atlas.sh
[2025-04-05 01:29:54]  *** The last 200 lines of the pilot log: ***
[2025-04-05 01:29:54] 2025-04-04 16:29:34,244 | INFO     | mean y: 2346880.4827586208
[2025-04-05 01:29:54] 2025-04-04 16:29:34,244 | INFO     | -- intersect: -232556891353.365
[2025-04-05 01:29:54] 2025-04-04 16:29:34,244 | INFO     | intersect: -232556891353.365
[2025-04-05 01:29:54] 2025-04-04 16:29:34,244 | INFO     | chi2: 5.109803232062932
[2025-04-05 01:29:54] 2025-04-04 16:29:34,245 | INFO     | current chi2=5.109803232062932 (change=0.6290793036325871 %)
[2025-04-05 01:29:54] 2025-04-04 16:29:34,245 | INFO     | right removable region: 115
[2025-04-05 01:29:54] 2025-04-04 16:29:34,245 | INFO     | sum of square deviations: 483927685.75
[2025-04-05 01:29:54] 2025-04-04 16:29:34,245 | INFO     | sum of deviations: 38676375071.25002
[2025-04-05 01:29:54] 2025-04-04 16:29:34,245 | INFO     | mean x: 1743780530.75
[2025-04-05 01:29:54] 2025-04-04 16:29:34,245 | INFO     | mean y: 2421018.043103448
[2025-04-05 01:29:54] 2025-04-04 16:29:34,245 | INFO     | -- intersect: -139363669898.42487
[2025-04-05 01:29:54] 2025-04-04 16:29:34,245 | INFO     | intersect: -139363669898.42487
[2025-04-05 01:29:54] 2025-04-04 16:29:34,245 | INFO     | chi2: 2.657187760606272
[2025-04-05 01:29:54] 2025-04-04 16:29:34,245 | INFO     | current chi2=2.657187760606272 (change=48.32536944324512 %)
[2025-04-05 01:29:54] 2025-04-04 16:29:34,245 | INFO     | sum of square deviations: 423992135.42342335
[2025-04-05 01:29:54] 2025-04-04 16:29:34,245 | INFO     | sum of deviations: 16274441045.585585
[2025-04-05 01:29:54] 2025-04-04 16:29:34,245 | INFO     | mean x: 1743780683.2612612
[2025-04-05 01:29:54] 2025-04-04 16:29:34,245 | INFO     | mean y: 2478018.5315315314
[2025-04-05 01:29:54] 2025-04-04 16:29:34,245 | INFO     | -- intersect: -66930499164.70173
[2025-04-05 01:29:54] 2025-04-04 16:29:34,245 | INFO     | intersect: -66930499164.70173
[2025-04-05 01:29:54] 2025-04-04 16:29:34,245 | INFO     | chi2: 1.2439927874532766
[2025-04-05 01:29:54] 2025-04-04 16:29:34,245 | INFO     | current chi2=1.2439927874532766 (change=53.183858291992 %)
[2025-04-05 01:29:54] 2025-04-04 16:29:34,245 | INFO     | sum of square deviations: 369219472.5660365
[2025-04-05 01:29:54] 2025-04-04 16:29:34,246 | INFO     | sum of deviations: -2348971675.9245276
[2025-04-05 01:29:54] 2025-04-04 16:29:34,246 | INFO     | mean x: 1743780835.7735848
[2025-04-05 01:29:54] 2025-04-04 16:29:34,246 | INFO     | mean y: 2529387.669811321
[2025-04-05 01:29:54] 2025-04-04 16:29:34,246 | INFO     | -- intersect: 11096450745.02603
[2025-04-05 01:29:54] 2025-04-04 16:29:34,246 | INFO     | intersect: 11096450745.02603
[2025-04-05 01:29:54] 2025-04-04 16:29:34,246 | INFO     | chi2: 0.030466569062131108
[2025-04-05 01:29:54] 2025-04-04 16:29:34,246 | INFO     | current chi2=0.030466569062131108 (change=97.55090468615154 %)
[2025-04-05 01:29:54] 2025-04-04 16:29:34,246 | INFO     | sum of square deviations: 319377134.67326707
[2025-04-05 01:29:54] 2025-04-04 16:29:34,246 | INFO     | sum of deviations: -3700841481.49505
[2025-04-05 01:29:54] 2025-04-04 16:29:34,246 | INFO     | mean x: 1743780988.2871287
[2025-04-05 01:29:54] 2025-04-04 16:29:34,246 | INFO     | mean y: 2533385.4653465347
[2025-04-05 01:29:54] 2025-04-04 16:29:34,246 | INFO     | -- intersect: 20208917360.628536
[2025-04-05 01:29:54] 2025-04-04 16:29:34,246 | INFO     | intersect: 20208917360.628536
[2025-04-05 01:29:54] 2025-04-04 16:29:34,246 | INFO     | chi2: 0.007442833503053662
[2025-04-05 01:29:54] 2025-04-04 16:29:34,246 | INFO     | current chi2=0.007442833503053662 (change=75.57049010712255 %)
[2025-04-05 01:29:54] 2025-04-04 16:29:34,246 | INFO     | sum of square deviations: 274232559.23958343
[2025-04-05 01:29:54] 2025-04-04 16:29:34,246 | INFO     | sum of deviations: -3158062746.7395835
[2025-04-05 01:29:54] 2025-04-04 16:29:34,246 | INFO     | mean x: 1743781140.8020833
[2025-04-05 01:29:54] 2025-04-04 16:29:34,246 | INFO     | mean y: 2531541.6979166665
[2025-04-05 01:29:54] 2025-04-04 16:29:34,246 | INFO     | -- intersect: 20083918939.69446
[2025-04-05 01:29:54] 2025-04-04 16:29:34,246 | INFO     | intersect: 20083918939.69446
[2025-04-05 01:29:54] 2025-04-04 16:29:34,246 | INFO     | chi2: 0.007397355913420051
[2025-04-05 01:29:54] 2025-04-04 16:29:34,247 | INFO     | current chi2=0.007397355913420051 (change=0.6110252179489539 %)
[2025-04-05 01:29:54] 2025-04-04 16:29:34,247 | INFO     | left removable region: 50
[2025-04-05 01:29:54] 2025-04-04 16:29:34,247 | INFO     | sum of square deviations: 85138676.98461549
[2025-04-05 01:29:54] 2025-04-04 16:29:34,247 | INFO     | sum of deviations: 175905616.23076925
[2025-04-05 01:29:54] 2025-04-04 16:29:34,247 | INFO     | mean x: 1743781720.0153847
[2025-04-05 01:29:54] 2025-04-04 16:29:34,247 | INFO     | mean y: 2516134.769230769
[2025-04-05 01:29:54] 2025-04-04 16:29:34,247 | INFO     | -- intersect: -3600323478.1447086
[2025-04-05 01:29:54] 2025-04-04 16:29:34,247 | INFO     | intersect: -3600323478.1447086
[2025-04-05 01:29:54] 2025-04-04 16:29:34,247 | INFO     | chi2: 0.00021798897909720028
[2025-04-05 01:29:54] 2025-04-04 16:29:34,247 | INFO     | -- intersect: -3600323478.1447086
[2025-04-05 01:29:54] 2025-04-04 16:29:34,247 | INFO     | current memory leak: 2.07 B/s (using 65 data points, chi2=0.00)
[2025-04-05 01:29:54] 2025-04-04 16:29:34,247 | INFO     | ..............................
[2025-04-05 01:29:54] 2025-04-04 16:29:34,247 | INFO     | . Timing measurements:
[2025-04-05 01:29:54] 2025-04-04 16:29:34,247 | INFO     | . get job = 0 s
[2025-04-05 01:29:54] 2025-04-04 16:29:34,247 | INFO     | . initial setup = 4 s
[2025-04-05 01:29:54] 2025-04-04 16:29:34,247 | INFO     | . payload setup = 2 s
[2025-04-05 01:29:54] 2025-04-04 16:29:34,247 | INFO     | . stage-in = 0 s
[2025-04-05 01:29:54] 2025-04-04 16:29:34,247 | INFO     | . payload execution = 7369 s
[2025-04-05 01:29:54] 2025-04-04 16:29:34,247 | INFO     | . stage-out = 0 s
[2025-04-05 01:29:54] 2025-04-04 16:29:34,247 | INFO     | . log creation = 0 s
[2025-04-05 01:29:54] 2025-04-04 16:29:34,247 | INFO     | ..............................
[2025-04-05 01:29:54] 2025-04-04 16:29:34,286 | INFO     | 
[2025-04-05 01:29:54] 2025-04-04 16:29:34,286 | INFO     | job summary report
[2025-04-05 01:29:54] 2025-04-04 16:29:34,286 | INFO     | --------------------------------------------------
[2025-04-05 01:29:54] 2025-04-04 16:29:34,286 | INFO     | PanDA job id: 6586398931
[2025-04-05 01:29:54] 2025-04-04 16:29:34,286 | INFO     | task id: 43634898
[2025-04-05 01:29:54] 2025-04-04 16:29:34,286 | INFO     | errors: (none)
[2025-04-05 01:29:54] 2025-04-04 16:29:34,286 | INFO     | status: LOG_TRANSFER = DONE 
[2025-04-05 01:29:54] 2025-04-04 16:29:34,286 | INFO     | pilot state: finished 
[2025-04-05 01:29:54] 2025-04-04 16:29:34,286 | INFO     | transexitcode: 0
[2025-04-05 01:29:54] 2025-04-04 16:29:34,287 | INFO     | exeerrorcode: 0
[2025-04-05 01:29:54] 2025-04-04 16:29:34,287 | INFO     | exeerrordiag: 
[2025-04-05 01:29:54] 2025-04-04 16:29:34,287 | INFO     | exitcode: 0
[2025-04-05 01:29:54] 2025-04-04 16:29:34,287 | INFO     | exitmsg: OK
[2025-04-05 01:29:54] 2025-04-04 16:29:34,287 | INFO     | cpuconsumptiontime: 37704 s
[2025-04-05 01:29:54] 2025-04-04 16:29:34,287 | INFO     | nevents: 400
[2025-04-05 01:29:54] 2025-04-04 16:29:34,287 | INFO     | neventsw: 0
[2025-04-05 01:29:54] 2025-04-04 16:29:34,287 | INFO     | pid: 3319805
[2025-04-05 01:29:54] 2025-04-04 16:29:34,287 | INFO     | pgrp: 3319805
[2025-04-05 01:29:54] 2025-04-04 16:29:34,287 | INFO     | corecount: 6
[2025-04-05 01:29:54] 2025-04-04 16:29:34,287 | INFO     | event service: False
[2025-04-05 01:29:54] 2025-04-04 16:29:34,287 | INFO     | sizes: {0: 2417228, 1: 2417427, 3: 2417633, 11: 2417661, 7374: 2444880, 7375: 2453821, 7377: 2454005, 7460: 2454175}
[2025-04-05 01:29:54] 2025-04-04 16:29:34,287 | INFO     | --------------------------------------------------
[2025-04-05 01:29:54] 2025-04-04 16:29:34,287 | INFO     | 
[2025-04-05 01:29:54] 2025-04-04 16:29:34,287 | INFO     | executing command: ls -lF /var/lib/boinc/slots/1
[2025-04-05 01:29:54] 2025-04-04 16:29:34,303 | INFO     | queue jobs had 0 job(s) [purged]
[2025-04-05 01:29:54] 2025-04-04 16:29:34,303 | INFO     | queue payloads had 0 job(s) [purged]
[2025-04-05 01:29:54] 2025-04-04 16:29:34,303 | INFO     | queue data_in had 0 job(s) [purged]
[2025-04-05 01:29:54] 2025-04-04 16:29:34,303 | INFO     | queue data_out had 0 job(s) [purged]
[2025-04-05 01:29:54] 2025-04-04 16:29:34,303 | INFO     | queue current_data_in had 0 job(s) [purged]
[2025-04-05 01:29:54] 2025-04-04 16:29:34,303 | INFO     | queue validated_jobs had 0 job(s) [purged]
[2025-04-05 01:29:54] 2025-04-04 16:29:34,303 | INFO     | queue validated_payloads had 0 job(s) [purged]
[2025-04-05 01:29:54] 2025-04-04 16:29:34,303 | INFO     | queue monitored_payloads had 0 job(s) [purged]
[2025-04-05 01:29:54] 2025-04-04 16:29:34,303 | INFO     | queue finished_jobs had 0 job(s) [purged]
[2025-04-05 01:29:54] 2025-04-04 16:29:34,303 | INFO     | queue finished_payloads had 0 job(s) [purged]
[2025-04-05 01:29:54] 2025-04-04 16:29:34,303 | INFO     | queue finished_data_in had 1 job(s) [purged]
[2025-04-05 01:29:54] 2025-04-04 16:29:34,303 | INFO     | queue finished_data_out had 0 job(s) [purged]
[2025-04-05 01:29:54] 2025-04-04 16:29:34,303 | INFO     | queue failed_jobs had 0 job(s) [purged]
[2025-04-05 01:29:54] 2025-04-04 16:29:34,303 | INFO     | queue failed_payloads had 0 job(s) [purged]
[2025-04-05 01:29:54] 2025-04-04 16:29:34,303 | INFO     | queue failed_data_in had 0 job(s) [purged]
[2025-04-05 01:29:54] 2025-04-04 16:29:34,303 | INFO     | queue failed_data_out had 0 job(s) [purged]
[2025-04-05 01:29:54] 2025-04-04 16:29:34,304 | INFO     | queue completed_jobs had 0 job(s) [purged]
[2025-04-05 01:29:54] 2025-04-04 16:29:34,304 | INFO     | queue completed_jobids has 1 job(s)
[2025-04-05 01:29:54] 2025-04-04 16:29:34,304 | INFO     | queue realtimelog_payloads had 0 job(s) [purged]
[2025-04-05 01:29:54] 2025-04-04 16:29:34,304 | INFO     | queue messages had 0 job(s) [purged]
[2025-04-05 01:29:54] 2025-04-04 16:29:34,304 | INFO     | job 6586398931 has completed (purged errors)
[2025-04-05 01:29:54] 2025-04-04 16:29:34,304 | INFO     | overall cleanup function is called
[2025-04-05 01:29:54] 2025-04-04 16:29:35,310 | INFO     | --- collectZombieJob: --- 10, [3319805]
[2025-04-05 01:29:54] 2025-04-04 16:29:35,310 | INFO     | zombie collector waiting for pid 3319805
[2025-04-05 01:29:54] 2025-04-04 16:29:35,310 | INFO     | harmless exception when collecting zombies: [Errno 10] No child processes
[2025-04-05 01:29:54] 2025-04-04 16:29:35,310 | INFO     | collected zombie processes
[2025-04-05 01:29:54] 2025-04-04 16:29:35,310 | INFO     | will attempt to kill all subprocesses of pid=3319805
[2025-04-05 01:29:54] 2025-04-04 16:29:35,340 | INFO     | process IDs to be killed: [3319805] (in reverse order)
[2025-04-05 01:29:54] 2025-04-04 16:29:35,358 | WARNING  | found no corresponding commands to process id(s)
[2025-04-05 01:29:54] 2025-04-04 16:29:35,358 | INFO     | Do not look for orphan processes in BOINC jobs
[2025-04-05 01:29:54] 2025-04-04 16:29:35,360 | INFO     | did not find any defunct processes belonging to 3319805
[2025-04-05 01:29:54] 2025-04-04 16:29:35,361 | INFO     | did not find any defunct processes belonging to 3319805
[2025-04-05 01:29:54] 2025-04-04 16:29:35,361 | INFO     | ready for new job
[2025-04-05 01:29:54] 2025-04-04 16:29:35,361 | INFO     | pilot has finished with previous job - re-establishing logging
[2025-04-05 01:29:54] 2025-04-04 16:29:35,362 | INFO     | **************************************
[2025-04-05 01:29:54] 2025-04-04 16:29:35,362 | INFO     | ***  PanDA Pilot version 3.10.2.2  ***
[2025-04-05 01:29:54] 2025-04-04 16:29:35,362 | INFO     | **************************************
[2025-04-05 01:29:54] 2025-04-04 16:29:35,362 | INFO     | 
[2025-04-05 01:29:54] 2025-04-04 16:29:35,362 | INFO     | architecture information:
[2025-04-05 01:29:54] 2025-04-04 16:29:35,362 | INFO     | executing command: cat /etc/os-release
[2025-04-05 01:29:54] 2025-04-04 16:29:35,368 | INFO     | cat /etc/os-release:
[2025-04-05 01:29:54] NAME="CentOS Linux"
[2025-04-05 01:29:54] VERSION="7 (Core)"
[2025-04-05 01:29:54] ID="centos"
[2025-04-05 01:29:54] ID_LIKE="rhel fedora"
[2025-04-05 01:29:54] VERSION_ID="7"
[2025-04-05 01:29:54] PRETTY_NAME="CentOS Linux 7 (Core)"
[2025-04-05 01:29:54] ANSI_COLOR="0;31"
[2025-04-05 01:29:54] CPE_NAME="cpe:/o:centos:centos:7"
[2025-04-05 01:29:54] HOME_URL="https://www.centos.org/"
[2025-04-05 01:29:54] BUG_REPORT_URL="https://bugs.centos.org/"
[2025-04-05 01:29:54] 
[2025-04-05 01:29:54] CENTOS_MANTISBT_PROJECT="CentOS-7"
[2025-04-05 01:29:54] CENTOS_MANTISBT_PROJECT_VERSION="7"
[2025-04-05 01:29:54] REDHAT_SUPPORT_PRODUCT="centos"
[2025-04-05 01:29:54] REDHAT_SUPPORT_PRODUCT_VERSION="7"
[2025-04-05 01:29:54] 
[2025-04-05 01:29:54] 2025-04-04 16:29:35,368 | INFO     | **************************************
[2025-04-05 01:29:54] 2025-04-04 16:29:35,870 | INFO     | executing command: df -mP /var/lib/boinc/slots/1
[2025-04-05 01:29:54] 2025-04-04 16:29:35,876 | INFO     | sufficient remaining disk space (272637100032 B)
[2025-04-05 01:29:54] 2025-04-04 16:29:35,876 | WARNING  | since timefloor is set to 0, pilot was only allowed to run one job
[2025-04-05 01:29:54] 2025-04-04 16:29:35,876 | INFO     | current server update state: UPDATING_FINAL
[2025-04-05 01:29:54] 2025-04-04 16:29:35,876 | INFO     | update_server=False
[2025-04-05 01:29:54] 2025-04-04 16:29:35,876 | WARNING  | setting graceful_stop since proceed_with_getjob() returned False (pilot will end)
[2025-04-05 01:29:54] 2025-04-04 16:29:35,876 | WARNING  | data:copytool_out:received graceful stop - abort after this iteration
[2025-04-05 01:29:54] 2025-04-04 16:29:35,876 | WARNING  | job:queue_monitor:received graceful stop - abort after this iteration
[2025-04-05 01:29:54] 2025-04-04 16:29:35,876 | WARNING  | aborting monitor loop since graceful_stop has been set (timing out remaining threads)
[2025-04-05 01:29:54] 2025-04-04 16:29:35,876 | INFO     | found 0 job(s) in 20 queues
[2025-04-05 01:29:54] 2025-04-04 16:29:35,877 | WARNING  | pilot monitor received instruction that args.graceful_stop has been set
[2025-04-05 01:29:54] 2025-04-04 16:29:35,877 | WARNING  | will wait for a maximum of 300 s for threads to finish
[2025-04-05 01:29:54] 2025-04-04 16:29:36,183 | INFO     | all data control threads have been joined
[2025-04-05 01:29:54] 2025-04-04 16:29:36,234 | INFO     | all job control threads have been joined
[2025-04-05 01:29:54] 2025-04-04 16:29:36,539 | WARNING  | job:job_monitor:received graceful stop - abort after this iteration
[2025-04-05 01:29:54] 2025-04-04 16:29:36,539 | INFO     | aborting loop
[2025-04-05 01:29:54] 2025-04-04 16:29:36,881 | INFO     | [job] queue monitor thread has finished
[2025-04-05 01:29:54] 2025-04-04 16:29:36,881 | INFO     | [job] retrieve thread has finished
[2025-04-05 01:29:54] 2025-04-04 16:29:36,973 | INFO     | [job] validate thread has finished
[2025-04-05 01:29:54] 2025-04-04 16:29:36,994 | INFO     | [job] create_data_payload thread has finished
[2025-04-05 01:29:54] 2025-04-04 16:29:37,014 | INFO     | [payload] validate_pre thread has finished
[2025-04-05 01:29:54] 2025-04-04 16:29:37,037 | INFO     | all payload control threads have been joined
[2025-04-05 01:29:54] 2025-04-04 16:29:37,188 | INFO     | [data] control thread has finished
[2025-04-05 01:29:54] 2025-04-04 16:29:37,239 | INFO     | [job] control thread has finished
[2025-04-05 01:29:54] 2025-04-04 16:29:37,544 | INFO     | [job] job monitor thread has finished
[2025-04-05 01:29:54] 2025-04-04 16:29:37,704 | INFO     | [payload] validate_post thread has finished
[2025-04-05 01:29:54] 2025-04-04 16:29:37,783 | INFO     | [payload] failed_post thread has finished
[2025-04-05 01:29:54] 2025-04-04 16:29:37,881 | INFO     | [data] copytool_out thread has finished
[2025-04-05 01:29:54] 2025-04-04 16:29:37,958 | WARNING  | data:queue_monitoring:received graceful stop - abort after this iteration
[2025-04-05 01:29:54] 2025-04-04 16:29:38,042 | INFO     | [payload] control thread has finished
[2025-04-05 01:29:54] 2025-04-04 16:29:38,136 | INFO     | [payload] execute_payloads thread has finished
[2025-04-05 01:29:54] 2025-04-04 16:29:38,267 | INFO     | [data] copytool_in thread has finished
[2025-04-05 01:29:54] 2025-04-04 16:29:41,963 | INFO     | [data] queue_monitor thread has finished
[2025-04-05 01:29:54] 2025-04-04 16:29:46,738 | INFO     | job.realtimelogging is not enabled
[2025-04-05 01:29:54] 2025-04-04 16:29:47,743 | INFO     | [payload] run_realtimelog thread has finished
[2025-04-05 01:29:54] 2025-04-04 16:29:49,608 | INFO     | only monitor.control thread still running - safe to abort: ['<_MainThread(MainThread, started 140458868025152)>', '<ExcThread(monitor, started 140458239977216)>']
[2025-04-05 01:29:54] 2025-04-04 16:29:49,934 | WARNING  | job_aborted has been set - aborting pilot monitoring
[2025-04-05 01:29:54] 2025-04-04 16:29:49,934 | INFO     | [monitor] control thread has ended
[2025-04-05 01:29:54] 2025-04-04 16:29:54,617 | INFO     | all workflow threads have been joined
[2025-04-05 01:29:54] 2025-04-04 16:29:54,617 | INFO     | end of generic workflow (traces error code: 0)
[2025-04-05 01:29:54] 2025-04-04 16:29:54,617 | INFO     | traces error code: 0
[2025-04-05 01:29:54] 2025-04-04 16:29:54,617 | INFO     | pilot has finished (exit code=0, shell exit code=0)
[2025-04-05 01:29:54] 2025-04-04 16:29:54,674 [wrapper] ==== pilot stdout END ====
[2025-04-05 01:29:54] 2025-04-04 16:29:54,675 [wrapper] ==== wrapper stdout RESUME ====
[2025-04-05 01:29:54] 2025-04-04 16:29:54,676 [wrapper] pilotpid: 3313195
[2025-04-05 01:29:54] 2025-04-04 16:29:54,677 [wrapper] Pilot exit status: 0
[2025-04-05 01:29:54] 2025-04-04 16:29:54,681 [wrapper] pandaids: 6586398931
[2025-04-05 01:29:54] 2025-04-04 16:29:54,691 [wrapper] cleanup supervisor_pilot 3375196 3313196
[2025-04-05 01:29:54] 2025-04-04 16:29:54,692 [wrapper] Test setup, not cleaning
[2025-04-05 01:29:54] 2025-04-04 16:29:54,693 [wrapper] apfmon messages muted
[2025-04-05 01:29:54] 2025-04-04 16:29:54,694 [wrapper] ==== wrapper stdout END ====
[2025-04-05 01:29:54] 2025-04-04 16:29:54,695 [wrapper] ==== wrapper stderr END ====
[2025-04-05 01:29:54]  *** Error codes and diagnostics ***
[2025-04-05 01:29:54]     "exeErrorCode": 0,
[2025-04-05 01:29:54]     "exeErrorDiag": "",
[2025-04-05 01:29:54]     "pilotErrorCode": 0,
[2025-04-05 01:29:54]     "pilotErrorDiag": "",
[2025-04-05 01:29:54]  *** Listing of results directory ***
[2025-04-05 01:29:54] total 660444
[2025-04-05 01:29:54] -rw-r--r--. 1 boinc boinc    495897 Apr  4 08:30 pilot3.tar.gz
[2025-04-05 01:29:54] -rw-r--r--. 1 boinc boinc      5111 Apr  4 08:41 queuedata.json
[2025-04-05 01:29:54] -rwx------. 1 boinc boinc     37140 Apr  4 08:42 runpilot2-wrapper.sh
[2025-04-05 01:29:54] -rw-r--r--. 1 boinc boinc       100 Apr  4 23:24 wrapper_26015_x86_64-pc-linux-gnu
[2025-04-05 01:29:54] -rwxr-xr-x. 1 boinc boinc      7986 Apr  4 23:24 run_atlas
[2025-04-05 01:29:54] -rw-r--r--. 1 boinc boinc       105 Apr  4 23:24 job.xml
[2025-04-05 01:29:54] -rw-r--r--. 2 boinc boinc    508890 Apr  4 23:24 input.tar.gz
[2025-04-05 01:29:54] -rw-r--r--. 1 boinc boinc      9509 Apr  4 23:24 init_data.xml
[2025-04-05 01:29:54] -rw-r--r--. 2 boinc boinc 358588249 Apr  4 23:24 EVNT.43634896._000698.pool.root.1
[2025-04-05 01:29:54] -rw-r--r--. 2 boinc boinc     17569 Apr  4 23:24 start_atlas.sh
[2025-04-05 01:29:54] drwxrwx--x. 2 boinc boinc        68 Apr  4 23:24 shared
[2025-04-05 01:29:54] -rw-r--r--. 1 boinc boinc         0 Apr  4 23:24 boinc_lockfile
[2025-04-05 01:29:54] -rw-r--r--. 1 boinc boinc      2599 Apr  4 23:24 pandaJob.out
[2025-04-05 01:29:54] -rw-------. 1 boinc boinc       424 Apr  4 23:24 setup.sh.local
[2025-04-05 01:29:54] -rw-------. 1 boinc boinc    979639 Apr  4 23:24 agis_schedconf.cvmfs.json
[2025-04-05 01:29:54] -rw-------. 1 boinc boinc   1603525 Apr  4 23:25 agis_ddmendpoints.agis.ALL.json
[2025-04-05 01:29:54] drwx------. 4 boinc boinc      4096 Apr  4 23:25 pilot3
[2025-04-05 01:29:54] -rw-------. 1 boinc boinc 311840930 Apr  5 01:27 HITS.43634898._018839.pool.root.1
[2025-04-05 01:29:54] -rw-r--r--. 1 boinc boinc       530 Apr  5 01:28 boinc_task_state.xml
[2025-04-05 01:29:54] -rw-------. 1 boinc boinc      1019 Apr  5 01:28 memory_monitor_summary.json
[2025-04-05 01:29:54] -rw-------. 1 boinc boinc    311401 Apr  5 01:28 log.43634898._018839.job.log.tgz.1
[2025-04-05 01:29:54] -rw-------. 1 boinc boinc        95 Apr  5 01:29 pilot_heartbeat.json
[2025-04-05 01:29:54] -rw-------. 1 boinc boinc      7663 Apr  5 01:29 heartbeat.json
[2025-04-05 01:29:54] -rw-r--r--. 1 boinc boinc      8192 Apr  5 01:29 boinc_mmap_file
[2025-04-05 01:29:54] -rw-r--r--. 1 boinc boinc        27 Apr  5 01:29 wrapper_checkpoint.txt
[2025-04-05 01:29:54] -rw-------. 1 boinc boinc      4549 Apr  5 01:29 pilotlog.txt
[2025-04-05 01:29:54] -rw-------. 1 boinc boinc    702617 Apr  5 01:29 log.43634898._018839.job.log.1
[2025-04-05 01:29:54] -rw-------. 1 boinc boinc       357 Apr  5 01:29 output.list
[2025-04-05 01:29:54] -rw-r--r--. 1 boinc boinc       620 Apr  5 01:29 runtime_log
[2025-04-05 01:29:54] -rw-------. 1 boinc boinc   1034240 Apr  5 01:29 result.tar.gz
[2025-04-05 01:29:54] -rw-r--r--. 1 boinc boinc     10971 Apr  5 01:29 runtime_log.err
[2025-04-05 01:29:54] -rw-------. 1 boinc boinc       644 Apr  5 01:29 fRjNDm2NjK7nsSi4ap6QjLDmwznN0nGgGQJmkKkKDmLFILDmJpT0Uo.diag
[2025-04-05 01:29:54] -rw-r--r--. 1 boinc boinc     20561 Apr  5 01:29 stderr.txt
[2025-04-05 01:29:54] HITS file was successfully produced:
[2025-04-05 01:29:54] -rw-------. 1 boinc boinc 311840930 Apr  5 01:27 shared/HITS.pool.root.1
[2025-04-05 01:29:54]  *** Contents of shared directory: ***
[2025-04-05 01:29:54] total 656256
[2025-04-05 01:29:54] -rw-r--r--. 2 boinc boinc    508890 Apr  4 23:24 input.tar.gz
[2025-04-05 01:29:54] -rw-r--r--. 2 boinc boinc 358588249 Apr  4 23:24 ATLAS.root_0
[2025-04-05 01:29:54] -rw-r--r--. 2 boinc boinc     17569 Apr  4 23:24 start_atlas.sh
[2025-04-05 01:29:54] -rw-------. 1 boinc boinc 311840930 Apr  5 01:27 HITS.pool.root.1
[2025-04-05 01:29:54] -rw-------. 1 boinc boinc   1034240 Apr  5 01:29 result.tar.gz
01:29:56 (3309475): run_atlas exited; CPU time 37753.393456
01:29:56 (3309475): called boinc_finish(0)

</stderr_txt>
]]>


©2025 CERN