| Name | Sd2NDm0rEwvnsSi4apGgGQJmABFKDmABFKDmTspPDmABFKDmS8xVBn_0 |
| Workunit | 127545920 |
| Created | 3 Dec 2019, 4:21:39 UTC |
| Sent | 3 Dec 2019, 5:37:32 UTC |
| Report deadline | 11 Dec 2019, 5:37:32 UTC |
| Received | 3 Dec 2019, 5:46:42 UTC |
| Server state | Over |
| Outcome | Success |
| Client state | Done |
| Exit status | 0 (0x00000000) |
| Computer ID | 10616344 |
| Run time | 8 min 1 sec |
| CPU time | 35 sec |
| Validate state | Initial |
| Credit | 0.00 |
| Device peak FLOPS | 12.10 GFLOPS |
| Application version | ATLAS Simulation v2.73 (native_mt) x86_64-pc-linux-gnu |
| Peak working set size | 148.70 MB |
| Peak swap size | 5.27 GB |
| Peak disk usage | 441.23 MB |
Stderr output
<core_client_version>7.7.0</core_client_version>
<![CDATA[
<stderr_txt>
00:38:15 (1309305): wrapper (7.7.26015): starting
00:38:15 (1309305): wrapper: running run_atlas (--nthreads 4)
2019-12-03 00:38:15,617: singularity image is /cvmfs/atlas.cern.ch/repo/containers/images/singularity/x86_64-centos7.img
2019-12-03 00:38:15,617: sys.argv = ['run_atlas', '--nthreads', '4']
2019-12-03 00:38:15,618: THREADS=4
2019-12-03 00:38:15,620: Checking for CVMFS
2019-12-03 00:38:20,823: CVMFS is installed
2019-12-03 00:38:20,823: Checking for singularity binary...
2019-12-03 00:38:20,823: Found singularity at /usr/bin/singularity
2019-12-03 00:38:20,823: Running singularity --version
2019-12-03 00:38:20,993: Singularity version is singularity version 3.4.2-1.1.osg34.el7
2019-12-03 00:38:20,993: Testing the function of Singularity...
2019-12-03 00:38:20,993: Checking singularity with cmd:singularity exec -B /cvmfs /cvmfs/atlas.cern.ch/repo/containers/images/singularity/x86_64-centos7.img hostname
2019-12-03 00:38:22,251: Singularity Works...
2019-12-03 00:38:22,251: copy /tmp/boinchome/slots/8/shared/ATLAS.root_0
2019-12-03 00:38:24,041: copy /tmp/boinchome/slots/8/shared/input.tar.gz
2019-12-03 00:38:24,042: copy /tmp/boinchome/slots/8/shared/start_atlas.sh
2019-12-03 00:38:24,042: copy /tmp/boinchome/slots/8/shared/RTE.tar.gz
2019-12-03 00:38:24,042: export ATHENA_PROC_NUMBER=4;
2019-12-03 00:38:24,108: start atlas job with PandaID=4566367059
2019-12-03 00:38:24,109: cmd = singularity exec --pwd /tmp/boinchome/slots/8 -B /cvmfs,/tmp /cvmfs/atlas.cern.ch/repo/containers/images/singularity/x86_64-centos7.img sh start_atlas.sh > runtime_log 2> runtime_log.err
2019-12-03 00:46:13,689: running cmd return value is 0
2019-12-03 00:46:13,691: **********************Error codes and diagnostics******************
2019-12-03 00:46:13,691: pilotErrorCode : 0
2019-12-03 00:46:13,717: exeErrorCode : 65
2019-12-03 00:46:13,717: pilotErrorDiag :
2019-12-03 00:46:13,717: exeErrorDiag : EVNTtoHITS got a SIGABRT signal (exit code 134)
2019-12-03 00:46:13,717: *****************The last 200 lines of the pilot log******************
2019-12-03 00:46:13,729: "cpuPWEfficiency": 0.0938,
"cpuTime": 3,
"cpuTimeTotal": 0,
"externalCpuTime": 1,
"trfPredata": null,
"wallTime": 32
}
}
}
2019-12-03 05:45:24,929 | DEBUG | queue_monitor | pilot.util.auxiliary.4566367059 | update_server | xml:will send fileinfo
2019-12-03 05:45:24,929 | INFO | queue_monitor | pilot.control.job.4566367059 | send_state | pilot will not update the server (heartbeat message will be written to file)
2019-12-03 05:45:24,930 | INFO | queue_monitor | pilot.control.job.4566367059 | send_state | job 4566367059 has failed - writing final server update
2019-12-03 05:45:24,930 | INFO | queue_monitor | pilot.control.job.4566367059 | verify_error_code | verified error code
2019-12-03 05:45:24,930 | DEBUG | queue_monitor | pilot.control.job.4566367059 | get_data_structure | building data structure to be sent to server with heartbeat
2019-12-03 05:45:24,930 | WARNING | queue_monitor | pilot.user.atlas.common | get_db_info | format EVNTtoHITS has no such key: dbData
2019-12-03 05:45:24,930 | WARNING | queue_monitor | pilot.user.atlas.common | get_db_info | format EVNTtoHITS has no such key: dbTime
2019-12-03 05:45:24,930 | WARNING | queue_monitor | pilot.info.jobdata | get_max_workdir_size | found no stored workdir sizes
2019-12-03 05:45:24,930 | INFO | queue_monitor | pilot.util.auxiliary.4566367059 | get_job_metrics | will not add max space = 0 B to job metrics
2019-12-03 05:45:24,944 | WARNING | queue_monitor | pilot.api.analytics | get_fitted_data | wrong length of table data, x=[1575351783.0], y=[163927.0] (must be same and length>=4)
2019-12-03 05:45:24,944 | DEBUG | queue_monitor | pilot.util.auxiliary.4566367059 | get_job_metrics | job metrics="coreCount=4 actualCoreCount=3"
2019-12-03 05:45:24,945 | INFO | queue_monitor | pilot.control.job.4566367059 | get_data_structure | payload/TRF did not report the number of read events
2019-12-03 05:45:24,955 | INFO | queue_monitor | pilot.user.atlas.utilities | get_memory_values | using path: /tmp/boinchome/slots/8/PanDA_Pilot-4566367059/memory_monitor_summary.json (trf name=prmon)
2019-12-03 05:45:24,979 | DEBUG | queue_monitor | pilot.user.atlas.utilities | get_memory_monitor_info | summary_dictionary={'Max': {'rx_packets': 295, 'nprocs': 5, 'nthreads': 0, 'rx_bytes': 22446, 'wtime': 98, 'rss': 191004, 'write_bytes': 245760, 'vmem': 1153736, 'read_bytes': 200506368, 'stime': 6, 'tx_bytes': 9201, 'pss': 163927, 'wchar': 2651264, 'rchar': 38673654, 'tx_packets': 51, 'swap': 0, 'utime': 7}, 'Avg': {'write_bytes': 2493, 'nprocs': 5, 'nthreads': 0, 'rx_bytes': 227, 'rx_packets': 2, 'vmem': 1153736, 'read_bytes': 2034358, 'swap': 0, 'tx_bytes': 93, 'pss': 163927, 'wchar': 26900, 'rchar': 392386, 'tx_packets': 0, 'rss': 191004}}
2019-12-03 05:45:24,984 | INFO | queue_monitor | pilot.user.atlas.utilities | get_memory_monitor_info | extracted standard info from prmon json
2019-12-03 05:45:24,984 | INFO | queue_monitor | pilot.user.atlas.utilities | get_memory_monitor_info | extracted standard memory fields from prmon json
2019-12-03 05:45:24,984 | INFO | queue_monitor | pilot.util.auxiliary.4566367059 | timing_report | ..............................
2019-12-03 05:45:24,984 | INFO | queue_monitor | pilot.util.auxiliary.4566367059 | timing_report | . Timing measurements:
2019-12-03 05:45:24,984 | INFO | queue_monitor | pilot.util.auxiliary.4566367059 | timing_report | . get job = 2 s
2019-12-03 05:45:24,984 | INFO | queue_monitor | pilot.util.auxiliary.4566367059 | timing_report | . initial setup = 5 s
2019-12-03 05:45:24,989 | INFO | queue_monitor | pilot.util.auxiliary.4566367059 | timing_report | . payload setup = 0 s
2019-12-03 05:45:24,989 | INFO | queue_monitor | pilot.util.auxiliary.4566367059 | timing_report | . total setup = 5 s
2019-12-03 05:45:24,989 | INFO | queue_monitor | pilot.util.auxiliary.4566367059 | timing_report | . stage-in = 1 s
2019-12-03 05:45:24,989 | INFO | queue_monitor | pilot.util.auxiliary.4566367059 | timing_report | . payload execution = 197 s
2019-12-03 05:45:24,989 | INFO | queue_monitor | pilot.util.auxiliary.4566367059 | timing_report | . stage-out = 5 s
2019-12-03 05:45:24,989 | INFO | queue_monitor | pilot.util.auxiliary.4566367059 | timing_report | ..............................
2019-12-03 05:45:24,989 | INFO | queue_monitor | pilot.util.auxiliary.4566367059 | get_log_extracts | building log extracts (sent to the server as 'pilotLog')
2019-12-03 05:45:24,994 | DEBUG | queue_monitor | pilot.util.auxiliary.4566367059 | get_panda_tracer_log | PanDA tracer log does not exist: /tmp/boinchome/slots/8/PanDA_Pilot-4566367059/pandatracerlog.txt (ignoring)
2019-12-03 05:45:24,994 | INFO | queue_monitor | pilot.util.container | execute | executing command: tail -n 20 /tmp/boinchome/slots/8/PanDA_Pilot-4566367059/pilotlog.txt
2019-12-03 05:45:25,077 | WARNING | queue_monitor | pilot.util.auxiliary.4566367059 | get_log_extracts | detected the following tail of warning/fatal messages in the pilot log:
- Log from pilotlog.txt -
2019-12-03 05:45:24,944 | WARNING | queue_monitor | pilot.api.analytics | get_fitted_data | wrong length of table data, x=[1575351783.0], y=[163927.0] (must be same and length>=4)
2019-12-03 05:45:24,944 | DEBUG | queue_monitor | pilot.util.auxiliary.4566367059 | get_job_metrics | job metrics="coreCount=4 actualCoreCount=3"
2019-12-03 05:45:24,945 | INFO | queue_monitor | pilot.control.job.4566367059 | get_data_structure | payload/TRF did not report the number of read events
2019-12-03 05:45:24,955 | INFO | queue_monitor | pilot.user.atlas.utilities | get_memory_values | using path: /tmp/boinchome/slots/8/PanDA_Pilot-4566367059/memory_monitor_summary.json (trf name=prmon)
2019-12-03 05:45:24,979 | DEBUG | queue_monitor | pilot.user.atlas.utilities | get_memory_monitor_info | summary_dictionary={'Max': {'rx_packets': 295, 'nprocs': 5, 'nthreads': 0, 'rx_bytes': 22446, 'wtime': 98, 'rss': 191004, 'write_bytes': 245760, 'vmem': 1153736, 'read_bytes': 200506368, 'stime': 6, 'tx_bytes': 9201, 'pss': 163927, 'wchar': 2651264, 'rchar': 38673654, 'tx_packets': 51, 'swap': 0, 'utime': 7}, 'Avg': {'write_bytes': 2493, 'nprocs': 5, 'nthreads': 0, 'rx_bytes': 227, 'rx_packets': 2, 'vmem': 1153736, 'read_bytes': 2034358, 'swap': 0, 'tx_bytes': 93, 'pss': 163927, 'wchar': 26900, 'rchar': 392386, 'tx_packets': 0, 'rss': 191004}}
2019-12-03 05:45:24,984 | INFO | queue_monitor | pilot.user.atlas.utilities | get_memory_monitor_info | extracted standard info from prmon json
2019-12-03 05:45:24,984 | INFO | queue_monitor | pilot.user.atlas.utilities | get_memory_monitor_info | extracted standard memory fields from prmon json
2019-12-03 05:45:24,984 | INFO | queue_monitor | pilot.util.auxiliary.4566367059 | timing_report | ..............................
2019-12-03 05:45:24,984 | INFO | queue_monitor | pilot.util.auxiliary.4566367059 | timing_report | . Timing measurements:
2019-12-03 05:45:24,984 | INFO | queue_monitor | pilot.util.auxiliary.4566367059 | timing_report | . get job = 2 s
2019-12-03 05:45:24,984 | INFO | queue_monitor | pilot.util.auxiliary.4566367059 | timing_report | . initial setup = 5 s
2019-12-03 05:45:24,989 | INFO | queue_monitor | pilot.util.auxiliary.4566367059 | timing_report | . payload setup = 0 s
2019-12-03 05:45:24,989 | INFO | queue_monitor | pilot.util.auxiliary.4566367059 | timing_report | . total setup = 5 s
2019-12-03 05:45:24,989 | INFO | queue_monitor | pilot.util.auxiliary.4566367059 | timing_report | . stage-in = 1 s
2019-12-03 05:45:24,989 | INFO | queue_monitor | pilot.util.auxiliary.4566367059 | timing_report | . payload execution = 197 s
2019-12-03 05:45:24,989 | INFO | queue_monitor | pilot.util.auxiliary.4566367059 | timing_report | . stage-out = 5 s
2019-12-03 05:45:24,989 | INFO | queue_monitor | pilot.util.auxiliary.4566367059 | timing_report | ..............................
2019-12-03 05:45:24,989 | INFO | queue_monitor | pilot.util.auxiliary.4566367059 | get_log_extracts | building log extracts (sent to the server as 'pilotLog')
2019-12-03 05:45:24,994 | DEBUG | queue_monitor | pilot.util.auxiliary.4566367059 | get_panda_tracer_log | PanDA tracer log does not exist: /tmp/boinchome/slots/8/PanDA_Pilot-4566367059/pandatracerlog.txt (ignoring)
2019-12-03 05:45:24,994 | INFO | queue_monitor | pilot.util.container | execute | executing command: tail -n 20 /tmp/boinchome/slots/8/PanDA_Pilot-4566367059/pilotlog.txt
2019-12-03 05:45:25,078 | WARNING | queue_monitor | pilot.control.job | add_timing_and_extracts |
XXXXXXXXXXXXXXXXXXXXX[begin log extracts]
- Log from pilotlog.txt -
2019-12-03 05:45:24,944 | WARNING | queue_monitor | pilot.api.analytics | get_fitted_data | wrong length of table data, x=[1575351783.0], y=[163927.0] (must be same and length>=4)
2019-12-03 05:45:24,944 | DEBUG | queue_monitor | pilot.util.auxiliary.4566367059 | get_job_metrics | job metrics="coreCount=4 actualCoreCount=3"
2019-12-03 05:45:24,945 | INFO | queue_monitor | pilot.control.job.4566367059 | get_data_structure | payload/TRF did not report the number of read events
2019-12-03 05:45:24,955 | INFO | queue_monitor | pilot.user.atlas.utilities | get_memory_values | using path: /tmp/boinchome/slots/8/PanDA_Pilot-4566367059/memory_monitor_summary.json (trf name=prmon)
2019-12-03 05:45:24,979 | DEBUG | queue_monitor | pilot.user.atlas.utilities | get_memory_monitor_info | summary_dictionary={'Max': {'rx_packets': 295, 'nprocs': 5, 'nthreads': 0, 'rx_bytes': 22446, 'wtime': 98, 'rss': 191004, 'write_bytes': 245760, 'vmem': 1153736, 'read_bytes': 200506368, 'stime': 6, 'tx_bytes': 9201, 'pss': 163927, 'wchar': 2651264, 'rchar': 38673654, 'tx_packets': 51, 'swap': 0, 'utime': 7}, 'Avg': {'write_bytes': 2493, 'nprocs': 5, 'nthreads': 0, 'rx_bytes': 227, 'rx_packets': 2, 'vmem': 1153736, 'read_bytes': 2034358, 'swap': 0, 'tx_bytes': 93, 'pss': 163927, 'wchar': 26900, 'rchar': 392386, 'tx_packets': 0, 'rss': 191004}}
2019-12-03 05:45:24,984 | INFO | queue_monitor | pilot.user.atlas.utilities | get_memory_monitor_info | extracted standard info from prmon json
2019-12-03 05:45:24,984 | INFO | queue_monitor | pilot.user.atlas.utilities | get_memory_monitor_info | extracted standard memory fields from prmon json
2019-12-03 05:45:24,984 | INFO | queue_monitor | pilot.util.auxiliary.4566367059 | timing_report | ..............................
2019-12-03 05:45:24,984 | INFO | queue_monitor | pilot.util.auxiliary.4566367059 | timing_report | . Timing measurements:
2019-12-03 05:45:24,984 | INFO | queue_monitor | pilot.util.auxiliary.4566367059 | timing_report | . get job = 2 s
2019-12-03 05:45:24,984 | INFO | queue_monitor | pilot.util.auxiliary.4566367059 | timing_report | . initial setup = 5 s
2019-12-03 05:45:24,989 | INFO | queue_monitor | pilot.util.auxiliary.4566367059 | timing_report | . payload setup = 0 s
2019-12-03 05:45:24,989 | INFO | queue_monitor | pilot.util.auxiliary.4566367059 | timing_report | . total setup = 5 s
2019-12-03 05:45:24,989 | INFO | queue_monitor | pilot.util.auxiliary.4566367059 | timing_report | . stage-in = 1 s
2019-12-03 05:45:24,989 | INFO | queue_monitor | pilot.util.auxiliary.4566367059 | timing_report | . payload execution = 197 s
2019-12-03 05:45:24,989 | INFO | queue_monitor | pilot.util.auxiliary.4566367059 | timing_report | . stage-out = 5 s
2019-12-03 05:45:24,989 | INFO | queue_monitor | pilot.util.auxiliary.4566367059 | timing_report | ..............................
2019-12-03 05:45:24,989 | INFO | queue_monitor | pilot.util.auxiliary.4566367059 | get_log_extracts | building log extracts (sent to the server as 'pilotLog')
2019-12-03 05:45:24,994 | DEBUG | queue_monitor | pilot.util.auxiliary.4566367059 | get_panda_tracer_log | PanDA tracer log does not exist: /tmp/boinchome/slots/8/PanDA_Pilot-4566367059/pandatracerlog.txt (ignoring)
2019-12-03 05:45:24,994 | INFO | queue_monitor | pilot.util.container | execute | executing command: tail -n 20 /tmp/boinchome/slots/8/PanDA_Pilot-4566367059/pilotlog.txt
XXXXXXXXXXXXXXXXXXXXX[end log extracts]
2019-12-03 05:45:25,079 | DEBUG | queue_monitor | pilot.control.job.4566367059 | send_state | wrote heartbeat to file /tmp/boinchome/slots/8/heartbeat.json
2019-12-03 05:45:25,079 | DEBUG | queue_monitor | pilot.control.job | queue_monitor | job 4566367059 was dequeued from the monitored payloads queue
2019-12-03 05:45:25,079 | DEBUG | queue_monitor | pilot.control.job | queue_monitor | tmp job object deleted
2019-12-03 05:45:25,095 | INFO | retrieve | pilot.util.auxiliary.4566367059 | make_job_report |
2019-12-03 05:45:25,095 | INFO | retrieve | pilot.util.auxiliary.4566367059 | make_job_report | job summary report
2019-12-03 05:45:25,095 | INFO | retrieve | pilot.util.auxiliary.4566367059 | make_job_report | --------------------------------------------------
2019-12-03 05:45:25,095 | INFO | retrieve | pilot.util.auxiliary.4566367059 | make_job_report | PanDA job id: 4566367059
2019-12-03 05:45:25,095 | INFO | retrieve | pilot.util.auxiliary.4566367059 | make_job_report | task id: 19905999
2019-12-03 05:45:25,095 | INFO | retrieve | pilot.util.auxiliary.4566367059 | make_job_report | errors: (none)
2019-12-03 05:45:25,095 | INFO | retrieve | pilot.util.auxiliary.4566367059 | make_job_report | status: LOG_TRANSFER = DONE
2019-12-03 05:45:25,095 | INFO | retrieve | pilot.util.auxiliary.4566367059 | make_job_report | pilot state: failed
2019-12-03 05:45:25,095 | INFO | retrieve | pilot.util.auxiliary.4566367059 | make_job_report | transexitcode: 65
2019-12-03 05:45:25,095 | INFO | retrieve | pilot.util.auxiliary.4566367059 | make_job_report | exeerrorcode: 65
2019-12-03 05:45:25,096 | INFO | retrieve | pilot.util.auxiliary.4566367059 | make_job_report | exeerrordiag: EVNTtoHITS got a SIGABRT signal (exit code 134)
2019-12-03 05:45:25,096 | INFO | retrieve | pilot.util.auxiliary.4566367059 | make_job_report | exitcode: 65
2019-12-03 05:45:25,096 | INFO | retrieve | pilot.util.auxiliary.4566367059 | make_job_report | exitmsg: EVNTtoHITS got a SIGABRT signal (exit code 134)
2019-12-03 05:45:25,096 | INFO | retrieve | pilot.util.auxiliary.4566367059 | make_job_report | cpuconsumptiontime: 32 s
2019-12-03 05:45:25,096 | INFO | retrieve | pilot.util.auxiliary.4566367059 | make_job_report | nevents: 0
2019-12-03 05:45:25,096 | INFO | retrieve | pilot.util.auxiliary.4566367059 | make_job_report | neventsw: 0
2019-12-03 05:45:25,096 | INFO | retrieve | pilot.util.auxiliary.4566367059 | make_job_report | pid: 1392681
2019-12-03 05:45:25,096 | INFO | retrieve | pilot.util.auxiliary.4566367059 | make_job_report | pgrp: 1392681
2019-12-03 05:45:25,096 | INFO | retrieve | pilot.util.auxiliary.4566367059 | make_job_report | corecount: 4
2019-12-03 05:45:25,096 | INFO | retrieve | pilot.util.auxiliary.4566367059 | make_job_report | event service: False
2019-12-03 05:45:25,096 | INFO | retrieve | pilot.util.auxiliary.4566367059 | make_job_report | --------------------------------------------------
2019-12-03 05:45:25,096 | INFO | retrieve | pilot.util.auxiliary.4566367059 | make_job_report |
2019-12-03 05:45:25,096 | INFO | retrieve | pilot.control.job.4566367059 | has_job_completed | job 4566367059 has completed
2019-12-03 05:45:25,096 | INFO | retrieve | pilot.util.processes | cleanup | overall cleanup function is called
2019-12-03 05:45:25,118 | DEBUG | retrieve | pilot.util.processes | cleanup | work directory was removed: /tmp/boinchome/slots/8/PanDA_Pilot-4566367059
2019-12-03 05:45:26,124 | INFO | retrieve | pilot.info.jobdata | collect_zombies | --- collectZombieJob: --- 10, [1392681]
2019-12-03 05:45:26,124 | INFO | retrieve | pilot.info.jobdata | collect_zombies | zombie collector trying to kill pid 1392681
2019-12-03 05:45:26,124 | INFO | retrieve | pilot.info.jobdata | collect_zombies | harmless exception when collecting zombies: [Errno 10] No child processes
2019-12-03 05:45:27,131 | INFO | retrieve | pilot.util.processes | cleanup | collected zombie processes
2019-12-03 05:45:27,131 | INFO | retrieve | pilot.util.processes | cleanup | will now attempt to kill all subprocesses of pid=1392681
2019-12-03 05:45:28,106 | INFO | retrieve | pilot.util.processes | kill_processes | process IDs to be killed: [1392681] (in reverse order)
2019-12-03 05:45:28,238 | WARNING | retrieve | pilot.util.processes | kill_processes | found no corresponding commands to process id(s)
2019-12-03 05:45:28,245 | INFO | retrieve | pilot.util.processes | kill_orphans | Do not look for orphan processes in BOINC jobs
2019-12-03 05:45:28,245 | INFO | retrieve | pilot.util.processes | cleanup | resetting job object
2019-12-03 05:45:28,245 | INFO | retrieve | pilot.control.job | retrieve | ready for new job
2019-12-03 05:45:28,251 | INFO | retrieve | root | retrieve | pilot has finished for previous job - re-establishing logging
mpi4py not found
2019-12-03 05:45:28,347 | INFO | retrieve | pilot.util.auxiliary | pilot_version_banner | ****************************************
2019-12-03 05:45:28,347 | INFO | retrieve | pilot.util.auxiliary | pilot_version_banner | *** PanDA Pilot version 2.3.3 (16) ***
2019-12-03 05:45:28,347 | INFO | retrieve | pilot.util.auxiliary | pilot_version_banner | ****************************************
2019-12-03 05:45:28,347 | INFO | retrieve | pilot.util.auxiliary | pilot_version_banner |
2019-12-03 05:45:28,364 | INFO | retrieve | pilot.util.auxiliary | display_architecture_info | architecture information:
2019-12-03 05:45:30,158 | INFO | retrieve | pilot.util.auxiliary | display_architecture_info |
LSB Version: :core-4.1-amd64:core-4.1-noarch
Distributor ID: CentOS
Description: CentOS Linux release 7.6.1810 (Core)
Release: 7.6.1810
Codename: Core
2019-12-03 05:45:30,158 | INFO | retrieve | pilot.util.auxiliary | pilot_version_banner | ****************************************
2019-12-03 05:45:30,663 | DEBUG | retrieve | pilot.util.monitoring | check_local_space | checking local space on /tmp/boinchome/slots/8
2019-12-03 05:45:30,983 | INFO | retrieve | pilot.util.monitoring | check_local_space | sufficient remaining disk space (1861270634496 B)
2019-12-03 05:45:30,983 | WARNING | retrieve | pilot.control.job | proceed_with_getjob | since timefloor is set to 0, pilot was only allowed to run one job
2019-12-03 05:45:30,984 | DEBUG | retrieve | pilot.control.job | retrieve | [job] retrieve thread has finished
2019-12-03 05:45:30,994 | WARNING | copytool_out | pilot.util.common | should_abort | data:copytool_out:received graceful stop - abort after this iteration
2019-12-03 05:45:30,994 | WARNING | queue_monitoring | pilot.util.common | should_abort | data:queue_monitoring:received graceful stop - abort after this iteration
2019-12-03 05:45:31,003 | WARNING | queue_monitor | pilot.util.common | should_abort | job:queue_monitor:received graceful stop - abort after this iteration
2019-12-03 05:45:31,003 | DEBUG | queue_monitor | pilot.control.job | queue_monitor | [job] queue monitor thread has finished
2019-12-03 05:45:31,039 | DEBUG | MainThread | pilot.workflow.generic | run | thread count now at 15 threads
2019-12-03 05:45:31,039 | DEBUG | MainThread | pilot.workflow.generic | run | enumerate: [<_MainThread(MainThread, started 140557427050304)>, <ExcThread(job, started 140557264631552)>, <ExcThread(job_monitor, started 140556529284864)>, <ExcThread(queue_monitoring, started 140557133264640)>, <ExcThread(validate_post, started 140556562855680)>, <ExcThread(monitor, started 140557141657344)>, <ExcThread(validate, started 140557256238848)>, <ExcThread(validate_pre, started 140557150050048)>, <ExcThread(failed_post, started 140556546070272)>, <ExcThread(execute_payloads, started 140556537677568)>, <ExcThread(copytool_out, started 140557166835456)>, <ExcThread(copytool_in, started 140557124871936)>, <ExcThread(data, started 140557247846144)>, <ExcThread(create_data_payload, started 140557116479232)>, <ExcThread(payload, started 140557158442752)>]
2019-12-03 05:45:31,102 | INFO | monitor | pilot.control.monitor | control | [monitor] control thread has ended
2019-12-03 05:45:31,213 | DEBUG | validate | pilot.control.job | validate | [job] validate thread has finished
2019-12-03 05:45:31,213 | INFO | validate_pre | pilot.control.payload | validate_pre | [payload] validate_pre thread has finished
2019-12-03 05:45:31,220 | DEBUG | create_data_payload | pilot.control.job | create_data_payload | [job] create_data_payload thread has finished
2019-12-03 05:45:31,226 | DEBUG | data | pilot.control.data | control | data control ending since graceful_stop has been set
2019-12-03 05:45:31,226 | DEBUG | data | pilot.control.data | control | [data] control thread has finished
2019-12-03 05:45:31,570 | INFO | failed_post | pilot.control.payload | failed_post | [payload] failed_post thread has finished
2019-12-03 05:45:31,826 | INFO | validate_post | pilot.control.payload | validate_post | [payload] validate_post thread has finished
2019-12-03 05:45:31,846 | DEBUG | payload | pilot.control.payload | control | payload control ending since graceful_stop has been set
2019-12-03 05:45:31,846 | DEBUG | payload | pilot.control.payload | control | [payload] control thread has finished
2019-12-03 05:45:31,996 | DEBUG | copytool_out | pilot.control.data | copytool_out | [data] copytool_out thread has finished
2019-12-03 05:45:32,091 | INFO | execute_payloads | pilot.control.payload | execute_payloads | [payload] execute_payloads thread has finished
2019-12-03 05:45:32,103 | DEBUG | copytool_in | pilot.control.data | copytool_in | [data] copytool_in thread has finished
2019-12-03 05:45:32,149 | DEBUG | MainThread | pilot.workflow.generic | run | thread count now at 4 threads
2019-12-03 05:45:32,149 | DEBUG | MainThread | pilot.workflow.generic | run | enumerate: [<_MainThread(MainThread, started 140557427050304)>, <ExcThread(job, started 140557264631552)>, <ExcThread(job_monitor, started 140556529284864)>, <ExcThread(queue_monitoring, started 140557133264640)>]
2019-12-03 05:45:32,234 | DEBUG | job | pilot.control.job | control | job control ending since graceful_stop has been set
2019-12-03 05:45:32,234 | DEBUG | job | pilot.control.job | control | [job] control thread has finished
2019-12-03 05:45:33,157 | DEBUG | MainThread | pilot.workflow.generic | run | thread count now at 3 threads
2019-12-03 05:45:33,157 | DEBUG | MainThread | pilot.workflow.generic | run | enumerate: [<_MainThread(MainThread, started 140557427050304)>, <ExcThread(job_monitor, started 140556529284864)>, <ExcThread(queue_monitoring, started 140557133264640)>]
2019-12-03 05:45:34,009 | DEBUG | queue_monitoring | pilot.control.data | queue_monitoring | [data] queue_monitor thread has finished
2019-12-03 05:45:34,162 | DEBUG | MainThread | pilot.workflow.generic | run | thread count now at 2 threads
2019-12-03 05:45:34,162 | DEBUG | MainThread | pilot.workflow.generic | run | enumerate: [<_MainThread(MainThread, started 140557427050304)>, <ExcThread(job_monitor, started 140556529284864)>]
2019-12-03 05:46:10,195 | WARNING | job_monitor | pilot.control.job | check_job_monitor_waiting_time | no jobs in monitored_payloads queue (waited for 62 s)
2019-12-03 05:46:10,195 | DEBUG | job_monitor | pilot.control.job | job_monitor | [job] job monitor thread has finished
2019-12-03 05:46:10,403 | INFO | MainThread | pilot.workflow.generic | run | end of generic workflow (traces error code: 0)
2019-12-03 05:46:10,403 | INFO | MainThread | root | wrap_up | traces error code: 0
2019-12-03 05:46:10,403 | INFO | MainThread | root | wrap_up | pilot has finished
2019-12-03 05:46:11 UTC [wrapper] ==== pilot stdout END ====
2019-12-03 05:46:11 UTC [wrapper] ==== wrapper stdout RESUME ====
2019-12-03 05:46:11 UTC [wrapper] Pilot exit status: 0
2019-12-03 05:46:11 UTC [wrapper] STATUSCODE: 0
2019-12-03 05:46:11 UTC [wrapper] apfmon messages muted
---- find pandaIDs.out ----
total 64
-rw------- 1 boincer umatlas 11357 Jul 25 10:38 LICENSE
-rw------- 1 boincer umatlas 20 Sep 9 07:04 MANIFEST.IN
-rw------- 1 boincer umatlas 11 Dec 3 00:39 pandaIDs.out
drwx------ 14 boincer umatlas 4096 Dec 3 00:39 pilot
-rwx------ 1 boincer umatlas 20750 Dec 2 05:02 pilot.py
-rw------- 1 boincer umatlas 8 Dec 2 05:02 PILOTVERSION
-rw------- 1 boincer umatlas 2212 Nov 14 05:01 README.md
-rw------- 1 boincer umatlas 766 Oct 10 10:01 setup.py
-rw------- 1 boincer umatlas 221 Jul 25 10:38 TODO.md
2019-12-03 05:46:12 UTC [wrapper] pandaIDs.out files:
-rw------- 1 boincer umatlas 11 Dec 3 00:39 /tmp/boinchome/slots/8/pilot2/pandaIDs.out
2019-12-03 05:46:12 UTC [wrapper] pandaIDs.out content:
4566367059
2019-12-03 05:46:12 UTC [wrapper] Test setup, not cleaning
2019-12-03 05:46:12 UTC [wrapper] ==== wrapper stdout END ====
2019-12-03 05:46:12 UTC [wrapper] ==== wrapper stderr END ====
2019-12-03 05:46:13 UTC [wrapper] wrapper wrapperexiting ec=0, duration=468
2019-12-03 05:46:13 UTC [wrapper] apfmon messages muted
2019-12-03 00:46:13,740: ***************diag file************
2019-12-03 00:46:13,740: runtimeenvironments=APPS/HEP/ATLAS-SITE;
systemsoftware=CentOS Linux 7 (Core)
Processors=4
Benchmark=HEPSPEC:1.0
WallTime=468.31s
KernelTime=23.65s
UserTime=34.86s
CPUUsage=12%
MaxResidentMemory=153660kB
AverageResidentMemory=0kB
AverageTotalMemory=0kB
AverageUnsharedMemory=0kB
AverageUnsharedStack=0kB
AverageSharedMemory=0kB
PageSize=4096B
MajorPageFaults=2177
MinorPageFaults=4304330
Swaps=0
ForcedSwitches=27825
WaitSwitches=519936
Inputs=648946
Outputs=30720
SocketReceived=0
SocketSent=0
Signals=0
nodename=AGLT2@c6-5-7-1.aglt2.org
exitcode=0
exitcode=0
2019-12-03 00:46:13,773: ******************************WorkDir***********************
2019-12-03 00:46:13,773: total 231048
drwxrwx--x 6 boincer umatlas 4096 Dec 3 00:46 .
drwxrwx--x 16 boincer umatlas 4096 Dec 3 00:01 ..
-rw------- 1 boincer umatlas 7756965 Dec 3 00:39 agis_ddmendpoints.json
-rw------- 1 boincer umatlas 3752879 Dec 3 00:39 agis_schedconf.cvmfs.json
drwx------ 2 boincer umatlas 4096 Dec 3 00:38 .alrb
drwxr-xr-x 3 boincer umatlas 4096 Dec 3 00:38 APPS
-rw------- 1 boincer umatlas 542 Dec 3 00:38 .asetup
-rw------- 1 boincer umatlas 4238 Dec 3 00:41 .asetup.save
-rw-r--r-- 1 boincer umatlas 0 Dec 3 00:38 boinc_lockfile
-rw-r--r-- 1 boincer umatlas 8192 Dec 3 00:46 boinc_mmap_file
-rw-r--r-- 1 boincer umatlas 524 Dec 3 00:43 boinc_task_state.xml
-rw-r--r-- 1 boincer umatlas 224165771 Dec 3 00:38 EVNT.19609563._000318.pool.root.1
-rw------- 1 boincer umatlas 7208 Dec 3 00:45 heartbeat.json
-rw-r--r-- 1 boincer umatlas 6238 Dec 3 00:40 init_data.xml
-rw-r--r-- 1 boincer umatlas 275094 Dec 3 00:38 input.tar.gz
-rw-r--r-- 1 boincer umatlas 105 Dec 3 00:38 job.xml
-rw------- 1 boincer umatlas 119303 Dec 3 00:46 log.19905999._015547.job.log.1
-rw------- 1 boincer umatlas 23482 Dec 3 00:43 log.19905999._015547.job.log.tgz.1
-rw------- 1 boincer umatlas 801 Dec 3 00:43 memory_monitor_summary.json
-rw------- 1 boincer umatlas 241 Dec 3 00:46 output.list
-rw------- 1 boincer umatlas 2892 Dec 2 23:21 pandaJobData.out
drwx------ 3 boincer umatlas 4096 Dec 3 00:39 pilot2
-rw-r--r-- 1 boincer umatlas 266835 Dec 2 22:28 pilot2.tar.gz
-rw------- 1 boincer umatlas 8424 Dec 3 00:46 pilotlog.txt
-rw-r--r-- 1 boincer umatlas 4474 Dec 2 23:20 queuedata.json
-rw-r--r-- 1 boincer umatlas 812 Dec 3 00:38 RTE.tar.gz
-rwxr-xr-x 1 boincer umatlas 8247 Dec 3 00:38 run_atlas
-rwx------ 1 boincer umatlas 12849 Dec 2 23:21 runpilot2-wrapper.sh
-rw-r--r-- 1 boincer umatlas 692 Dec 3 00:46 runtime_log
-rw-r--r-- 1 boincer umatlas 7915 Dec 3 00:46 runtime_log.err
-rw------- 1 boincer umatlas 562 Dec 3 00:46 Sd2NDm0rEwvnsSi4apGgGQJmABFKDmABFKDmTspPDmABFKDmS8xVBn.diag
drwxrwx--x 2 boincer umatlas 4096 Dec 3 00:46 shared
-rw-r--r-- 1 boincer umatlas 15088 Dec 3 00:38 start_atlas.sh
-rw-r--r-- 1 boincer umatlas 32268 Dec 3 00:46 stderr.txt
-rw-r--r-- 1 boincer umatlas 100 Dec 3 00:38 wrapper_26015_x86_64-pc-linux-gnu
-rw-r--r-- 1 boincer umatlas 23 Dec 3 00:46 wrapper_checkpoint.txt
2019-12-03 00:46:13,773: running start_atlas return value is 0
2019-12-03 00:46:13,773: Parent exit 0
2019-12-03 00:46:13,776: child process exit 0
00:46:14 (1309305): run_atlas exited; CPU time 35.487201
00:46:14 (1309305): called boinc_finish(0)
</stderr_txt>
]]>
©2025 CERN