Name | zR4KDm7F7ttnlyackoJh5iwnABFKDmABFKDmX6hbDmABFKDmOMowfm_0 |
Workunit | 105220344 |
Created | 20 Dec 2018, 6:45:16 UTC |
Sent | 20 Dec 2018, 9:54:22 UTC |
Report deadline | 28 Dec 2018, 9:54:22 UTC |
Received | 20 Dec 2018, 11:11:15 UTC |
Server state | Over |
Outcome | Success |
Client state | Done |
Exit status | 0 (0x00000000) |
Computer ID | 10648042 |
Run time | 5 min 2 sec |
CPU time | 2 min 4 sec |
Validate state | Valid |
Credit | 21.98 |
Device peak FLOPS | 28.40 GFLOPS |
Application version | ATLAS Simulation v2.54 (native_mt) x86_64-pc-linux-gnu |
Peak working set size | 1.70 GB |
Peak swap size | 2.26 GB |
Peak disk usage | 393.54 MB |
<core_client_version>7.9.3</core_client_version> <![CDATA[ <stderr_txt> 11:05:29 (4398): wrapper (7.7.26015): starting 11:05:29 (4398): wrapper: running run_atlas (--nthreads 6) singularity image is /cvmfs/atlas.cern.ch/repo/images/singularity/x86_64-slc6.img sys.argv = ['run_atlas', '--nthreads', '6'] THREADS=6 Checking for CVMFS CVMFS is installed OS:cat: /etc/redhat-release: No such file or directory This is not SLC6, need to run with Singularity.... Checking Singularity... Singularity is installed copy /var/lib/boinc-client/slots/10/shared/input.tar.gz copy /var/lib/boinc-client/slots/10/shared/RTE.tar.gz copy /var/lib/boinc-client/slots/10/shared/ATLAS.root_0 copy /var/lib/boinc-client/slots/10/shared/start_atlas.sh export ATHENA_PROC_NUMBER=6;start atlas job with PandaID=4184709929 Testing the function of Singularity... check singularity with cmd:singularity exec -B /cvmfs /cvmfs/atlas.cern.ch/repo/images/singularity/x86_64-slc6.img hostname Singularity Works... cmd = singularity exec --pwd /var/lib/boinc-client/slots/10 -B /cvmfs,/var /cvmfs/atlas.cern.ch/repo/images/singularity/x86_64-slc6.img sh start_atlas.sh > runtime_log 2> runtime_log.err running cmd return value is 0 ***********************log_extracts.txt************************* - Last 10 lines from /var/lib/boinc-client/slots/10/Panda_Pilot_5666_1545300349/PandaJob/athena_stdout.txt - PyJobTransforms.trfExe.preExecute 2018-12-20 11:06:15,289 INFO Batch/grid running - command outputs will not be echoed. Logs for EVNTtoHITS are in log.EVNTtoHITS PyJobTransforms.trfExe.preExecute 2018-12-20 11:06:15,290 INFO Now writing wrapper for substep executor EVNTtoHITS PyJobTransforms.trfExe._writeAthenaWrapper 2018-12-20 11:06:15,291 INFO Valgrind not engaged PyJobTransforms.trfExe.preExecute 2018-12-20 11:06:15,291 INFO Athena will be executed in a subshell via ['./runwrapper.EVNTtoHITS.sh'] PyJobTransforms.trfExe.execute 2018-12-20 11:06:15,291 INFO Starting execution of EVNTtoHITS (['./runwrapper.EVNTtoHITS.sh']) PyJobTransforms.trfExe.execute 2018-12-20 11:08:04,486 INFO EVNTtoHITS executor returns 65 PyJobTransforms.trfExe.validate 2018-12-20 11:08:05,392 ERROR Validation of return code failed: Non-zero return code from EVNTtoHITS (65) (Error code 65) PyJobTransforms.trfExe.validate 2018-12-20 11:08:05,407 INFO Scanning logfile log.EVNTtoHITS for errors PyJobTransforms.transform.execute 2018-12-20 11:08:05,721 CRITICAL Transform executor raised TransformValidationException: Non-zero return code from EVNTtoHITS (65); Logfile error in log.EVNTtoHITS: "AthMpEvtLoopMgr FATAL makePool failed for AthMpEvtLoopMgr.SharedEvtQueueProvider" PyJobTransforms.transform.execute 2018-12-20 11:08:08,828 WARNING Transform now exiting early with exit code 65 (Non-zero return code from EVNTtoHITS (65); Logfile error in log.EVNTtoHITS: "AthMpEvtLoopMgr FATAL makePool failed for AthMpEvtLoopMgr.SharedEvtQueueProvider") - Walltime - JobRetrival=0, StageIn=8, Execution=143, StageOut=0, CleanUp=9 ***********************pilot_error_report.json********************* { "4184709929": { "2": [ { "pilotErrorCode": 0, "pilotErrorDiag": "Job failed: Non-zero failed job return code: 65" } ] } } *****************The last 100 lines of the pilot log****************** "sepath": "/atlas/disk/atlasdatadisk/rucio", "seprodpath": "/atlas/disk/atlasdatadisk/rucio", "setokens": "ATLASDATADISK", "site": "BOINC", "siteid": "BOINC_MCORE", "sitershare": null, "space": 0, "special_par": null, "stageinretry": 2, "stageoutretry": 2, "status": "brokeroff", "statusoverride": "offline", "sysconfig": "manual", "system": "arc", "tags": "arc", "tier": "T3", "timefloor": 0, "tmpdir": null, "transferringlimit": 20000, "tspace": "2070-01-01T00:00:00", "use_newmover": "True", "validatedreleases": "True", "version": null, "wansinklimit": null, "wansourcelimit": null, "wnconnectivity": "full", "wntmpdir": null, "workflow": null } 2018-12-20 10:05:49|5666|SiteInformat| Queuedata was successfully downloaded by pilot wrapper script 2018-12-20 10:05:49|5666|ATLASSiteInf| curl command returned valid queuedata 2018-12-20 10:05:49|5666|ATLASSiteInf| Site BOINC_MCORE is currently in brokeroff mode 2018-12-20 10:05:49|5666|ATLASSiteInf| Job recovery turned off 2018-12-20 10:05:49|5666|ATLASSiteInf| Confirmed correctly formatted rucio sepath 2018-12-20 10:05:49|5666|ATLASSiteInf| Confirmed correctly formatted rucio seprodpath 2018-12-20 10:05:49|5666|SiteInformat| Evaluating queuedata 2018-12-20 10:05:49|5666|SiteInformat| Setting unset pilot variables using queuedata 2018-12-20 10:05:49|5666|SiteInformat| appdir: 2018-12-20 10:05:49|5666|pUtil.py | File registration will be done by server 2018-12-20 10:05:49|5666|pUtil.py | Updated stage-in retry number to 2 2018-12-20 10:05:49|5666|pUtil.py | Updated stage-out retry number to 2 2018-12-20 10:05:49|5666|pUtil.py | Detected unset (NULL) release/homepackage string 2018-12-20 10:05:49|5666|ATLASExperim| Application dir confirmed: /var/lib/boinc-client/slots/10/ 2018-12-20 10:05:49|5666|pilot.py | Pilot will serve experiment: Nordugrid-ATLAS 2018-12-20 10:05:49|5666|ATLASExperim| Architecture information: 2018-12-20 10:05:49|5666|ATLASExperim| Excuting command: lsb_release -a 2018-12-20 10:05:49|5666|ATLASExperim| sh: lsb_release: command not found 2018-12-20 10:05:49|5666|pUtil.py | getSiteInformation: got experiment=ATLAS 2018-12-20 10:05:49|5666|ATLASExperim| appdirs = ['/cvmfs/atlas.cern.ch/repo/sw'] 2018-12-20 10:05:49|5666|ATLASExperim| head of /cvmfs/atlas.cern.ch/repo/sw/ChangeLog: -------------------------------------------------------------------------------- 2018-12-20 11:00:41 Alessandro De Salvo * + AGISData 20181220110041 2018-12-20 10:00:51 Alessandro De Salvo * + AGISData 20181220100051 2018-12-20 09:01:00 Alessandro De Salvo * + AGISData 20181220090100 2018-12-20 08:02:19 Alessandro De Salvo -------------------------------------------------------------------------------- 2018-12-20 10:05:49|5666|ATLASExperim| ATLAS_PYTHON_PILOT set to /usr/bin/python 2018-12-20 10:05:49|5666|pUtil.py | getSiteInformation: got experiment=ATLAS 2018-12-20 10:05:49|5666|ATLASExperim| Executing command: export ATLAS_LOCAL_ROOT_BASE=/cvmfs/atlas.cern.ch/repo/ATLASLocalRootBase;$ATLAS_LOCAL_ROOT_BASE/utilities/checkValidity.sh (time-out: 300) 2018-12-20 10:05:49|5666|pUtil.py | Executing command: export ATLAS_LOCAL_ROOT_BASE=/cvmfs/atlas.cern.ch/repo/ATLASLocalRootBase;$ATLAS_LOCAL_ROOT_BASE/utilities/checkValidity.sh (protected by timed_command, timeout: 300 s) 2018-12-20 10:05:50|5666|pUtil.py | Elapsed time: 1 2018-12-20 10:05:50|5666|ATLASExperim| Diagnostics tool has verified CVMFS 2018-12-20 10:05:50|5666|Node.py | Collecting machine features 2018-12-20 10:05:50|5666|Node.py | $MACHINEFEATURES not defined locally 2018-12-20 10:05:50|5666|Node.py | $JOBFEATURES not defined locally 2018-12-20 10:05:50|5666|Node.py | Executing command: hostname -i 2018-12-20 10:05:50|5666|Node.py | IP number of worker node: 127.0.1.1 2018-12-20 10:05:50|5666|pUtil.py | getSiteInformation: got experiment=Nordugrid-ATLAS 2018-12-20 10:05:50|5666|pilot.py | Using site information for experiment: Nordugrid-ATLAS 2018-12-20 10:05:50|5666|pilot.py | Will attempt to create workdir: /var/lib/boinc-client/slots/10/Panda_Pilot_5666_1545300349 2018-12-20 10:05:50|5666|pilot.py | Creating file: /var/lib/boinc-client/slots/10/CURRENT_SITEWORKDIR 2018-12-20 10:05:50|5666|pUtil.py | Wrote string "/var/lib/boinc-client/slots/10/Panda_Pilot_5666_1545300349" to file: /var/lib/boinc-client/slots/10/CURRENT_SITEWORKDIR 2018-12-20 10:05:50|5666|ATLASExperim| ATLAS_POOLCOND_PATH not set by wrapper 2018-12-20 10:05:50|5666|pilot.py | Preparing to execute Cleaner 2018-12-20 10:05:50|5666|pilot.py | Cleaning /var/lib/boinc-client/slots/10 2018-12-20 10:05:50|5666|Cleaner.py | Cleaner initialized with clean-up limit: 2 hours 2018-12-20 10:05:50|5666|Cleaner.py | Cleaner will scan for lost directories in verified path: /var/lib/boinc-client/slots/10 2018-12-20 10:05:50|5666|Cleaner.py | Executing empty dirs clean-up, stage 1/5 2018-12-20 10:05:50|5666|Cleaner.py | Purged 0 empty directories 2018-12-20 10:05:50|5666|Cleaner.py | Executing work dir clean-up, stage 2/5 2018-12-20 10:05:50|5666|Cleaner.py | Purged 0 single workDirs directories 2018-12-20 10:05:50|5666|Cleaner.py | Executing maxed-out dirs clean-up, stage 3/5 2018-12-20 10:05:50|5666|Cleaner.py | Purged 0 empty directories 2018-12-20 10:05:50|5666|Cleaner.py | Executing AthenaMP clean-up, stage 4/5 <SKIPPED> 2018-12-20 10:05:50|5666|Cleaner.py | Executing PanDA Pilot dir clean-up, stage 5/5 2018-12-20 10:05:50|5666|Cleaner.py | Number of found job state files: 0 2018-12-20 10:05:50|5666|Cleaner.py | No job state files were found, aborting clean-up 2018-12-20 10:05:50|5666|pilot.py | Update frequencies: 2018-12-20 10:05:50|5666|pilot.py | ...Processes: 300 s 2018-12-20 10:05:50|5666|pilot.py | .......Space: 600 s 2018-12-20 10:05:50|5666|pilot.py | ......Server: 1800 s 2018-12-20 10:05:50|5666|pUtil.py | Timefloor set to zero in queuedata (multi-jobs disabled) ***************diag file************ runtimeenvironments=APPS/HEP/ATLAS-SITE; Processors=1 WallTime=282.46s KernelTime=8.41s UserTime=124.15s CPUUsage=46% MaxResidentMemory=2036204kB AverageResidentMemory=0kB AverageTotalMemory=0kB AverageUnsharedMemory=0kB AverageUnsharedStack=0kB AverageSharedMemory=0kB PageSize=4096B MajorPageFaults=8421 MinorPageFaults=2333466 Swaps=0 ForcedSwitches=127507 WaitSwitches=505087 Inputs=3760273 Outputs=63071 SocketReceived=0 SocketSent=0 Signals=0 nodename=Hydrosaure@boinc-ab350m exitcode=0 ******************************WorkDir*********************** total 192134 drwxrwx--x 6 boinc boinc 48 Dec 20 11:10 . drwxrwx--x 25 boinc boinc 25 Dec 18 16:52 .. -rw------- 1 boinc boinc 7158688 Dec 20 11:05 agis_ddmendpoints.cvmfs.json -rw------- 1 boinc boinc 5694493 Dec 20 11:06 agis_schedconf.cvmfs.json drwx------ 2 boinc boinc 2 Dec 20 11:06 .alrb drwxr-xr-x 3 boinc boinc 3 Dec 20 11:05 APPS -rwx------ 1 boinc boinc 2441 Dec 20 07:45 ARCpilot -rw------- 1 boinc boinc 550 Dec 20 11:05 .asetup -rw------- 1 boinc boinc 11051 Dec 20 11:06 .asetup.save -rw-r--r-- 1 boinc boinc 0 Dec 20 11:05 boinc_lockfile -rw-r--r-- 1 boinc boinc 8192 Dec 20 11:10 boinc_mmap_file -rw-r--r-- 1 boinc boinc 525 Dec 20 11:08 boinc_task_state.xml -rw------- 1 boinc boinc 58 Dec 20 11:05 CURRENT_SITEWORKDIR -rw-r--r-- 1 boinc boinc 193938300 Dec 20 11:05 EVNT.15754943._003171.pool.root.1 -rw-r--r-- 1 boinc boinc 6534 Dec 20 11:05 init_data.xml -rw-r--r-- 1 boinc boinc 1088718 Dec 20 11:05 input.tar.gz -rw------- 1 boinc boinc 4032 Dec 20 11:10 jobSmallFiles.tgz -rw-r--r-- 1 boinc boinc 105 Dec 20 11:05 job.xml -rw------- 1 boinc boinc 165253 Dec 20 11:10 log.16442994._137453.job.log.1 -rw------- 1 boinc boinc 156364 Dec 20 11:08 log.16442994._137453.job.log.tgz.1 -rw------- 1 boinc boinc 1730 Dec 20 11:09 log_extracts.txt -rw------- 1 boinc boinc 311 Dec 20 11:08 memory_monitor_summary.json -rw------- 1 boinc boinc 599 Dec 20 11:10 metadata-surl.xml -rw------- 1 boinc boinc 241 Dec 20 11:09 output.list -rw------- 1 boinc boinc 11 Dec 20 11:05 pandaIDs.out -rw------- 1 boinc boinc 2919 Dec 20 11:05 pandaJobData_1.out -rw------- 1 boinc boinc 2919 Dec 20 11:05 pandaJobData.out -rw------- 1 boinc boinc 10529 Dec 20 11:09 panda_node_struct.pickle -rw------- 1 boinc boinc 203 Dec 20 11:08 pilot_error_report.json -rw------- 1 boinc boinc 30 Dec 20 11:05 PILOT_INITDIR -rw------- 1 boinc boinc 137 Dec 20 11:10 pilotlog-last.txt -rw------- 1 boinc boinc 11375 Dec 20 11:05 pilotlog.txt drwx------ 3 boinc boinc 3 Dec 20 11:06 .pki -rw------- 1 boinc boinc 3801 Dec 20 11:05 queuedata.json -rw-r--r-- 1 boinc boinc 4436 Dec 20 07:40 queuedata.pilot.json -rw-r--r-- 1 boinc boinc 606 Dec 20 11:05 RTE.tar.gz -rwxr-xr-x 1 boinc boinc 8356 Dec 20 11:05 run_atlas -rw-r--r-- 1 boinc boinc 604 Dec 20 11:10 runtime_log -rw-r--r-- 1 boinc boinc 10380 Dec 20 11:10 runtime_log.err drwxrwx--x 2 boinc boinc 7 Dec 20 11:10 shared -rw-r--r-- 1 boinc boinc 14361 Dec 20 11:05 start_atlas.sh -rw------- 1 boinc boinc 19 Dec 20 11:05 START_TIME_4184709929 -rw------- 1 boinc boinc 1 Dec 20 11:05 STATUSCODE -rw-r--r-- 1 boinc boinc 9877 Dec 20 11:10 stderr.txt -rw------- 1 boinc boinc 46 Dec 20 11:08 workdir_size-4184709929.json -rw-r--r-- 1 boinc boinc 100 Dec 20 11:05 wrapper_26015_x86_64-pc-linux-gnu -rw-r--r-- 1 boinc boinc 23 Dec 20 11:10 wrapper_checkpoint.txt -rw------- 1 boinc boinc 494 Dec 20 11:10 zR4KDm7F7ttnlyackoJh5iwnABFKDmABFKDmX6hbDmABFKDmOMowfm.diag running start_atlas return value is 0 Parent exit 0 child process exit 0 11:10:31 (4398): run_atlas exited; CPU time 124.469259 11:10:31 (4398): called boinc_finish(0) </stderr_txt> ]]>
©2025 CERN