Message boards : ATLAS application : ATLAS native version 2.81
Message board moderation

To post messages, you must log in.

AuthorMessage
David Cameron
Project administrator
Project developer
Project scientist

Send message
Joined: 13 May 14
Posts: 387
Credit: 15,314,184
RAC: 0
Message 41374 - Posted: 27 Jan 2020, 11:20:57 UTC

With 2.80 there were problems on many hosts running CentOS7 without Singularity. Version 2.81 now requires Singularity for all hosts. Please let us know if there are any other problems.

As a reminder, this new version doesn't require python any more.
ID: 41374 · Report as offensive     Reply Quote
computezrmle
Volunteer moderator
Volunteer developer
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 15 Jun 08
Posts: 2386
Credit: 223,034,941
RAC: 136,683
Message 41407 - Posted: 27 Jan 2020, 22:02:10 UTC - in response to Message 41374.  

... any other problems.

ATLAS now writes a messed and incomplete stderr.txt.
Example:
https://lhcathome.cern.ch/lhcathome/result.php?resultid=260611424
ID: 41407 · Report as offensive     Reply Quote
David Cameron
Project administrator
Project developer
Project scientist

Send message
Joined: 13 May 14
Posts: 387
Credit: 15,314,184
RAC: 0
Message 41417 - Posted: 28 Jan 2020, 19:45:06 UTC - in response to Message 41407.  

Strange.. it looks like the output is truncated. Does it happen with all your tasks? This one produced a valid HITS file so the task did finish properly. Maybe you can attach a "tail -f" to a running task's stderr.txt to see if the full stderr is produced, to see whether it's the upload at the end of the task that is truncating it.
ID: 41417 · Report as offensive     Reply Quote
computezrmle
Volunteer moderator
Volunteer developer
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 15 Jun 08
Posts: 2386
Credit: 223,034,941
RAC: 136,683
Message 41420 - Posted: 29 Jan 2020, 13:14:52 UTC

Logs from v2.73 (taken from different hosts) look like this:
https://lhcathome.cern.ch/lhcathome/result.php?resultid=260556228
https://lhcathome.cern.ch/lhcathome/result.php?resultid=260504032

https://lhcathome.cern.ch/lhcathome/result.php?resultid=260565904
https://lhcathome.cern.ch/lhcathome/result.php?resultid=260565904

https://lhcathome.cern.ch/lhcathome/result.php?resultid=260566471
https://lhcathome.cern.ch/lhcathome/result.php?resultid=260546480



Same hosts but now the logs from v2.81:
https://lhcathome.cern.ch/lhcathome/result.php?resultid=260644836
https://lhcathome.cern.ch/lhcathome/result.php?resultid=260676781

https://lhcathome.cern.ch/lhcathome/result.php?resultid=260582954
https://lhcathome.cern.ch/lhcathome/result.php?resultid=260566230

https://lhcathome.cern.ch/lhcathome/result.php?resultid=260601877
https://lhcathome.cern.ch/lhcathome/result.php?resultid=260602160


Looks like it happens also on other user's hosts (including David's and that from Agile Boincers):
https://lhcathome.cern.ch/lhcathome/result.php?resultid=260806947
https://lhcathome.cern.ch/lhcathome/result.php?resultid=260803398
https://lhcathome.cern.ch/lhcathome/result.php?resultid=260817739


Remarkable:
The call to "boinc_finish()" is no longer logged at the end although it should be the very last function to be called.



Tailed one of my tasks.
https://lhcathome.cern.ch/lhcathome/result.php?resultid=260805920
01:35:56 (35863): wrapper (7.7.26015): starting
01:35:56 (35863): wrapper: running run_atlas (--nthreads 1)
Mi 29. Jan 01:35:56 CET 2020: Arguments: --nthreads 1
Mi 29. Jan 01:35:56 CET 2020: Threads: 1
Mi 29. Jan 01:35:56 CET 2020: Checking for CVMFS
Mi 29. Jan 01:35:57 CET 2020: Probing /cvmfs/atlas.cern.ch... OK
Mi 29. Jan 01:35:57 CET 2020: Probing /cvmfs/atlas-condb.cern.ch... OK
Mi 29. Jan 01:35:58 CET 2020: Probing /cvmfs/grid.cern.ch... OK
Mi 29. Jan 01:35:58 CET 2020: Probing /cvmfs/cernvm-prod.cern.ch... OK
Mi 29. Jan 01:35:59 CET 2020: Probing /cvmfs/sft.cern.ch... OK
Mi 29. Jan 01:36:00 CET 2020: Probing /cvmfs/alice.cern.ch... OK
Mi 29. Jan 01:36:01 CET 2020: VERSION PID UPTIME(M) MEM(K) REVISION EXPIRES(M) NOCATALOGS CACHEUSE(K) CACHEMAX(K) NOFDUSE NOFDMAX NOIOERR NOOPEN HITRATE(%) RX(K) SPEED(K/S) HOST PROXY ONLINE
Mi 29. Jan 01:36:01 CET 2020: 2.7.0.0 18751 7991 53804 59736 2 61 6690448 7077889 4725 65024 0 3745861 99.9594 3272721 11952 http://s1cern-cvmfs.openhtc.io/cvmfs/atlas.cern.ch http://<IP_censored_by_volunteer/>:3128 1
Mi 29. Jan 01:36:01 CET 2020: CVMFS is ok
Mi 29. Jan 01:36:01 CET 2020: Using singularity image /cvmfs/atlas.cern.ch/repo/containers/images/singularity/x86_64-centos7.img
Mi 29. Jan 01:36:01 CET 2020: Checking for singularity binary...
Mi 29. Jan 01:36:01 CET 2020: which: no singularity in (/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin)
Mi 29. Jan 01:36:01 CET 2020: Singularity is not installed, using version from CVMFS
Mi 29. Jan 01:36:01 CET 2020: Checking singularity works with /cvmfs/atlas.cern.ch/repo/containers/sw/singularity/x86_64-el7/current/bin/singularity exec -B /cvmfs /cvmfs/atlas.cern.ch/repo/containers/images/singularity/x86_64-centos7.img hostname
Mi 29. Jan 01:36:12 CET 2020: INFO:  Convert SIF file to sandbox... <hostname_censored_by_volunteer/> INFO:  Cleaning up image...
Mi 29. Jan 01:36:12 CET 2020: Singularity works
Mi 29. Jan 01:36:13 CET 2020: Starting ATLAS job with PandaID=4622986354
Mi 29. Jan 01:36:13 CET 2020: Running command: /cvmfs/atlas.cern.ch/repo/containers/sw/singularity/x86_64-el7/current/bin/singularity exec --pwd /home/boinc4/BOINC_ATLAS/slots/2 -B /cvmfs,/home /cvmfs/atlas.cern.ch/repo/containers/images/singularity/x86_64-centos7.img sh start_atlas.sh
Mi 29. Jan 09:07:12 CET 2020:  *** The last 200 lines of the pilot log: ***
Mi 29. Jan 09:07:12 CET 2020:         "preExe": {
Mi 29. Jan 09:07:12 CET 2020:           "cpuTime": 1, 
Mi 29. Jan 09:07:12 CET 2020:           "wallTime": 28
Mi 29. Jan 09:07:12 CET 2020:         }, 
Mi 29. Jan 09:07:12 CET 2020:         "total": {
Mi 29. Jan 09:07:12 CET 2020:           "cpuTime": 23998, 
Mi 29. Jan 09:07:12 CET 2020:           "wallTime": 26222
Mi 29. Jan 09:07:12 CET 2020:         }, 
Mi 29. Jan 09:07:12 CET 2020:         "validation": {
Mi 29. Jan 09:07:12 CET 2020:           "cpuTime": 0, 
Mi 29. Jan 09:07:12 CET 2020:           "wallTime": 2
Mi 29. Jan 09:07:12 CET 2020:         }, 
Mi 29. Jan 09:07:12 CET 2020:         "wallTime": 26190
Mi 29. Jan 09:07:12 CET 2020:       }
Mi 29. Jan 09:07:12 CET 2020:     }, 
Mi 29. Jan 09:07:12 CET 2020:     "machine": {
Mi 29. Jan 09:07:12 CET 2020:       "cpu_family": "23", 
Mi 29. Jan 09:07:12 CET 2020:       "linux_distribution": [
Mi 29. Jan 09:07:12 CET 2020:         "CentOS Linux", 
Mi 29. Jan 09:07:12 CET 2020:         "7.6.1810", 
Mi 29. Jan 09:07:12 CET 2020:         "Core"
Mi 29. Jan 09:07:12 CET 2020:       ], 
Mi 29. Jan 09:07:12 CET 2020:       "model": "8", 
Mi 29. Jan 09:07:12 CET 2020:       "model_name": "AMD Ryzen Threadripper 2950X 16-Core Processor", 
Mi 29. Jan 09:07:12 CET 2020:       "node": "<hostname_censored_by_volunteer/>", 
Mi 29. Jan 09:07:12 CET 2020:       "platform": "Linux-4.12.14-lp151.28.36-default-x86_64-with-centos-7.6.1810-Core"
Mi 29. Jan 09:07:12 CET 2020:     }, 
Mi 29. Jan 09:07:13 CET 2020:     "transform": {
Mi 29. Jan 09:07:13 CET 2020:       "cpuEfficiency": 0.9087, 
Mi 29. Jan 09:07:13 CET 2020:       "cpuPWEfficiency": 0.9087, 
Mi 29. Jan 09:07:13 CET 2020:       "cpuTime": 7, 
09:07:13 (35863): run_atlas exited; CPU time 24289.052160
09:07:13 (35863): called boinc_finish(0)
Mi 29. Jan 09:07:13 CET 2020:       "cpuTimeTotal": 23998, 
Mi 29. Jan 09:07:13 CET 2020:       "externalCpuTime": 118, 
Mi 29. Jan 09:07:13 CET 2020:       "processedEvents": 200, 
Mi 29. Jan 09:07:13 CET 2020:       "trfPredata": null, 
Mi 29. Jan 09:07:13 CET 2020:       "wallTime": 26289
Mi 29. Jan 09:07:13 CET 2020:     }
Mi 29. Jan 09:07:13 CET 2020:   }
Mi 29. Jan 09:07:13 CET 2020: }
Mi 29. Jan 09:07:13 CET 2020: 2020-01-29 08:06:13,096 | DEBUG    | queue_monitor       | pilot.util.auxiliary.4622986354  | update_server             | xml:will send fileinfo
Mi 29. Jan 09:07:13 CET 2020: 2020-01-29 08:06:13,096 | DEBUG    | queue_monitor       | pilot.control.job                | get_proper_state          | state=finished
Mi 29. Jan 09:07:13 CET 2020: 2020-01-29 08:06:13,096 | DEBUG    | queue_monitor       | pilot.control.job                | get_proper_state          | serverstate=running
Mi 29. Jan 09:07:13 CET 2020: 2020-01-29 08:06:13,096 | DEBUG    | queue_monitor       | pilot.control.job                | get_proper_state          | serverstate=finished
Mi 29. Jan 09:07:13 CET 2020: 2020-01-29 08:06:13,096 | INFO     | queue_monitor       | pilot.control.job.4622986354     | send_state                | pilot will not update the server (heartbeat message will be written to file)
Mi 29. Jan 09:07:13 CET 2020: 2020-01-29 08:06:13,096 | INFO     | queue_monitor       | pilot.control.job.4622986354     | send_state                | job 4622986354 has finished - writing final server update
Mi 29. Jan 09:07:13 CET 2020: 2020-01-29 08:06:13,096 | DEBUG    | queue_monitor       | pilot.control.job.4622986354     | get_data_structure        | building data structure to be sent to server with heartbeat
Mi 29. Jan 09:07:13 CET 2020: 2020-01-29 08:06:13,097 | INFO     | queue_monitor       | pilot.util.auxiliary.4622986354  | get_job_metrics           | will not add max space = -244901878 B to job metrics
Mi 29. Jan 09:07:13 CET 2020: 2020-01-29 08:06:13,104 | DEBUG    | queue_monitor       | pilot.api.analytics              | get_fitted_data           | removing tails from data to be fitted
Mi 29. Jan 09:07:13 CET 2020: 2020-01-29 08:06:13,104 | INFO     | queue_monitor       | pilot.api.analytics              | get_fitted_data           | fitting pss+swap vs Time
Mi 29. Jan 09:07:13 CET 2020: 2020-01-29 08:06:13,105 | INFO     | queue_monitor       | pilot.api.analytics              | get_fitted_data           | current memory leak: 7.89 B/s (using 426 data points, chi2=9404948)
Mi 29. Jan 09:07:13 CET 2020: 2020-01-29 08:06:13,106 | DEBUG    | queue_monitor       | pilot.util.auxiliary.4622986354  | get_job_metrics           | job metrics="coreCount=1 actualCoreCount=1 nEvents=200 leak=7.89 chi2=9404948"
Mi 29. Jan 09:07:13 CET 2020: 2020-01-29 08:06:13,106 | INFO     | queue_monitor       | pilot.control.job.4622986354     | get_data_structure        | total number of processed events: 200 (read)
Mi 29. Jan 09:07:13 CET 2020: 2020-01-29 08:06:13,107 | INFO     | queue_monitor       | pilot.user.atlas.utilities       | get_memory_values         | using path: /home/boinc4/BOINC_ATLAS/slots/2/PanDA_Pilot-4622986354/memory_monitor_summary.json (trf name=prmon)
Mi 29. Jan 09:07:13 CET 2020: 2020-01-29 08:06:13,109 | DEBUG    | queue_monitor       | pilot.user.atlas.utilities       | get_memory_monitor_info   | summary_dictionary={'Max': {'rx_packets': 3964589, 'nprocs': 10, 'nthreads': 1, 'rx_bytes': 5439601189, 'wtime': 26661, 'rss': 2149436, 'write_bytes': 0, 'vmem': 3364108, 'read_bytes': 0, 'stime': 98, 'tx_bytes': 6554250252, 'pss': 2088708, 'wchar': 0, 'rchar': 0, 'tx_packets': 4788198, 'swap': 0, 'utime': 24157}, 'Avg': {'write_bytes': 0, 'nprocs': 5, 'nthreads': 0, 'rx_bytes': 204021, 'rx_packets': 148, 'vmem': 3269241, 'read_bytes': 0, 'swap': 0, 'tx_bytes': 245828, 'pss': 1920860, 'wchar': 0, 'rchar': 0, 'tx_packets': 179, 'rss': 1975430}}
Mi 29. Jan 09:07:13 CET 2020: 2020-01-29 08:06:13,109 | INFO     | queue_monitor       | pilot.user.atlas.utilities       | get_memory_monitor_info   | extracted standard info from prmon json
Mi 29. Jan 09:07:13 CET 2020: 2020-01-29 08:06:13,109 | INFO     | queue_monitor       | pilot.user.atlas.utilities       | get_memory_monitor_info   | extracted standard memory fields from prmon json
Mi 29. Jan 09:07:13 CET 2020: 2020-01-29 08:06:13,109 | INFO     | queue_monitor       | pilot.util.auxiliary.4622986354  | timing_report             | ..............................
Mi 29. Jan 09:07:13 CET 2020: 2020-01-29 08:06:13,109 | INFO     | queue_monitor       | pilot.util.auxiliary.4622986354  | timing_report             | . Timing measurements:
Mi 29. Jan 09:07:13 CET 2020: 2020-01-29 08:06:13,109 | INFO     | queue_monitor       | pilot.util.auxiliary.4622986354  | timing_report             | . get job = 0 s
Mi 29. Jan 09:07:13 CET 2020: 2020-01-29 08:06:13,109 | INFO     | queue_monitor       | pilot.util.auxiliary.4622986354  | timing_report             | . initial setup = 5 s
Mi 29. Jan 09:07:13 CET 2020: 2020-01-29 08:06:13,109 | INFO     | queue_monitor       | pilot.util.auxiliary.4622986354  | timing_report             | . payload setup = 0 s
Mi 29. Jan 09:07:13 CET 2020: 2020-01-29 08:06:13,109 | INFO     | queue_monitor       | pilot.util.auxiliary.4622986354  | timing_report             | . total setup = 5 s
Mi 29. Jan 09:07:13 CET 2020: 2020-01-29 08:06:13,109 | INFO     | queue_monitor       | pilot.util.auxiliary.4622986354  | timing_report             | . stage-in = 1 s
Mi 29. Jan 09:07:13 CET 2020: 2020-01-29 08:06:13,109 | INFO     | queue_monitor       | pilot.util.auxiliary.4622986354  | timing_report             | . payload execution = 26884 s
Mi 29. Jan 09:07:13 CET 2020: 2020-01-29 08:06:13,110 | INFO     | queue_monitor       | pilot.util.auxiliary.4622986354  | timing_report             | . stage-out = 4 s
Mi 29. Jan 09:07:13 CET 2020: 2020-01-29 08:06:13,110 | INFO     | queue_monitor       | pilot.util.auxiliary.4622986354  | timing_report             | ..............................
Mi 29. Jan 09:07:13 CET 2020: 2020-01-29 08:06:13,111 | DEBUG    | queue_monitor       | pilot.control.job.4622986354     | send_state                | wrote heartbeat to file /home/boinc4/BOINC_ATLAS/slots/2/heartbeat.json
Mi 29. Jan 09:07:13 CET 2020: 2020-01-29 08:06:13,111 | DEBUG    | queue_monitor       | pilot.control.job                | queue_monitor             | job 4622986354 was dequeued from the monitored payloads queue
Mi 29. Jan 09:07:13 CET 2020: 2020-01-29 08:06:13,111 | DEBUG    | queue_monitor       | pilot.control.job                | queue_monitor             | tmp job object deleted
Mi 29. Jan 09:07:13 CET 2020: 2020-01-29 08:06:13,495 | INFO     | retrieve            | pilot.util.auxiliary.4622986354  | make_job_report           | 
Mi 29. Jan 09:07:14 CET 2020: 2020-01-29 08:06:13,495 | INFO     | retrieve            | pilot.util.auxiliary.4622986354  | make_job_report           | job summary report
Mi 29. Jan 09:07:14 CET 2020: 2020-01-29 08:06:13,495 | INFO     | retrieve            | pilot.util.auxiliary.4622986354  | make_job_report           | --------------------------------------------------
Mi 29. Jan 09:07:14 CET 2020: 2020-01-29 08:06:13,495 | INFO     | retrieve            | pilot.util.auxiliary.4622986354  | make_job_report           | PanDA job id: 4622986354
Mi 29. Jan 09:07:14 CET 2020: 2020-01-29 08:06:13,495 | INFO     | retrieve            | pilot.util.auxiliary.4622986354  | make_job_report           | task id: 20360903
Mi 29. Jan 09:07:14 CET 2020: 2020-01-29 08:06:13,495 | INFO     | retrieve            | pilot.util.auxiliary.4622986354  | make_job_report           | errors: (none)
Mi 29. Jan 09:07:14 CET 2020: 2020-01-29 08:06:13,495 | INFO     | retrieve            | pilot.util.auxiliary.4622986354  | make_job_report           | status: LOG_TRANSFER = DONE 
Mi 29. Jan 09:07:14 CET 2020: 2020-01-29 08:06:13,496 | INFO     | retrieve            | pilot.util.auxiliary.4622986354  | make_job_report           | pilot state: finished 
Mi 29. Jan 09:07:14 CET 2020: 2020-01-29 08:06:13,496 | INFO     | retrieve            | pilot.util.auxiliary.4622986354  | make_job_report           | transexitcode: 0
Mi 29. Jan 09:07:14 CET 2020: 2020-01-29 08:06:13,496 | INFO     | retrieve            | pilot.util.auxiliary.4622986354  | make_job_report           | exeerrorcode: 0
Mi 29. Jan 09:07:14 CET 2020: 2020-01-29 08:06:13,496 | INFO     | retrieve            | pilot.util.auxiliary.4622986354  | make_job_report           | exeerrordiag: 
Mi 29. Jan 09:07:14 CET 2020: 2020-01-29 08:06:13,496 | INFO     | retrieve            | pilot.util.auxiliary.4622986354  | make_job_report           | exitcode: 0
Mi 29. Jan 09:07:14 CET 2020: 2020-01-29 08:06:13,496 | INFO     | retrieve            | pilot.util.auxiliary.4622986354  | make_job_report           | exitmsg: OK
Mi 29. Jan 09:07:14 CET 2020: 2020-01-29 08:06:13,496 | INFO     | retrieve            | pilot.util.auxiliary.4622986354  | make_job_report           | cpuconsumptiontime: 24352 s
Mi 29. Jan 09:07:14 CET 2020: 2020-01-29 08:06:13,496 | INFO     | retrieve            | pilot.util.auxiliary.4622986354  | make_job_report           | nevents: 200
Mi 29. Jan 09:07:14 CET 2020: 2020-01-29 08:06:13,496 | INFO     | retrieve            | pilot.util.auxiliary.4622986354  | make_job_report           | neventsw: 0
Mi 29. Jan 09:07:14 CET 2020: 2020-01-29 08:06:13,496 | INFO     | retrieve            | pilot.util.auxiliary.4622986354  | make_job_report           | pid: 65308
Mi 29. Jan 09:07:14 CET 2020: 2020-01-29 08:06:13,496 | INFO     | retrieve            | pilot.util.auxiliary.4622986354  | make_job_report           | pgrp: 65308
Mi 29. Jan 09:07:14 CET 2020: 2020-01-29 08:06:13,496 | INFO     | retrieve            | pilot.util.auxiliary.4622986354  | make_job_report           | corecount: 1
Mi 29. Jan 09:07:14 CET 2020: 2020-01-29 08:06:13,497 | INFO     | retrieve            | pilot.util.auxiliary.4622986354  | make_job_report           | event service: False
Mi 29. Jan 09:07:14 CET 2020: 2020-01-29 08:06:13,497 | INFO     | retrieve            | pilot.util.auxiliary.4622986354  | make_job_report           | --------------------------------------------------
Mi 29. Jan 09:07:14 CET 2020: 2020-01-29 08:06:13,497 | INFO     | retrieve            | pilot.util.auxiliary.4622986354  | make_job_report           | 
Mi 29. Jan 09:07:14 CET 2020: 2020-01-29 08:06:13,497 | INFO     | retrieve            | pilot.util.queuehandling         | queue_report              | queue jobs has 0 job(s)
Mi 29. Jan 09:07:14 CET 2020: 2020-01-29 08:06:13,497 | INFO     | retrieve            | pilot.util.queuehandling         | queue_report              | queue payloads has 0 job(s)
Mi 29. Jan 09:07:14 CET 2020: 2020-01-29 08:06:13,497 | INFO     | retrieve            | pilot.util.queuehandling         | queue_report              | queue data_in has 0 job(s)
Mi 29. Jan 09:07:14 CET 2020: 2020-01-29 08:06:13,497 | INFO     | retrieve            | pilot.util.queuehandling         | queue_report              | queue data_out has 0 job(s)
Mi 29. Jan 09:07:14 CET 2020: 2020-01-29 08:06:13,497 | INFO     | retrieve            | pilot.util.queuehandling         | queue_report              | queue current_data_in has 0 job(s)
Mi 29. Jan 09:07:14 CET 2020: 2020-01-29 08:06:13,497 | INFO     | retrieve            | pilot.util.queuehandling         | queue_report              | queue validated_jobs has 0 job(s)
Mi 29. Jan 09:07:14 CET 2020: 2020-01-29 08:06:13,497 | INFO     | retrieve            | pilot.util.queuehandling         | queue_report              | queue validated_payloads has 0 job(s)
Mi 29. Jan 09:07:14 CET 2020: 2020-01-29 08:06:13,497 | INFO     | retrieve            | pilot.util.queuehandling         | queue_report              | queue monitored_payloads has 0 job(s)
Mi 29. Jan 09:07:14 CET 2020: 2020-01-29 08:06:13,498 | INFO     | retrieve            | pilot.util.queuehandling         | queue_report              | queue finished_jobs has 0 job(s)
Mi 29. Jan 09:07:14 CET 2020: 2020-01-29 08:06:13,498 | INFO     | retrieve            | pilot.util.queuehandling         | queue_report              | queue finished_payloads has 0 job(s)
Mi 29. Jan 09:07:14 CET 2020: 2020-01-29 08:06:13,498 | INFO     | retrieve            | pilot.util.queuehandling         | queue_report              | queue finished_data_in has 1 job(s)
Mi 29. Jan 09:07:14 CET 2020: 2020-01-29 08:06:13,498 | INFO     | retrieve            | pilot.util.queuehandling         | queue_report              | queue finished_data_out has 0 job(s)
Mi 29. Jan 09:07:14 CET 2020: 2020-01-29 08:06:13,498 | INFO     | retrieve            | pilot.util.queuehandling         | queue_report              | queue failed_jobs has 0 job(s)
Mi 29. Jan 09:07:14 CET 2020: 2020-01-29 08:06:13,498 | INFO     | retrieve            | pilot.util.queuehandling         | queue_report              | queue failed_payloads has 0 job(s)
Mi 29. Jan 09:07:14 CET 2020: 2020-01-29 08:06:13,498 | INFO     | retrieve            | pilot.util.queuehandling         | queue_report              | queue failed_data_in has 0 job(s)
Mi 29. Jan 09:07:14 CET 2020: 2020-01-29 08:06:13,498 | INFO     | retrieve            | pilot.util.queuehandling         | queue_report              | queue failed_data_out has 0 job(s)
Mi 29. Jan 09:07:14 CET 2020: 2020-01-29 08:06:13,498 | INFO     | retrieve            | pilot.util.queuehandling         | queue_report              | queue completed_jobs has 0 job(s)
Mi 29. Jan 09:07:14 CET 2020: 2020-01-29 08:06:13,498 | INFO     | retrieve            | pilot.util.queuehandling         | queue_report              | queue completed_jobids has 1 job(s)
Mi 29. Jan 09:07:14 CET 2020: 2020-01-29 08:06:13,498 | INFO     | retrieve            | pilot.control.job.4622986354     | has_job_completed         | job 4622986354 has completed (purged errors)
Mi 29. Jan 09:07:14 CET 2020: 2020-01-29 08:06:13,499 | INFO     | retrieve            | pilot.util.processes             | cleanup                   | overall cleanup function is called
Mi 29. Jan 09:07:15 CET 2020: 2020-01-29 08:06:13,499 | DEBUG    | retrieve            | pilot.util.processes             | cleanup                   | work directory was removed: /home/boinc4/BOINC_ATLAS/slots/2/PanDA_Pilot-4622986354
Mi 29. Jan 09:07:15 CET 2020: 2020-01-29 08:06:14,511 | INFO     | retrieve            | pilot.info.jobdata               | collect_zombies           | --- collectZombieJob: --- 10, [65308]
Mi 29. Jan 09:07:15 CET 2020: 2020-01-29 08:06:14,511 | INFO     | retrieve            | pilot.info.jobdata               | collect_zombies           | zombie collector trying to kill pid 65308
Mi 29. Jan 09:07:15 CET 2020: 2020-01-29 08:06:14,511 | INFO     | retrieve            | pilot.info.jobdata               | collect_zombies           | harmless exception when collecting zombies: [Errno 10] No child processes
Mi 29. Jan 09:07:15 CET 2020: 2020-01-29 08:06:15,515 | INFO     | retrieve            | pilot.util.processes             | cleanup                   | collected zombie processes
Mi 29. Jan 09:07:15 CET 2020: 2020-01-29 08:06:15,515 | INFO     | retrieve            | pilot.util.processes             | cleanup                   | will now attempt to kill all subprocesses of pid=65308
Mi 29. Jan 09:07:15 CET 2020: 2020-01-29 08:06:15,823 | INFO     | retrieve            | pilot.util.processes             | kill_processes            | process IDs to be killed: [65308] (in reverse order)
Mi 29. Jan 09:07:15 CET 2020: 2020-01-29 08:06:15,941 | WARNING  | retrieve            | pilot.util.processes             | kill_processes            | found no corresponding commands to process id(s)
Mi 29. Jan 09:07:15 CET 2020: 2020-01-29 08:06:15,941 | INFO     | retrieve            | pilot.util.processes             | kill_orphans              | Do not look for orphan processes in BOINC jobs
Mi 29. Jan 09:07:15 CET 2020: 2020-01-29 08:06:15,941 | INFO     | retrieve            | pilot.control.job                | retrieve                  | ready for new job
Mi 29. Jan 09:07:15 CET 2020: 2020-01-29 08:06:15,941 | INFO     | retrieve            | root                             | retrieve                  | pilot has finished for previous job - re-establishing logging
Mi 29. Jan 09:07:15 CET 2020: mpi4py not found
Mi 29. Jan 09:07:15 CET 2020: 2020-01-29 08:06:15,944 | INFO     | retrieve            | pilot.util.auxiliary             | pilot_version_banner      | ****************************************
Mi 29. Jan 09:07:15 CET 2020: 2020-01-29 08:06:15,944 | INFO     | retrieve            | pilot.util.auxiliary             | pilot_version_banner      | ***  PanDA Pilot version 2.3.4 (12)  ***
Mi 29. Jan 09:07:15 CET 2020: 2020-01-29 08:06:15,944 | INFO     | retrieve            | pilot.util.auxiliary             | pilot_version_banner      | ****************************************
Mi 29. Jan 09:07:15 CET 2020: 2020-01-29 08:06:15,945 | INFO     | retrieve            | pilot.util.auxiliary             | pilot_version_banner      | 
Mi 29. Jan 09:07:15 CET 2020: 2020-01-29 08:06:15,946 | INFO     | retrieve            | pilot.util.auxiliary             | display_architecture_info | architecture information:
Mi 29. Jan 09:07:15 CET 2020: 2020-01-29 08:06:16,724 | INFO     | retrieve            | pilot.util.auxiliary             | display_architecture_info | 
Mi 29. Jan 09:07:15 CET 2020: LSB Version:	:core-4.1-amd64:core-4.1-noarch
Mi 29. Jan 09:07:15 CET 2020: Distributor ID:	CentOS
Mi 29. Jan 09:07:15 CET 2020: Description:	CentOS Linux release 7.6.1810 (Core) 
Mi 29. Jan 09:07:15 CET 2020: Release:	7.6.1810
Mi 29. Jan 09:07:15 CET 2020: Codename:	Core
Mi 29. Jan 09:07:15 CET 2020: 2020-01-29 08:06:16,724 | INFO     | retrieve            | pilot.util.auxiliary             | pilot_version_banner      | ****************************************
Mi 29. Jan 09:07:15 CET 2020: 2020-01-29 08:06:17,230 | DEBUG    | retrieve            | pilot.util.monitoring            | check_local_space         | checking local space on /home/boinc4/BOINC_ATLAS/slots/2
Mi 29. Jan 09:07:15 CET 2020: 2020-01-29 08:06:17,270 | INFO     | retrieve            | pilot.util.monitoring            | check_local_space         | sufficient remaining disk space (100103356416 B)
Mi 29. Jan 09:07:15 CET 2020: 2020-01-29 08:06:17,271 | WARNING  | retrieve            | pilot.control.job                | proceed_with_getjob       | since timefloor is set to 0, pilot was only allowed to run one job
Mi 29. Jan 09:07:15 CET 2020: 2020-01-29 08:06:17,271 | DEBUG    | retrieve            | pilot.control.job                | retrieve                  | [job] retrieve thread has finished
Mi 29. Jan 09:07:15 CET 2020: 2020-01-29 08:06:17,287 | DEBUG    | job                 | pilot.control.job                | control                   | job control ending since graceful_stop has been set
Mi 29. Jan 09:07:15 CET 2020: 2020-01-29 08:06:17,287 | DEBUG    | job                 | pilot.control.job                | control                   | [job] control thread has finished
Mi 29. Jan 09:07:15 CET 2020: 2020-01-29 08:06:17,313 | WARNING  | copytool_out        | pilot.util.common                | should_abort              | data:copytool_out:received graceful stop - abort after this iteration
Mi 29. Jan 09:07:15 CET 2020: 2020-01-29 08:06:17,316 | WARNING  | monitor             | pilot.control.monitor            | control                   | aborting monitor loop since graceful_stop has been set
Mi 29. Jan 09:07:15 CET 2020: 2020-01-29 08:06:17,316 | INFO     | monitor             | pilot.control.monitor            | control                   | [monitor] control thread has ended
Mi 29. Jan 09:07:15 CET 2020: 2020-01-29 08:06:17,371 | DEBUG    | MainThread          | pilot.workflow.generic           | run                       | thread count now at 14 threads
Mi 29. Jan 09:07:15 CET 2020: 2020-01-29 08:06:17,372 | DEBUG    | MainThread          | pilot.workflow.generic           | run                       | enumerate: [<_MainThread(MainThread, started 140543163242304)>, <ExcThread(queue_monitor, started 140542000232192)>, <ExcThread(queue_monitoring, started 140542570641152)>, <ExcThread(validate, started 140542992168704)>, <ExcThread(execute_payloads, started 140541991839488)>, <ExcThread(validate_post, started 140542595819264)>, <ExcThread(data, started 140542983776000)>, <ExcThread(copytool_in, started 140542553855744)>, <ExcThread(create_data_payload, started 140542950205184)>, <ExcThread(failed_post, started 140542579033856)>, <ExcThread(validate_pre, started 140542958597888)>, <ExcThread(job_monitor, started 140542562248448)>, <ExcThread(copytool_out, started 140542975383296)>, <ExcThread(payload, started 140542966990592)>]
Mi 29. Jan 09:07:15 CET 2020: 2020-01-29 08:06:17,488 | DEBUG    | create_data_payload | pilot.control.job                | create_data_payload       | [job] create_data_payload thread has finished
Mi 29. Jan 09:07:15 CET 2020: 2020-01-29 08:06:17,543 | WARNING  | queue_monitoring    | pilot.util.common                | should_abort              | data:queue_monitoring:received graceful stop - abort after this iteration
Mi 29. Jan 09:07:15 CET 2020: 2020-01-29 08:06:17,583 | INFO     | validate_pre        | pilot.control.payload            | validate_pre              | [payload] validate_pre thread has finished
Mi 29. Jan 09:07:15 CET 2020: 2020-01-29 08:06:17,671 | DEBUG    | copytool_in         | pilot.control.data               | copytool_in               | [data] copytool_in thread has finished
Mi 29. Jan 09:07:15 CET 2020: 2020-01-29 08:06:17,793 | INFO     | failed_post         | pilot.control.payload            | failed_post               | [payload] failed_post thread has finished
Mi 29. Jan 09:07:15 CET 2020: 2020-01-29 08:06:18,063 | DEBUG    | validate            | pilot.control.job                | validate                  | [job] validate thread has finished
Mi 29. Jan 09:07:15 CET 2020: 2020-01-29 08:06:18,155 | WARNING  | queue_monitor       | pilot.util.common                | should_abort              | job:queue_monitor:received graceful stop - abort after this iteration
Mi 29. Jan 09:07:15 CET 2020: 2020-01-29 08:06:18,155 | DEBUG    | queue_monitor       | pilot.control.job                | queue_monitor             | [job] queue monitor thread has finished
Mi 29. Jan 09:07:15 CET 2020: 2020-01-29 08:06:18,287 | DEBUG    | data                | pilot.control.data               | control                   | data control ending since graceful_stop has been set
Mi 29. Jan 09:07:15 CET 2020: 2020-01-29 08:06:18,287 | DEBUG    | data                | pilot.control.data               | control                   | [data] control thread has finished
Mi 29. Jan 09:07:15 CET 2020: 2020-01-29 08:06:18,318 | DEBUG    | copytool_out        | pilot.control.data               | copytool_out              | [data] copytool_out thread has finished
Mi 29. Jan 09:07:15 CET 2020: 2020-01-29 08:06:18,348 | INFO     | validate_post       | pilot.control.payload            | validate_post             | [payload] validate_post thread has finished
Mi 29. Jan 09:07:16 CET 2020: 2020-01-29 08:06:18,383 | DEBUG    | payload             | pilot.control.payload            | control                   | payload control ending since graceful_stop has been set
Mi 29. Jan 09:07:16 CET 2020: 2020-01-29 08:06:18,383 | DEBUG    | payload             | pilot.control.payload            | control                   | [payload] control thread has finished
Mi 29. Jan 09:07:16 CET 2020: 2020-01-29 08:06:18,387 | DEBUG    | MainThread          | pilot.workflow.generic           | run                       | thread count now at 4 threads
Mi 29. Jan 09:07:16 CET 2020: 2020-01-29 08:06:18,387 | DEBUG    | MainThread          | pilot.workflow.generic           | run                       | enumerate: [<_MainThread(MainThread, started 140543163242304)>, <ExcThread(queue_monitoring, started 140542570641152)>, <ExcThread(execute_payloads, started 140541991839488)>, <ExcThread(job_monitor, started 140542562248448)>]
Mi 29. Jan 09:07:16 CET 2020: 2020-01-29 08:06:18,563 | INFO     | execute_payloads    | pilot.control.payload            | execute_payloads          | [payload] execute_payloads thread has finished
Mi 29. Jan 09:07:16 CET 2020: 2020-01-29 08:06:19,403 | DEBUG    | MainThread          | pilot.workflow.generic           | run                       | thread count now at 3 threads
Mi 29. Jan 09:07:16 CET 2020: 2020-01-29 08:06:19,403 | DEBUG    | MainThread          | pilot.workflow.generic           | run                       | enumerate: [<_MainThread(MainThread, started 140543163242304)>, <ExcThread(queue_monitoring, started 140542570641152)>, <ExcThread(job_monitor, started 140542562248448)>]
Mi 29. Jan 09:07:16 CET 2020: 2020-01-29 08:06:20,546 | DEBUG    | queue_monitoring    | pilot.control.data               | queue_monitoring          | [data] queue_monitor thread has finished
Mi 29. Jan 09:07:16 CET 2020: 2020-01-29 08:06:21,423 | DEBUG    | MainThread          | pilot.workflow.generic           | run                       | thread count now at 2 threads
Mi 29. Jan 09:07:16 CET 2020: 2020-01-29 08:06:21,423 | DEBUG    | MainThread          | pilot.workflow.generic           | run                       | enumerate: [<_MainThread(MainThread, started 140543163242304)>, <ExcThread(job_monitor, started 140542562248448)>]
Mi 29. Jan 09:07:16 CET 2020: 2020-01-29 08:06:56,145 | WARNING  | job_monitor         | pilot.control.job                | check_job_monitor_waiting_time | no jobs in monitored_payloads queue (waited for 74 s)
Mi 29. Jan 09:07:16 CET 2020: 2020-01-29 08:06:56,145 | DEBUG    | job_monitor         | pilot.control.job                | job_monitor               | [job] job monitor thread has finished
Mi 29. Jan 09:07:16 CET 2020: 2020-01-29 08:06:56,679 | INFO     | MainThread          | pilot.workflow.generic           | run                       | end of generic workflow (traces error code: 0)
Mi 29. Jan 09:07:16 CET 2020: 2020-01-29 08:06:56,679 | INFO     | MainThread          | root                             | wrap_up                   | traces error code: 0
Mi 29. Jan 09:07:16 CET 2020: 2020-01-29 08:06:56,679 | INFO     | MainThread          | root                             | wrap_up                   | pilot has finished
Mi 29. Jan 09:07:16 CET 2020: 2020-01-29 08:06:57,354 [wrapper] ==== pilot stdout END ====
Mi 29. Jan 09:07:16 CET 2020: 2020-01-29 08:06:57,609 [wrapper] ==== wrapper stdout RESUME ====
Mi 29. Jan 09:07:16 CET 2020: 2020-01-29 08:06:58,639 [wrapper] Pilot exit status: 0
Mi 29. Jan 09:07:16 CET 2020: 2020-01-29 08:06:59,802 [wrapper] STATUSCODE: 0
Mi 29. Jan 09:07:16 CET 2020: 2020-01-29 08:06:59,989 [wrapper] apfmon messages muted
Mi 29. Jan 09:07:16 CET 2020: ---- find pandaIDs.out ----
Mi 29. Jan 09:07:16 CET 2020: total 60
Mi 29. Jan 09:07:16 CET 2020: -rw-------  1 boinc4 boinc 11357 Jul 25  2019 LICENSE
Mi 29. Jan 09:07:16 CET 2020: -rw-------  1 boinc4 boinc    20 Sep  9 13:04 MANIFEST.IN
Mi 29. Jan 09:07:16 CET 2020: -rw-------  1 boinc4 boinc     8 Dec 12 19:00 PILOTVERSION
Mi 29. Jan 09:07:16 CET 2020: -rw-------  1 boinc4 boinc  2212 Nov 14 11:01 README.md
Mi 29. Jan 09:07:16 CET 2020: -rw-------  1 boinc4 boinc   221 Jul 25  2019 TODO.md
Mi 29. Jan 09:07:16 CET 2020: -rw-------  1 boinc4 boinc    11 Jan 29 01:37 pandaIDs.out
Mi 29. Jan 09:07:16 CET 2020: drwx------ 14 boinc4 boinc   320 Jan 29 01:37 pilot
Mi 29. Jan 09:07:16 CET 2020: -rwx------  1 boinc4 boinc 21225 Dec 12 19:00 pilot.py
Mi 29. Jan 09:07:16 CET 2020: -rw-------  1 boinc4 boinc   766 Oct 10 16:01 setup.py
Mi 29. Jan 09:07:16 CET 2020: 
Mi 29. Jan 09:07:16 CET 2020: 2020-01-29 08:07:00,779 [wrapper] pandaIDs.out files:
Mi 29. Jan 09:07:16 CET 2020: -rw------- 1 boinc4 boinc 11 Jan 29 01:37 /home/boinc4/BOINC_ATLAS/slots/2/pilot2/pandaIDs.out
Mi 29. Jan 09:07:16 CET 2020: 2020-01-29 08:07:03,550 [wrapper] pandaIDs.out content:
Mi 29. Jan 09:07:16 CET 2020: 4622986354
Mi 29. Jan 09:07:16 CET 2020: 
Mi 29. Jan 09:07:16 CET 2020: 2020-01-29 08:07:04,906 [wrapper] Test setup, not cleaning
Mi 29. Jan 09:07:16 CET 2020: 2020-01-29 08:07:05,829 [wrapper] ==== wrapper stdout END ====
Mi 29. Jan 09:07:16 CET 2020: 2020-01-29 08:07:06,373 [wrapper] ==== wrapper stderr END ====
Mi 29. Jan 09:07:16 CET 2020: 2020-01-29 08:07:08,893 [wrapper] wrapper wrapperexiting ec=0, duration=27042
Mi 29. Jan 09:07:16 CET 2020: 2020-01-29 08:07:09,882 [wrapper] apfmon messages muted
Mi 29. Jan 09:07:16 CET 2020:  *** Error codes and diagnostics ***
Mi 29. Jan 09:07:16 CET 2020:     "exeErrorCode": 0,
Mi 29. Jan 09:07:16 CET 2020:     "exeErrorDiag": "",
Mi 29. Jan 09:07:16 CET 2020:     "pilotErrorCode": 0,
Mi 29. Jan 09:07:16 CET 2020:     "pilotErrorDiag": "",
Mi 29. Jan 09:07:16 CET 2020:  *** Listing of results directory ***
Mi 29. Jan 09:07:16 CET 2020: insgesamt 372420
Mi 29. Jan 09:07:16 CET 2020: -rw-r--r-- 1 boinc4 boinc    267260 28. Jan 22:24 pilot2.tar.gz
Mi 29. Jan 09:07:17 CET 2020: -rw-r--r-- 1 boinc4 boinc      4492 28. Jan 22:36 queuedata.json
Mi 29. Jan 09:07:17 CET 2020: -rwx------ 1 boinc4 boinc     16499 28. Jan 22:36 runpilot2-wrapper.sh
Mi 29. Jan 09:07:17 CET 2020: -rw-r--r-- 1 boinc4 boinc       100 29. Jan 01:35 wrapper_26015_x86_64-pc-linux-gnu
Mi 29. Jan 09:07:17 CET 2020: -rwxr-xr-x 1 boinc4 boinc      5432 29. Jan 01:35 run_atlas
Mi 29. Jan 09:07:17 CET 2020: -rw-r--r-- 1 boinc4 boinc       105 29. Jan 01:35 job.xml
Mi 29. Jan 09:07:17 CET 2020: drwxrwx--x 2 boinc4 boinc       120 29. Jan 01:35 shared
Mi 29. Jan 09:07:17 CET 2020: -rw-r--r-- 1 boinc4 boinc      6392 29. Jan 01:35 init_data.xml
Mi 29. Jan 09:07:17 CET 2020: -rw-r--r-- 1 boinc4 boinc      8192 29. Jan 01:35 boinc_mmap_file
Mi 29. Jan 09:07:17 CET 2020: -rw-r--r-- 1 boinc4 boinc         0 29. Jan 01:35 boinc_lockfile
Mi 29. Jan 09:07:17 CET 2020: -rw-r--r-- 1 boinc4 boinc     15218 29. Jan 01:36 start_atlas.sh
Mi 29. Jan 09:07:17 CET 2020: -rw-r--r-- 1 boinc4 boinc       858 29. Jan 01:36 RTE.tar.gz
Mi 29. Jan 09:07:17 CET 2020: -rw-r--r-- 1 boinc4 boinc    276437 29. Jan 01:36 input.tar.gz
Mi 29. Jan 09:07:17 CET 2020: -rw-r--r-- 1 boinc4 boinc 246141021 29. Jan 01:36 EVNT.19609587._000498.pool.root.1
Mi 29. Jan 09:07:17 CET 2020: -rw-r--r-- 1 boinc4 boinc      2903 29. Jan 01:36 pandaJob.out
Mi 29. Jan 09:07:17 CET 2020: drwxr-xr-x 3 boinc4 boinc        60 29. Jan 01:36 APPS
Mi 29. Jan 09:07:17 CET 2020: -rw------- 1 boinc4 boinc       397 29. Jan 01:36 setup.sh.local
Mi 29. Jan 09:07:17 CET 2020: -rw------- 1 boinc4 boinc   3671389 29. Jan 01:37 agis_schedconf.cvmfs.json
Mi 29. Jan 09:07:17 CET 2020: -rw------- 1 boinc4 boinc   7849349 29. Jan 01:37 agis_ddmendpoints.json
Mi 29. Jan 09:07:17 CET 2020: drwx------ 3 boinc4 boinc       300 29. Jan 01:37 pilot2
Mi 29. Jan 09:07:17 CET 2020: -rw------- 1 boinc4 boinc 119402527 29. Jan 09:04 HITS.20360903._025614.pool.root.1
Mi 29. Jan 09:07:17 CET 2020: -rw------- 1 boinc4 boinc       801 29. Jan 09:05 memory_monitor_summary.json
Mi 29. Jan 09:07:17 CET 2020: -rw------- 1 boinc4 boinc    313307 29. Jan 09:06 log.20360903._025614.job.log.tgz.1
Mi 29. Jan 09:07:17 CET 2020: -rw------- 1 boinc4 boinc     58912 29. Jan 09:06 heartbeat.json
Mi 29. Jan 09:07:17 CET 2020: -rw------- 1 boinc4 boinc      8586 29. Jan 09:06 pilotlog.txt
Mi 29. Jan 09:07:17 CET 2020: -rw------- 1 boinc4 boinc   1416718 29. Jan 09:07 log.20360903._025614.job.log.1
Mi 29. Jan 09:07:17 CET 2020: -rw------- 1 boinc4 boinc       463 29. Jan 09:07 output.list
Mi 29. Jan 09:07:17 CET 2020: -rw-r--r-- 1 boinc4 boinc      9372 29. Jan 09:07 runtime_log.err
Mi 29. Jan 09:07:17 CET 2020: -rw-r--r-- 1 boinc4 boinc       692 29. Jan 09:07 runtime_log
Mi 29. Jan 09:07:17 CET 2020: -rw------- 1 boinc4 boinc   1802240 29. Jan 09:07 result.tar.gz
Mi 29. Jan 09:07:17 CET 2020: -rw------- 1 boinc4 boinc       568 29. Jan 09:07 5jSKDm7hnGwn9Rq4apoT9bVoABFKDmABFKDmANOODmABFKDm30xRam.diag
Mi 29. Jan 09:07:17 CET 2020: -rw-r--r-- 1 boinc4 boinc      2336 29. Jan 09:07 stderr.txt
Mi 29. Jan 09:07:17 CET 2020: HITS file was successfully produced:
Mi 29. Jan 09:07:17 CET 2020: -rw------- 1 boinc4 boinc 119402527 29. Jan 09:04 shared/HITS.pool.root.1
Mi 29. Jan 09:07:17 CET 2020:  *** Contents of shared directory: ***
Mi 29. Jan 09:07:17 CET 2020: insgesamt 359036
Mi 29. Jan 09:07:17 CET 2020: -rw-r--r-- 1 boinc4 boinc 246141021 29. Jan 01:35 ATLAS.root_0
Mi 29. Jan 09:07:17 CET 2020: -rw-r--r-- 1 boinc4 boinc     15218 29. Jan 01:35 start_atlas.sh
Mi 29. Jan 09:07:17 CET 2020: -rw-r--r-- 1 boinc4 boinc       858 29. Jan 01:35 RTE.tar.gz
Mi 29. Jan 09:07:17 CET 2020: -rw-r--r-- 1 boinc4 boinc    276437 29. Jan 01:35 input.tar.gz
Mi 29. Jan 09:07:17 CET 2020: -rw------- 1 boinc4 boinc 119402527 29. Jan 09:04 HITS.pool.root.1
Mi 29. Jan 09:07:17 CET 2020: -rw------- 1 boinc4 boinc   1802240 29. Jan 09:07 result.tar.gz
ID: 41420 · Report as offensive     Reply Quote
David Cameron
Project administrator
Project developer
Project scientist

Send message
Joined: 13 May 14
Posts: 387
Credit: 15,314,184
RAC: 0
Message 41435 - Posted: 31 Jan 2020, 10:31:33 UTC - in response to Message 41420.  

I think this is caused by the way logging to stderr works in the new version, where stdout is redirected to a function. If the function is slow (e.g. because it calls "date" for every line) then it will write the output with some delay. The boinc_finish() messages on the other hand are written directly by the BOINC wrapper so appear immediately.

If someone knows a better way to redirect stdout to stderr with timestamps, the new wrapper is hosted in github and contributions are welcome :)

https://github.com/davidgcameron/boinc-scripts/blob/master/native/run_atlas.sh
ID: 41435 · Report as offensive     Reply Quote

Message boards : ATLAS application : ATLAS native version 2.81


©2024 CERN