Name p6QKDmcxSJ7nsSi4ap6QjLDmwznN0nGgGQJmkKkKDmlMBLDm6RZuGo_0
Workunit 231250704
Created 31 Mar 2025, 11:58:45 UTC
Sent 31 Mar 2025, 13:36:25 UTC
Report deadline 8 Apr 2025, 13:36:25 UTC
Received 11 Apr 2025, 14:27:12 UTC
Server state Over
Outcome Success
Client state Done
Exit status 0 (0x00000000)
Computer ID 10851963
Run time 1 hours 32 min 25 sec
CPU time 2 hours 34 min 36 sec
Validate state Task was reported too late to validate
Credit 0.00
Device peak FLOPS 12.05 GFLOPS
Application version ATLAS Simulation v3.01 (vbox64_mt_mcore_atlas)
windows_x86_64
Peak working set size 90.80 MB
Peak swap size 89.57 MB
Peak disk usage 1.88 GB

Stderr output

<core_client_version>8.0.2</core_client_version>
<![CDATA[
<stderr_txt>
.2
2025-03-31 15:37:11 (5508): Detected: VirtualBox VboxManage Interface (Version: 7.0.20)
2025-03-31 15:37:11 (5508): Successfully copied 'init_data.xml' to the shared directory.
2025-03-31 15:37:11 (5508): Create VM. (boinc_c2aac6a50a1fa0b4, slot#2)
2025-03-31 15:37:12 (5508): Setting Memory Size for VM. (4300MB)
2025-03-31 15:37:12 (5508): Setting CPU Count for VM. (3)
2025-03-31 15:37:12 (5508): Setting Chipset Options for VM.
2025-03-31 15:37:13 (5508): Setting Graphics Controller Options for VM.
2025-03-31 15:37:13 (5508): Setting Boot Options for VM.
2025-03-31 15:37:13 (5508): Setting Network Configuration for NAT.
2025-03-31 15:37:13 (5508): Enabling VM Network Access.
2025-03-31 15:37:14 (5508): Disabling USB Support for VM.
2025-03-31 15:37:14 (5508): Disabling COM Port Support for VM.
2025-03-31 15:37:14 (5508): Disabling LPT Port Support for VM.
2025-03-31 15:37:14 (5508): Disabling Audio Support for VM.
2025-03-31 15:37:15 (5508): Disabling Clipboard Support for VM.
2025-03-31 15:37:15 (5508): Disabling Drag and Drop Support for VM.
2025-03-31 15:37:15 (5508): Adding storage controller(s) to VM.
2025-03-31 15:37:15 (5508): Adding virtual disk drive to VM. (ATLAS_vbox_3.01_image.vdi)
2025-03-31 15:37:18 (5508): Adding VirtualBox Guest Additions to VM.
2025-03-31 15:37:19 (5508): Adding network bandwidth throttle group to VM. (Defaulting to 1024GB)
2025-03-31 15:37:19 (5508): forwarding host port 55350 to guest port 80
2025-03-31 15:37:19 (5508): Enabling remote desktop for VM.
2025-03-31 15:37:19 (5508): Required extension pack not installed, remote desktop not enabled.
2025-03-31 15:37:19 (5508): Enabling shared directory for VM.
2025-03-31 15:37:20 (5508): Starting VM using VBoxManage interface. (boinc_c2aac6a50a1fa0b4, slot#2)
2025-03-31 15:37:26 (5508): Successfully started VM. (PID = '7172')
2025-03-31 15:37:26 (5508): Reporting VM Process ID to BOINC.
2025-03-31 15:37:26 (5508): Guest Log: BIOS: VirtualBox 7.0.20
2025-03-31 15:37:26 (5508): Guest Log: CPUID EDX: 0x178bfbff
2025-03-31 15:37:26 (5508): Guest Log: BIOS: No PCI IDE controller, not probing IDE
2025-03-31 15:37:26 (5508): Guest Log: BIOS: AHCI 0-P#0: PCHS=16383/16/63 LCHS=1024/255/63 0x0000000002800000 sectors
2025-03-31 15:37:26 (5508): VM state change detected. (old = 'poweredoff', new = 'running')
2025-03-31 15:37:26 (5508): Detected: Web Application Enabled (http://localhost:55350)
2025-03-31 15:37:26 (5508): Preference change detected
2025-03-31 15:37:26 (5508): Setting CPU throttle for VM. (100%)
2025-03-31 15:37:26 (5508): Setting checkpoint interval to 900 seconds. (Higher value of (Preference: 60 seconds) or (Vbox_job.xml: 900 seconds))
2025-03-31 15:37:28 (5508): Guest Log: BIOS: Boot : bseqnr=1, bootseq=0032
2025-03-31 15:37:28 (5508): Guest Log: BIOS: Booting from Hard Disk...
2025-03-31 15:37:32 (5508): Guest Log: BIOS: KBD: unsupported int 16h function 03
2025-03-31 15:37:32 (5508): Guest Log: BIOS: AX=0305 BX=0000 CX=0000 DX=0000 
2025-03-31 15:37:32 (5508): Guest Log: int13_harddisk_ext: function 41, unmapped device for ELDL=81
2025-03-31 15:37:32 (5508): Guest Log: int13_harddisk: function 02, unmapped device for ELDL=81
2025-03-31 15:37:32 (5508): Guest Log: int13_harddisk_ext: function 41, unmapped device for ELDL=82
2025-03-31 15:37:32 (5508): Guest Log: int13_harddisk: function 02, unmapped device for ELDL=82
2025-03-31 15:37:32 (5508): Guest Log: int13_harddisk_ext: function 41, unmapped device for ELDL=83
2025-03-31 15:37:32 (5508): Guest Log: int13_harddisk: function 02, unmapped device for ELDL=83
2025-03-31 15:37:32 (5508): Guest Log: int13_harddisk_ext: function 41, unmapped device for ELDL=84
2025-03-31 15:37:32 (5508): Guest Log: int13_harddisk: function 02, unmapped device for ELDL=84
2025-03-31 15:37:32 (5508): Guest Log: int13_harddisk_ext: function 41, unmapped device for ELDL=85
2025-03-31 15:37:32 (5508): Guest Log: int13_harddisk: function 02, unmapped device for ELDL=85
2025-03-31 15:37:32 (5508): Guest Log: int13_harddisk_ext: function 41, unmapped device for ELDL=86
2025-03-31 15:37:32 (5508): Guest Log: int13_harddisk: function 02, unmapped device for ELDL=86
2025-03-31 15:37:32 (5508): Guest Log: int13_harddisk_ext: function 41, unmapped device for ELDL=87
2025-03-31 15:37:32 (5508): Guest Log: int13_harddisk: function 02, unmapped device for ELDL=87
2025-03-31 15:37:32 (5508): Guest Log: int13_harddisk_ext: function 41, unmapped device for ELDL=88
2025-03-31 15:37:32 (5508): Guest Log: int13_harddisk: function 02, unmapped device for ELDL=88
2025-03-31 15:37:32 (5508): Guest Log: int13_harddisk_ext: function 41, unmapped device for ELDL=89
2025-03-31 15:37:32 (5508): Guest Log: int13_harddisk: function 02, unmapped device for ELDL=89
2025-03-31 15:37:32 (5508): Guest Log: int13_harddisk_ext: function 41, unmapped device for ELDL=8a
2025-03-31 15:37:32 (5508): Guest Log: int13_harddisk: function 02, unmapped device for ELDL=8a
2025-03-31 15:37:32 (5508): Guest Log: int13_harddisk_ext: function 41, unmapped device for ELDL=8b
2025-03-31 15:37:32 (5508): Guest Log: int13_harddisk: function 02, unmapped device for ELDL=8b
2025-03-31 15:37:32 (5508): Guest Log: int13_harddisk_ext: function 41, unmapped device for ELDL=8c
2025-03-31 15:37:32 (5508): Guest Log: int13_harddisk: function 02, unmapped device for ELDL=8c
2025-03-31 15:37:32 (5508): Guest Log: int13_harddisk_ext: function 41, unmapped device for ELDL=8d
2025-03-31 15:37:32 (5508): Guest Log: int13_harddisk: function 02, unmapped device for ELDL=8d
2025-03-31 15:37:32 (5508): Guest Log: int13_harddisk_ext: function 41, unmapped device for ELDL=8e
2025-03-31 15:37:32 (5508): Guest Log: int13_harddisk: function 02, unmapped device for ELDL=8e
2025-03-31 15:37:32 (5508): Guest Log: int13_harddisk_ext: function 41, unmapped device for ELDL=8f
2025-03-31 15:37:32 (5508): Guest Log: int13_harddisk: function 02, unmapped device for ELDL=8f
2025-03-31 15:37:42 (5508): Guest Log: vgdrvHeartbeatInit: Setting up heartbeat to trigger every 2000 milliseconds
2025-03-31 15:37:42 (5508): Guest Log: vboxguest: misc device minor 58, IRQ 20, I/O port d020, MMIO at 00000000f0400000 (size 0x400000)
2025-03-31 15:37:59 (5508): Guest Log: VBoxService 5.2.32 r132073 (verbosity: 0) linux.amd64 (Jul 12 2019 10:32:28) release log
2025-03-31 15:37:59 (5508): Guest Log: 00:00:00.000146 main     Log opened 2025-03-31T15:37:58.207419000Z
2025-03-31 15:37:59 (5508): Guest Log: 00:00:00.000256 main     OS Product: Linux
2025-03-31 15:37:59 (5508): Guest Log: 00:00:00.000317 main     OS Release: 3.10.0-957.27.2.el7.x86_64
2025-03-31 15:37:59 (5508): Guest Log: 00:00:00.000358 main     OS Version: #1 SMP Mon Jul 29 17:46:05 UTC 2019
2025-03-31 15:37:59 (5508): Guest Log: 00:00:00.000400 main     Executable: /opt/VBoxGuestAdditions-5.2.32/sbin/VBoxService
2025-03-31 15:37:59 (5508): Guest Log: 00:00:00.000402 main     Process ID: 1283
2025-03-31 15:37:59 (5508): Guest Log: 00:00:00.000403 main     Package type: LINUX_64BITS_GENERIC
2025-03-31 15:37:59 (5508): Guest Log: 00:00:00.002125 main     5.2.32 r132073 started. Verbose level = 0
2025-03-31 15:38:00 (5508): Guest Log: [INFO] Probing /cvmfs/atlas.cern.ch...
2025-03-31 15:38:00 (5508): Guest Log: [INFO] Mounting shared directory
2025-03-31 15:38:00 (5508): Guest Log: [INFO] Checking for init_data.xml
2025-03-31 15:38:09 (5508): Guest Log: 00:00:10.006953 timesync vgsvcTimeSyncWorker: Radical guest time change: -7 188 657 769 000ns (GuestNow=1 743 428 289 554 663 000 ns GuestLast=1 743 435 478 212 432 000 ns fSetTimeLastLoop=true )
2025-03-31 15:38:10 (5508): Guest Log: [INFO] Probing /cvmfs/atlas.cern.ch... OK
2025-03-31 15:38:10 (5508): Guest Log: [INFO] Detected branch: prod
2025-03-31 15:38:10 (5508): Guest Log: This is the prod version of the ATLAS job wrapper
2025-03-31 15:38:10 (5508): Guest Log: Copying input files
2025-03-31 15:38:14 (5508): Guest Log: Copied input files into RunAtlas.
2025-03-31 15:38:14 (5508): Guest Log: This VM did not configure a local http proxy via BOINC.
2025-03-31 15:38:14 (5508): Guest Log: Small home clusters do not require a local http proxy but it is suggested if
2025-03-31 15:38:14 (5508): Guest Log: more than 10 cores throughout the same LAN segment are regularly running ATLAS like tasks.
2025-03-31 15:38:14 (5508): Guest Log: Further information can be found at the LHC@home message board.
2025-03-31 15:38:24 (5508): Guest Log: Running cvmfs_config stat atlas.cern.ch
2025-03-31 15:38:24 (5508): Guest Log: VERSION PID UPTIME(M) MEM(K) REVISION EXPIRES(M) NOCATALOGS CACHEUSE(K) CACHEMAX(K) NOFDUSE NOFDMAX NOIOERR NOOPEN HITRATE(%) RX(K) SPEED(K/S) HOST PROXY ONLINE
2025-03-31 15:38:24 (5508): Guest Log: 2.6.3.0 1577 0 32024 144328 4 1 3117321 4096000 0 65024 0 0 n/a 0 0 http://s1ihep-cvmfs.openhtc.io:8080/cvmfs/atlas.cern.ch DIRECT 1
2025-03-31 15:38:28 (5508): Guest Log: copied the webapp to /var/www
2025-03-31 15:38:28 (5508): Guest Log: ATHENA_PROC_NUMBER=3
2025-03-31 15:38:28 (5508): Guest Log: ATHENA_CORE_NUMBER=3
2025-03-31 15:38:33 (5508): Guest Log: Apptainer command /cvmfs/atlas.cern.ch/repo/containers/sw/apptainer/x86_64-el7/current/bin/apptainer exec -B /cvmfs,/data,/home/atlas/RunAtlas /cvmfs/atlas.cern.ch/repo/containers/fs/singularity/x86_64-centos7 
2025-03-31 15:38:33 (5508): Guest Log:  *** Starting ATLAS job. (PandaID=6581219873 taskID=43634882) ***
2025-03-31 15:40:12 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 15:40:32 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 15:49:54 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 15:50:14 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 16:01:05 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 16:01:26 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 16:05:27 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 16:05:47 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 16:05:57 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 16:06:17 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 16:06:37 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 16:07:17 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 16:07:27 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 16:07:47 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 16:12:28 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 16:12:48 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 16:16:09 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 16:16:29 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 16:25:12 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 16:25:32 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 16:25:42 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 16:26:02 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 16:26:12 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 16:26:32 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 16:26:42 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 16:27:02 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 16:27:12 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 16:27:32 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 16:27:42 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 16:28:02 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 16:28:12 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 16:28:32 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 16:28:42 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 16:29:02 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 16:29:12 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 16:29:32 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 16:29:42 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 16:30:02 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 16:30:13 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 16:30:33 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 16:30:43 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 16:31:03 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 16:31:13 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 16:31:33 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 16:31:43 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 16:32:03 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 16:32:13 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 16:32:34 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 16:32:44 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 16:33:04 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 16:33:14 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 16:33:34 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 16:33:44 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 16:34:04 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 16:34:14 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 16:34:34 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 16:34:44 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 16:35:04 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 16:35:14 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 16:35:34 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 16:35:44 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 16:36:04 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 16:36:14 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 16:36:34 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 16:36:44 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 16:37:04 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 16:37:14 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 16:37:34 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 16:37:44 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 16:38:04 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 16:38:14 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 16:38:35 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 16:38:45 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 16:39:05 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 16:39:15 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 16:39:35 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 16:39:45 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 16:40:05 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 16:40:15 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 16:40:35 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 16:40:45 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 16:41:05 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 16:41:15 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 16:41:35 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 16:41:45 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 16:42:05 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 16:42:15 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 16:42:36 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 16:42:46 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 16:43:06 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 16:43:16 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 16:43:36 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 16:43:46 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 16:44:06 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 16:44:16 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 16:44:36 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 16:44:46 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 16:45:06 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 16:45:16 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 16:45:36 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 16:45:46 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 16:46:06 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 16:46:16 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 16:46:36 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 16:46:46 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 16:47:06 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 16:47:16 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 16:47:37 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 16:47:47 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 16:48:07 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 16:48:17 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 16:48:37 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 16:48:47 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 16:49:07 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 16:49:17 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 16:49:37 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 16:49:48 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 16:50:08 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 16:50:18 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 16:50:38 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 16:50:48 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 16:51:08 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 16:51:18 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 16:51:38 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 16:51:48 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 16:52:08 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 16:52:18 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 16:52:38 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 16:52:48 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 16:53:08 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 16:53:18 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 16:53:38 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 16:53:48 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 16:54:08 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 16:54:18 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 16:54:38 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 16:54:48 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 16:55:08 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 16:55:18 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 16:55:38 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 16:55:48 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 16:56:08 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 16:56:18 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 16:56:38 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 16:56:48 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 16:57:08 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 16:57:18 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 16:57:38 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 16:57:58 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 16:58:18 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 16:58:39 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 16:58:59 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 16:59:19 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 16:59:39 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 16:59:49 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 17:00:09 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 17:00:19 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 17:00:39 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 17:00:49 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 17:01:09 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 17:01:19 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 17:01:39 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 17:01:49 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 17:02:09 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 17:02:19 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 17:02:39 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 17:02:49 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 17:03:09 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 17:03:19 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 17:03:39 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 17:03:49 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 17:04:09 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 17:04:19 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 17:04:39 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 17:04:49 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 17:05:09 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 17:05:19 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 17:05:39 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 17:05:49 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 17:06:09 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 17:06:19 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 17:06:39 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 17:06:49 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 17:07:09 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 17:07:19 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 17:07:39 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 17:07:49 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 17:08:09 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 17:08:19 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 17:08:39 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 17:08:49 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 17:09:09 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 17:09:19 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 17:09:39 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 17:09:50 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 17:10:10 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 17:10:30 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 17:10:50 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 17:11:10 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 17:11:30 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 17:11:40 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 17:12:00 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 17:12:10 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 17:12:30 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 17:12:50 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 17:13:10 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 17:13:30 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 17:13:50 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 17:14:00 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 17:14:20 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 17:14:30 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 17:14:50 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 17:15:00 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 17:15:20 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 17:15:30 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 17:15:50 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 17:16:00 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 17:16:20 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 17:16:30 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 17:16:50 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 17:17:00 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 17:17:20 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 17:17:40 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 17:18:00 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 17:18:21 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 17:18:41 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 17:18:51 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 17:19:11 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 17:19:21 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 17:19:41 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 17:19:51 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 17:20:12 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 17:20:32 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 17:20:52 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 17:21:12 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 17:21:32 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 17:21:52 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 17:22:12 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 17:22:32 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 17:22:52 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 17:23:02 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 17:23:22 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 17:23:42 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 17:24:02 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 17:24:22 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 17:24:42 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 17:25:02 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 17:25:22 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 17:25:32 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 17:25:52 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 17:26:12 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 17:26:32 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 17:26:52 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 17:27:12 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 17:27:22 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 17:27:42 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 17:28:02 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 17:28:22 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 17:28:42 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 17:29:02 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 17:29:22 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 17:29:42 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 17:30:03 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 17:30:23 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 17:30:33 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 17:30:53 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 17:31:13 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 17:31:33 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 17:31:53 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 17:32:13 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 17:32:23 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 17:32:43 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 17:32:53 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 17:33:13 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 17:33:23 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 17:33:43 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 17:34:03 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 17:34:23 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 17:34:43 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 17:35:04 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 17:35:24 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 17:35:44 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 17:36:04 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 17:36:24 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 17:36:44 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 17:37:04 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 17:37:15 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 17:37:35 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 17:37:55 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 17:38:15 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 17:38:36 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 17:38:56 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 17:39:16 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 17:39:36 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 17:39:56 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 17:40:16 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 17:40:36 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 17:40:56 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 17:41:16 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 17:41:36 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 17:41:56 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 17:42:16 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 17:42:36 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 17:42:56 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 17:43:16 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 17:43:36 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 17:43:56 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 17:44:16 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 17:44:36 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 17:44:56 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 17:45:16 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 17:45:36 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 17:45:56 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 17:46:16 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 17:46:36 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 17:46:56 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 17:47:17 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 17:47:37 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 17:47:57 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 17:48:17 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 17:48:37 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 17:48:57 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 17:49:17 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 17:49:37 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 17:49:57 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 17:50:17 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 17:50:37 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 17:50:57 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 17:51:17 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 17:51:37 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 17:51:57 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 17:52:17 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 17:52:37 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 17:52:57 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 17:53:17 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 17:53:37 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 17:53:57 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 17:54:17 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 17:54:37 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 17:54:57 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 17:55:17 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 17:55:37 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 17:55:57 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 17:56:17 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 17:56:37 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 17:56:57 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 17:57:17 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 17:57:37 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 17:57:47 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 17:58:08 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 17:58:18 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 17:58:38 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 17:58:48 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 17:59:08 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 17:59:19 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 17:59:38 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 17:59:49 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 18:00:08 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 18:00:19 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 18:00:38 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 18:00:49 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 18:01:08 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 18:01:19 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 18:01:38 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 18:01:49 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 18:02:08 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 18:02:19 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 18:02:39 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 18:02:49 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 18:03:09 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 18:03:29 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 18:03:49 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 18:03:59 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 18:04:19 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 18:04:30 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 18:04:50 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 18:05:00 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 18:05:20 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 18:05:40 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 18:06:00 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 18:06:10 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 18:06:30 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 18:06:40 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 18:07:00 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 18:07:10 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 18:07:30 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 18:07:50 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 18:08:10 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 18:08:20 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 18:08:40 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 18:08:50 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 18:09:10 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 18:09:20 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 18:09:41 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 18:09:51 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 18:10:11 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 18:10:31 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 18:10:51 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 18:11:01 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 18:11:21 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 18:11:31 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 18:11:52 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 18:12:12 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 18:12:32 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 18:12:42 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 18:13:02 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 18:13:12 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 18:13:32 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 18:13:42 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 18:14:02 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 18:14:12 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 18:14:32 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 18:14:42 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 18:15:02 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 18:15:12 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 18:15:32 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 18:15:42 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 18:16:02 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 18:16:12 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 18:16:32 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 18:16:42 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 18:17:02 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 18:17:12 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 18:17:32 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 18:17:42 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 18:18:03 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 18:18:13 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 18:18:33 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 18:18:43 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 18:19:03 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 18:19:13 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 18:19:33 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 18:19:43 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 18:20:03 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 18:20:13 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 18:20:33 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 18:20:43 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 18:21:03 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 18:21:13 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 18:21:33 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 18:21:44 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 18:22:04 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 18:22:14 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 18:22:34 (5508): VM state change detected. (old = 'paused', new = 'running')
2025-03-31 18:22:44 (5508): VM state change detected. (old = 'running', new = 'paused')
2025-03-31 18:23:02 (5508): Stopping VM.
2025-03-31 18:23:23 (5508): Successfully stopped VM.
2025-04-11 16:25:27 (13692): Detected: vboxwrapper 26206
2025-04-11 16:25:27 (13692): Detected: BOINC client v8.0.2
2025-04-11 16:25:28 (13692): Detected: VirtualBox VboxManage Interface (Version: 7.0.20)
2025-04-11 16:25:29 (13692): Guest Log: BIOS: VirtualBox 7.0.20
2025-04-11 16:25:29 (13692): Guest Log: CPUID EDX: 0x178bfbff
2025-04-11 16:25:29 (13692): Guest Log: BIOS: No PCI IDE controller, not probing IDE
2025-04-11 16:25:29 (13692): Guest Log: BIOS: AHCI 0-P#0: PCHS=16383/16/63 LCHS=1024/255/63 0x0000000002800000 sectors
2025-04-11 16:25:29 (13692): Guest Log: BIOS: Boot : bseqnr=1, bootseq=0032
2025-04-11 16:25:29 (13692): Guest Log: BIOS: Booting from Hard Disk...
2025-04-11 16:25:29 (13692): Guest Log: BIOS: KBD: unsupported int 16h function 03
2025-04-11 16:25:29 (13692): Guest Log: BIOS: AX=0305 BX=0000 CX=0000 DX=0000 
2025-04-11 16:25:29 (13692): Guest Log: int13_harddisk_ext: function 41, unmapped device for ELDL=81
2025-04-11 16:25:29 (13692): Guest Log: int13_harddisk: function 02, unmapped device for ELDL=81
2025-04-11 16:25:29 (13692): Guest Log: int13_harddisk_ext: function 41, unmapped device for ELDL=82
2025-04-11 16:25:29 (13692): Guest Log: int13_harddisk: function 02, unmapped device for ELDL=82
2025-04-11 16:25:29 (13692): Guest Log: int13_harddisk_ext: function 41, unmapped device for ELDL=83
2025-04-11 16:25:29 (13692): Guest Log: int13_harddisk: function 02, unmapped device for ELDL=83
2025-04-11 16:25:29 (13692): Guest Log: int13_harddisk_ext: function 41, unmapped device for ELDL=84
2025-04-11 16:25:29 (13692): Guest Log: int13_harddisk: function 02, unmapped device for ELDL=84
2025-04-11 16:25:29 (13692): Guest Log: int13_harddisk_ext: function 41, unmapped device for ELDL=85
2025-04-11 16:25:29 (13692): Guest Log: int13_harddisk: function 02, unmapped device for ELDL=85
2025-04-11 16:25:29 (13692): Guest Log: int13_harddisk_ext: function 41, unmapped device for ELDL=86
2025-04-11 16:25:29 (13692): Guest Log: int13_harddisk: function 02, unmapped device for ELDL=86
2025-04-11 16:25:29 (13692): Guest Log: int13_harddisk_ext: function 41, unmapped device for ELDL=87
2025-04-11 16:25:29 (13692): Guest Log: int13_harddisk: function 02, unmapped device for ELDL=87
2025-04-11 16:25:29 (13692): Guest Log: int13_harddisk_ext: function 41, unmapped device for ELDL=88
2025-04-11 16:25:29 (13692): Guest Log: int13_harddisk: function 02, unmapped device for ELDL=88
2025-04-11 16:25:29 (13692): Guest Log: int13_harddisk_ext: function 41, unmapped device for ELDL=89
2025-04-11 16:25:29 (13692): Guest Log: int13_harddisk: function 02, unmapped device for ELDL=89
2025-04-11 16:25:29 (13692): Guest Log: int13_harddisk_ext: function 41, unmapped device for ELDL=8a
2025-04-11 16:25:29 (13692): Guest Log: int13_harddisk: function 02, unmapped device for ELDL=8a
2025-04-11 16:25:29 (13692): Guest Log: int13_harddisk_ext: function 41, unmapped device for ELDL=8b
2025-04-11 16:25:29 (13692): Guest Log: int13_harddisk: function 02, unmapped device for ELDL=8b
2025-04-11 16:25:29 (13692): Guest Log: int13_harddisk_ext: function 41, unmapped device for ELDL=8c
2025-04-11 16:25:29 (13692): Guest Log: int13_harddisk: function 02, unmapped device for ELDL=8c
2025-04-11 16:25:29 (13692): Guest Log: int13_harddisk_ext: function 41, unmapped device for ELDL=8d
2025-04-11 16:25:29 (13692): Guest Log: int13_harddisk: function 02, unmapped device for ELDL=8d
2025-04-11 16:25:29 (13692): Guest Log: int13_harddisk_ext: function 41, unmapped device for ELDL=8e
2025-04-11 16:25:29 (13692): Guest Log: int13_harddisk: function 02, unmapped device for ELDL=8e
2025-04-11 16:25:29 (13692): Guest Log: int13_harddisk_ext: function 41, unmapped device for ELDL=8f
2025-04-11 16:25:29 (13692): Guest Log: int13_harddisk: function 02, unmapped device for ELDL=8f
2025-04-11 16:25:29 (13692): Guest Log: vgdrvHeartbeatInit: Setting up heartbeat to trigger every 2000 milliseconds
2025-04-11 16:25:29 (13692): Guest Log: vboxguest: misc device minor 58, IRQ 20, I/O port d020, MMIO at 00000000f0400000 (size 0x400000)
2025-04-11 16:25:29 (13692): Guest Log: VBoxService 5.2.32 r132073 (verbosity: 0) linux.amd64 (Jul 12 2019 10:32:28) release log
2025-04-11 16:25:29 (13692): Guest Log: 00:00:00.000146 main     Log opened 2025-03-31T15:37:58.207419000Z
2025-04-11 16:25:29 (13692): Guest Log: 00:00:00.000256 main     OS Product: Linux
2025-04-11 16:25:29 (13692): Guest Log: 00:00:00.000317 main     OS Release: 3.10.0-957.27.2.el7.x86_64
2025-04-11 16:25:29 (13692): Guest Log: 00:00:00.000358 main     OS Version: #1 SMP Mon Jul 29 17:46:05 UTC 2019
2025-04-11 16:25:29 (13692): Guest Log: 00:00:00.000400 main     Executable: /opt/VBoxGuestAdditions-5.2.32/sbin/VBoxService
2025-04-11 16:25:29 (13692): Guest Log: 00:00:00.000402 main     Process ID: 1283
2025-04-11 16:25:29 (13692): Guest Log: 00:00:00.000403 main     Package type: LINUX_64BITS_GENERIC
2025-04-11 16:25:29 (13692): Guest Log: 00:00:00.002125 main     5.2.32 r132073 started. Verbose level = 0
2025-04-11 16:25:29 (13692): Guest Log: [INFO] Probing /cvmfs/atlas.cern.ch...
2025-04-11 16:25:29 (13692): Guest Log: [INFO] Mounting shared directory
2025-04-11 16:25:29 (13692): Guest Log: [INFO] Checking for init_data.xml
2025-04-11 16:25:29 (13692): Guest Log: 00:00:10.006953 timesync vgsvcTimeSyncWorker: Radical guest time change: -7 188 657 769 000ns (GuestNow=1 743 428 289 554 663 000 ns GuestLast=1 743 435 478 212 432 000 ns fSetTimeLastLoop=true )
2025-04-11 16:25:29 (13692): Guest Log: [INFO] Probing /cvmfs/atlas.cern.ch... OK
2025-04-11 16:25:29 (13692): Guest Log: [INFO] Detected branch: prod
2025-04-11 16:25:29 (13692): Guest Log: This is the prod version of the ATLAS job wrapper
2025-04-11 16:25:29 (13692): Guest Log: Copying input files
2025-04-11 16:25:29 (13692): Guest Log: Copied input files into RunAtlas.
2025-04-11 16:25:29 (13692): Guest Log: This VM did not configure a local http proxy via BOINC.
2025-04-11 16:25:29 (13692): Guest Log: Small home clusters do not require a local http proxy but it is suggested if
2025-04-11 16:25:29 (13692): Guest Log: more than 10 cores throughout the same LAN segment are regularly running ATLAS like tasks.
2025-04-11 16:25:29 (13692): Guest Log: Further information can be found at the LHC@home message board.
2025-04-11 16:25:29 (13692): Guest Log: Running cvmfs_config stat atlas.cern.ch
2025-04-11 16:25:29 (13692): Guest Log: VERSION PID UPTIME(M) MEM(K) REVISION EXPIRES(M) NOCATALOGS CACHEUSE(K) CACHEMAX(K) NOFDUSE NOFDMAX NOIOERR NOOPEN HITRATE(%) RX(K) SPEED(K/S) HOST PROXY ONLINE
2025-04-11 16:25:29 (13692): Guest Log: 2.6.3.0 1577 0 32024 144328 4 1 3117321 4096000 0 65024 0 0 n/a 0 0 http://s1ihep-cvmfs.openhtc.io:8080/cvmfs/atlas.cern.ch DIRECT 1
2025-04-11 16:25:29 (13692): Guest Log: copied the webapp to /var/www
2025-04-11 16:25:29 (13692): Guest Log: ATHENA_PROC_NUMBER=3
2025-04-11 16:25:29 (13692): Guest Log: ATHENA_CORE_NUMBER=3
2025-04-11 16:25:29 (13692): Guest Log: Apptainer command /cvmfs/atlas.cern.ch/repo/containers/sw/apptainer/x86_64-el7/current/bin/apptainer exec -B /cvmfs,/data,/home/atlas/RunAtlas /cvmfs/atlas.cern.ch/repo/containers/fs/singularity/x86_64-centos7 
2025-04-11 16:25:29 (13692): Guest Log:  *** Starting ATLAS job. (PandaID=6581219873 taskID=43634882) ***
2025-04-11 16:25:29 (13692): Starting VM using VBoxManage interface. (boinc_c2aac6a50a1fa0b4, slot#2)
2025-04-11 16:26:09 (13692): Successfully started VM. (PID = '7712')
2025-04-11 16:26:09 (13692): Reporting VM Process ID to BOINC.
2025-04-11 16:26:09 (13692): VM state change detected. (old = 'poweredoff', new = 'running')
2025-04-11 16:26:09 (13692): Detected: Web Application Enabled (http://localhost:55350)
2025-04-11 16:26:09 (13692): Preference change detected
2025-04-11 16:26:09 (13692): Setting CPU throttle for VM. (100%)
2025-04-11 16:26:09 (13692): Setting checkpoint interval to 900 seconds. (Higher value of (Preference: 60 seconds) or (Vbox_job.xml: 900 seconds))
2025-04-11 16:26:14 (13692): Guest Log: 01:30:22.357669 timesync vgsvcTimeSyncWorker: Radical host time change: 943 415 267 000 000ns (HostNow=1 744 381 574 430 000 000 ns HostLast=1 743 438 159 163 000 000 ns)
2025-04-11 16:26:24 (13692): Guest Log: 01:30:32.366688 timesync vgsvcTimeSyncWorker: Radical guest time change: 944 274 255 467 000ns (GuestNow=1 744 381 584 439 091 000 ns GuestLast=1 743 437 310 183 624 000 ns fSetTimeLastLoop=true )
2025-04-11 16:26:48 (13692): Guest Log:  *** Job finished ***
2025-04-11 16:26:48 (13692): Guest Log:  *** The last 20 lines of the pilot log: ***
2025-04-11 16:26:48 (13692): Guest Log: 2025-04-11 14:26:40,511 | INFO     | caller=run is remaining thread - safe to abort (names=['<_MainThread(MainThread, started 140409306818368)>'])
2025-04-11 16:26:48 (13692): Guest Log: 2025-04-11 14:26:45,517 | INFO     | all workflow threads have been joined
2025-04-11 16:26:48 (13692): Guest Log: 2025-04-11 14:26:45,517 | INFO     | end of generic workflow (traces error code: 1213)
2025-04-11 16:26:48 (13692): Guest Log: 2025-04-11 14:26:45,518 | INFO     | traces error code: 1213
2025-04-11 16:26:48 (13692): Guest Log: 2025-04-11 14:26:45,518 | INFO     | an exit code was already set: 1213 (will be converted to a standard shell code)
2025-04-11 16:26:48 (13692): Guest Log: no translation to shell exit code for error code 1213
2025-04-11 16:26:48 (13692): Guest Log: 2025-04-11 14:26:45,519 | INFO     | pilot has finished (exit code=1213, shell exit code=1)
2025-04-11 16:26:48 (13692): Guest Log: 2025-04-11 14:26:45,762 [wrapper] ==== pilot stdout END ====
2025-04-11 16:26:48 (13692): Guest Log: 2025-04-11 14:26:45,773 [wrapper] ==== wrapper stdout RESUME ====
2025-04-11 16:26:48 (13692): Guest Log: 2025-04-11 14:26:45,782 [wrapper] pilotpid: 5917
2025-04-11 16:26:48 (13692): Guest Log: 2025-04-11 14:26:45,791 [wrapper] Pilot exit status: 1
2025-04-11 16:26:48 (13692): Guest Log: 2025-04-11 14:26:47,772 [wrapper] pandaids: 6581219873
2025-04-11 16:26:48 (13692): Guest Log: 2025-04-11 14:26:47,781 [wrapper] WARNING: pilot exitcode non-zero
2025-04-11 16:26:48 (13692): Guest Log: 2025-04-11 14:26:47,790 [wrapper] WARNING: pilot exitcode non-zero
2025-04-11 16:26:48 (13692): Guest Log: 2025-04-11 14:26:47,799 [wrapper] apfmon messages muted
2025-04-11 16:26:48 (13692): Guest Log: 2025-04-11 14:26:47,860 [wrapper] cleanup supervisor_pilot  3303 5918
2025-04-11 16:26:48 (13692): Guest Log: 2025-04-11 14:26:47,866 [wrapper] Test setup, not cleaning
2025-04-11 16:26:48 (13692): Guest Log: 2025-04-11 14:26:47,872 [wrapper] apfmon messages muted
2025-04-11 16:26:48 (13692): Guest Log: 2025-04-11 14:26:47,878 [wrapper] ==== wrapper stdout END ====
2025-04-11 16:26:48 (13692): Guest Log: 2025-04-11 14:26:47,884 [wrapper] ==== wrapper stderr END ====
2025-04-11 16:26:48 (13692): Guest Log:  *** Error codes and diagnostics ***
2025-04-11 16:26:48 (13692): Guest Log:     "exeErrorCode": 0,
2025-04-11 16:26:48 (13692): Guest Log:     "exeErrorDiag": "",
2025-04-11 16:26:48 (13692): Guest Log:     "pilotErrorCode": 1213,
2025-04-11 16:26:48 (13692): Guest Log:     "pilotErrorDiag": "the max batch system time limit has been reached",
2025-04-11 16:26:48 (13692): Guest Log:  *** Listing of results directory ***
2025-04-11 16:26:48 (13692): Guest Log: total 4892
2025-04-11 16:26:48 (13692): Guest Log: -rw-r--r--. 1 atlas atlas  494584 Mar 31 11:45 pilot3.tar.gz
2025-04-11 16:26:48 (13692): Guest Log: -rw-r--r--. 1 atlas atlas    5112 Mar 31 11:56 queuedata.json
2025-04-11 16:26:48 (13692): Guest Log: -rwx------. 1 atlas atlas   37140 Mar 31 11:58 runpilot2-wrapper.sh
2025-04-11 16:26:48 (13692): Guest Log: -rwxr-xr-x. 1 atlas atlas    9052 Mar 31 13:38 init_data.xml
2025-04-11 16:26:48 (13692): Guest Log: -rwxr-xr-x. 1 atlas atlas  507261 Mar 31 13:38 input.tar.gz
2025-04-11 16:26:48 (13692): Guest Log: -rwxr-xr-x. 1 atlas atlas   17571 Mar 31 13:38 start_atlas.sh
2025-04-11 16:26:48 (13692): Guest Log: lrwxrwxrwx. 1 atlas atlas      20 Mar 31 13:38 EVNT.43634880._000631.pool.root.1 -> /data/./ATLAS.root_0
2025-04-11 16:26:48 (13692): Guest Log: -rw-r--r--. 1 atlas atlas    2617 Mar 31 13:38 pandaJob.out
2025-04-11 16:26:48 (13692): Guest Log: -rw-------. 1 atlas atlas     424 Mar 31 13:39 setup.sh.local
2025-04-11 16:26:48 (13692): Guest Log: -rw-------. 1 atlas atlas  987184 Mar 31 13:44 agis_schedconf.cvmfs.json
2025-04-11 16:26:48 (13692): Guest Log: -rw-------. 1 atlas atlas 1599134 Mar 31 13:44 agis_ddmendpoints.agis.ALL.json
2025-04-11 16:26:48 (13692): Guest Log: drwx------. 4 atlas atlas    4096 Mar 31 13:45 pilot3
2025-04-11 16:26:48 (13692): Guest Log: -rw-------. 1 atlas atlas      97 Mar 31 16:08 pilot_heartbeat.json
2025-04-11 16:26:48 (13692): Guest Log: drwxrwx---. 2 atlas atlas    4096 Apr 11 14:26 PanDA_Pilot-6581219873
2025-04-11 16:26:48 (13692): Guest Log: -rw-------. 1 atlas atlas    1025 Apr 11 14:26 memory_monitor_summary.json
2025-04-11 16:26:48 (13692): Guest Log: -rw-------. 1 atlas atlas    2449 Apr 11 14:26 heartbeat.json
2025-04-11 16:26:48 (13692): Guest Log: -rw-------. 1 atlas atlas  413516 Apr 11 14:26 pilotlog.txt
2025-04-11 16:26:48 (13692): Guest Log: -rw-------. 1 atlas atlas  425745 Apr 11 14:26 log.43634882._003836.job.log.1
2025-04-11 16:26:48 (13692): Guest Log: -rw-r--r--. 1 atlas atlas     571 Apr 11 14:26 runtime_log
2025-04-11 16:26:48 (13692): Guest Log: -rw-------. 1 atlas atlas  440320 Apr 11 14:26 result.tar.gz
2025-04-11 16:26:48 (13692): Guest Log: -rw-r--r--. 1 atlas atlas   10274 Apr 11 14:26 runtime_log.err
2025-04-11 16:26:48 (13692): Guest Log: -rw-------. 1 atlas atlas     648 Apr 11 14:26 p6QKDmcxSJ7nsSi4ap6QjLDmwznN0nGgGQJmkKkKDmlMBLDm6RZuGo.diag
2025-04-11 16:26:48 (13692): Guest Log: Looking for outputfile HITS.43634882._003836.pool.root.1
2025-04-11 16:26:48 (13692): Guest Log: No HITS file was produced
2025-04-11 16:26:48 (13692): Guest Log: Successfully finished the ATLAS job!
2025-04-11 16:26:48 (13692): Guest Log: Copying the results back to the shared directory!
2025-04-11 16:26:49 (13692): Guest Log:  *** Contents of shared directory: ***
2025-04-11 16:26:49 (13692): Guest Log: total 335524
2025-04-11 16:26:49 (13692): Guest Log: -rwxrwxrwx. 1 root root 342593462 Mar 31 13:37 ATLAS.root_0
2025-04-11 16:26:49 (13692): Guest Log: -rwxrwxrwx. 1 root root      9


©2025 CERN