1) Message boards : Theory Application : 1 (0x00000001) error on Theory Simulation v300.02 (vbox64_theory) (Message 41179)
Posted 6 Jan 2020 by droople
Post:
2019-12-29 12:47:43 (8476): Required extension pack not installed, remote desktop not enabled.
If you have Win 10pro or a Linux, you should install Virtualbox extension pack.

@Gunde
Thank you guys. I find that half of my ATLAS and Theory tasks were error, so I decide to opt-out these tasks and focus on Sixtrack.
2) Message boards : Theory Application : 1 (0x00000001) error on Theory Simulation v300.02 (vbox64_theory) (Message 41104)
Posted 29 Dec 2019 by droople
Post:
Dear All,

I got some 1 (0x00000001) errors on Theory Simulation v300.02 (vbox64_theory).

I'm just wondering that anyone have any ideas?

Cheers


https://lhcathome.cern.ch/lhcathome/result.php?resultid=256971517

Here is the details
Name Theory_2279-764797-200_0
Workunit 129335623
Created 29 Dec 2019, 2:25:51 UTC
Sent 29 Dec 2019, 3:20:49 UTC
Report deadline 9 Jan 2020, 3:20:49 UTC
Received 29 Dec 2019, 5:47:28 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 1 (0x00000001) Unknown error code
Computer ID 10613404
Run time 10 min 21 sec
CPU time 2 min 1 sec
Validate state Invalid
Credit 0.00
Device peak FLOPS 4.04 GFLOPS
Application version Theory Simulation v300.02 (vbox64_theory)
windows_x86_64
Peak working set size 51.41 MB
Peak swap size 78.33 MB
Peak disk usage 799.65 MB

Stderr output
<core_client_version>7.14.2</core_client_version>
<![CDATA[
<message>
Incorrect function.
(0x1) - exit code 1 (0x1)</message>
<stderr_txt>
2019-12-29 12:47:27 (8476): Detected: vboxwrapper 26197
2019-12-29 12:47:27 (8476): Detected: BOINC client v7.7
2019-12-29 12:47:28 (8476): Detected: VirtualBox VboxManage Interface (Version: 6.0.12)
2019-12-29 12:47:29 (8476): Detected: Heartbeat check (file: 'heartbeat' every 1200.000000 seconds)
2019-12-29 12:47:29 (8476): Successfully copied 'init_data.xml' to the shared directory.
2019-12-29 12:47:29 (8476): Successfully copied 'input' to the shared directory.
2019-12-29 12:47:31 (8476): Create VM. (boinc_4f3bf0b0259f4bbe, slot#4)
2019-12-29 12:47:33 (8476): Setting Memory Size for VM. (630MB)
2019-12-29 12:47:33 (8476): Setting CPU Count for VM. (1)
2019-12-29 12:47:34 (8476): Setting Chipset Options for VM.
2019-12-29 12:47:34 (8476): Setting Boot Options for VM.
2019-12-29 12:47:35 (8476): Setting Network Configuration for NAT.
2019-12-29 12:47:35 (8476): Enabling VM Network Access.
2019-12-29 12:47:36 (8476): Disabling USB Support for VM.
2019-12-29 12:47:37 (8476): Disabling COM Port Support for VM.
2019-12-29 12:47:37 (8476): Disabling LPT Port Support for VM.
2019-12-29 12:47:38 (8476): Disabling Audio Support for VM.
2019-12-29 12:47:38 (8476): Disabling Clipboard Support for VM.
2019-12-29 12:47:39 (8476): Disabling Drag and Drop Support for VM.
2019-12-29 12:47:39 (8476): Adding storage controller(s) to VM.
2019-12-29 12:47:40 (8476): Adding virtual disk drive to VM. (vm_image.vdi)
2019-12-29 12:47:41 (8476): Adding VirtualBox Guest Additions to VM.
2019-12-29 12:47:41 (8476): Adding network bandwidth throttle group to VM. (Defaulting to 1024GB)
2019-12-29 12:47:42 (8476): forwarding host port 53366 to guest port 80
2019-12-29 12:47:43 (8476): Enabling remote desktop for VM.
2019-12-29 12:47:43 (8476): Required extension pack not installed, remote desktop not enabled.
2019-12-29 12:47:43 (8476): Enabling shared directory for VM.
2019-12-29 12:47:43 (8476): Starting VM using VBoxManage interface. (boinc_4f3bf0b0259f4bbe, slot#4)
2019-12-29 12:47:51 (8476): Successfully started VM. (PID = '5096')
2019-12-29 12:47:51 (8476): Reporting VM Process ID to BOINC.
2019-12-29 12:47:51 (8476): Guest Log: BIOS: VirtualBox 6.0.12

2019-12-29 12:47:51 (8476): Guest Log: CPUID EDX: 0x178bfbff

2019-12-29 12:47:51 (8476): Guest Log: BIOS: ata0-0: PCHS=16383/16/63 LCHS=1024/255/63

2019-12-29 12:47:51 (8476): VM state change detected. (old = 'PoweredOff', new = 'Running')
2019-12-29 12:47:51 (8476): Detected: Web Application Enabled (http://localhost:53366)
2019-12-29 12:47:51 (8476): Preference change detected
2019-12-29 12:47:51 (8476): Setting CPU throttle for VM. (70%)
2019-12-29 12:47:52 (8476): Setting checkpoint interval to 600 seconds. (Higher value of (Preference: 60 seconds) or (Vbox_job.xml: 600 seconds))
2019-12-29 12:47:53 (8476): Guest Log: BIOS: Boot : bseqnr=1, bootseq=0032

2019-12-29 12:47:53 (8476): Guest Log: BIOS: Booting from Hard Disk...

2019-12-29 12:47:56 (8476): Guest Log: BIOS: KBD: unsupported int 16h function 03

2019-12-29 12:47:56 (8476): Guest Log: BIOS: AX=0305 BX=0000 CX=0000 DX=0000

2019-12-29 12:48:52 (8476): Guest Log: vgdrvHeartbeatInit: Setting up heartbeat to trigger every 2000 milliseconds

2019-12-29 12:48:53 (8476): Guest Log: vboxguest: misc device minor 56, IRQ 20, I/O port d020, MMIO at 00000000f0400000 (size 0x400000)

2019-12-29 12:49:06 (8476): Guest Log: VBoxService 5.2.6 r120293 (verbosity: 0) linux.amd64 (Jan 15 2018 14:51:00) release log

2019-12-29 12:49:06 (8476): Guest Log: 00:00:00.000329 main Log opened 2019-12-29T04:49:19.858828000Z

2019-12-29 12:49:06 (8476): Guest Log: 00:00:00.000652 main OS Product: Linux

2019-12-29 12:49:06 (8476): Guest Log: 00:00:00.000756 main OS Release: 4.14.76-13.cernvm.x86_64

2019-12-29 12:49:06 (8476): Guest Log: 00:00:00.000850 main OS Version: #1 SMP Tue Oct 16 18:26:15 CEST 2018

2019-12-29 12:49:06 (8476): Guest Log: 00:00:00.000938 main Executable: /usr/sbin/VBoxService

2019-12-29 12:49:06 (8476): Guest Log: 00:00:00.000941 main Process ID: 3213

2019-12-29 12:49:06 (8476): Guest Log: 00:00:00.000944 main Package type: LINUX_64BITS_GENERIC

2019-12-29 12:49:06 (8476): Guest Log: 00:00:00.007659 main 5.2.6 r120293 started. Verbose level = 0

2019-12-29 12:51:24 (8476): Guest Log: 12:51:37 CST +08:00 2019-12-29: cranky: [INFO] Detected Theory App

2019-12-29 12:51:24 (8476): Guest Log: 12:51:37 CST +08:00 2019-12-29: cranky: [INFO] Checking CVMFS.

2019-12-29 12:52:07 (8476): Guest Log: 12:52:20 CST +08:00 2019-12-29: cranky: [INFO] Checking runc.

2019-12-29 12:52:07 (8476): Guest Log: 12:52:20 CST +08:00 2019-12-29: cranky: [INFO] Creating the filesystem.

2019-12-29 12:52:07 (8476): Guest Log: 12:52:20 CST +08:00 2019-12-29: cranky: [INFO] Using /cvmfs/cernvm-prod.cern.ch/cvm3

2019-12-29 12:52:08 (8476): Guest Log: 12:52:21 CST +08:00 2019-12-29: cranky: [INFO] Updating config.json.

2019-12-29 12:52:08 (8476): Guest Log: 12:52:21 CST +08:00 2019-12-29: cranky: [INFO] Running Container 'runc'.

2019-12-29 12:52:09 (8476): Guest Log: job: stdoutf=

2019-12-29 12:52:09 (8476): Guest Log: job: stderrf=

2019-12-29 12:52:09 (8476): Guest Log: job: htmld=/shared/html/job

2019-12-29 12:52:10 (8476): Guest Log: 12:52:23 CST +08:00 2019-12-29: cranky: [INFO] ===> [runRivet] Sun Dec 29 04:52:22 UTC 2019 [boinc pp bbbar 7000 - - pythia6 6.426 px 100000 200]

2019-12-29 12:52:36 (8476): Guest Log: job: CPU usage:

2019-12-29 12:52:36 (8476): Guest Log: 0m0.064s 0m0.088s

2019-12-29 12:52:36 (8476): Guest Log: 0m0.111s 0m0.184s

2019-12-29 12:52:36 (8476): Guest Log: 12:52:49 CST +08:00 2019-12-29: cranky: [ERROR] Container 'runc' terminated with status code 1.

2019-12-29 12:52:37 (8476): Guest Log: [ERROR] Job Failed

2019-12-29 12:52:37 (8476): Guest Log: [INFO] Shutting Down.

2019-12-29 12:52:37 (8476): VM Completion File Detected.
2019-12-29 12:52:37 (8476): VM Completion Message: Job Failed
.
2019-12-29 12:52:37 (8476): Powering off VM.
2019-12-29 12:57:38 (8476): VM did not power off when requested.
2019-12-29 12:57:38 (8476): VM was successfully terminated.
2019-12-29 12:57:38 (8476): Deregistering VM. (boinc_4f3bf0b0259f4bbe, slot#4)
2019-12-29 12:57:39 (8476): Removing network bandwidth throttle group from VM.
2019-12-29 12:57:40 (8476): Removing VM from VirtualBox.
12:57:45 (8476): called boinc_finish(1)

</stderr_txt>
]]>
3) Message boards : Theory Application : 0 (0x00000000) Error on 300.02 Theory Simulation (vbox64_theory) (Message 41103)
Posted 29 Dec 2019 by droople
Post:
Thank you, Crystal. I put the 25% to 50% solved the issues.
4) Message boards : ATLAS application : Aborted 202 error on 2.00 ATLAS simulation (Message 41079)
Posted 26 Dec 2019 by droople
Post:
Hi Droople,
Atlas have a small window to finish the tasks (max.7 Days).
Some time it is shorter needed from Cern.
Your tasks started from a other User and finished. So this tasks from you are obsolet.


Thank you so much maeax,
5) Message boards : ATLAS application : Aborted 202 error on 2.00 ATLAS simulation (Message 41069)
Posted 25 Dec 2019 by droople
Post:
Dear All,

I got some Aborted (202) error on 2.00 ATLAS simulation.

I'm just wondering anyone knows the reason?

Cheers

Task details:

https://lhcathome.cern.ch/lhcathome/result.php?resultid=256286867

Name k8BNDmGts2vnsSi4apGgGQJmABFKDmABFKDmoVmaDmABFKDmFW2Lkm_0
Workunit 128911692
Created 21 Dec 2019, 12:18:35 UTC
Sent 21 Dec 2019, 14:42:14 UTC
Report deadline 29 Dec 2019, 14:42:14 UTC
Received 25 Dec 2019, 11:13:04 UTC
Server state Over
Outcome Computation error
Client state Cancelled by server
Exit status 202 (0x000000CA) EXIT_ABORTED_BY_PROJECT
Computer ID 10625858
Run time

CPU time

Validate state Invalid
Credit 0.00
Device peak FLOPS 43.41 GFLOPS
Application version ATLAS Simulation v2.00 (vbox64_mt_mcore_atlas)
windows_x86_64

Stderr output
<core_client_version>7.14.2</core_client_version>

Nothing else
6) Message boards : Theory Application : 0 (0x00000000) Error on 300.02 Theory Simulation (vbox64_theory) (Message 41068)
Posted 25 Dec 2019 by droople
Post:
Dear All,

I got several 0 (0x00000000) Error on 300.02 Theory Simulation (vbox64_theory),not sure if anything caused this.

.
.
2019-12-20 16:22:43 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 16:23:04 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 16:23:23 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 16:23:33 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 16:50:53 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 16:51:03 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 16:52:54 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 16:53:04 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 16:58:55 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 16:59:05 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 17:06:58 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 17:07:08 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 17:37:00 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 17:37:50 (70060): VM state change detected. (old = 'Paused', new = 'Running')
etc etc

Virtual Machines don't like this.
It's maybe caused by your BOINC preferences. Have you ticked "Suspend when non-BOINC CPU usage is above ... % (untick that or set it to 100%)
or a very low number for "Switch between tasks every ... seconds"
When you want to reduce cpu-usage you could keep a core idle by setting "Use at most ...% of the CPUs" lower
or set "Use at most ...% of CPU time" lower.


Thank you very much, what's the ideal number for "Switch between tasks every ... seconds" do you recommend? I usually use the default number 60 minutes

Cheers
7) Message boards : Theory Application : 0 (0x00000000) Error on 300.02 Theory Simulation (vbox64_theory) (Message 41049)
Posted 24 Dec 2019 by droople
Post:
Dear All,

I got several 0 (0x00000000) Error on 300.02 Theory Simulation (vbox64_theory),not sure if anything caused this.

https://lhcathome.cern.ch/lhcathome/result.php?resultid=255203739

Name Theory_2279-773832-193_0
Workunit 128286998
Created 12 Dec 2019, 2:57:32 UTC
Sent 12 Dec 2019, 4:28:00 UTC
Report deadline 23 Dec 2019, 4:28:00 UTC
Received 21 Dec 2019, 3:33:06 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 0 (0x00000000)
Computer ID 10410466
Run time 4 days 5 hours 3 min 41 sec
CPU time 3 days 2 hours 52 min 38 sec
Validate state Invalid
Credit 0.00
Device peak FLOPS 3.10 GFLOPS
Application version Theory Simulation v300.02 (vbox64_theory)
windows_x86_64
Peak working set size 60.53 MB
Peak swap size 105.28 MB
Peak disk usage 1.06 GB

Stderr output
<core_client_version>7.14.2</core_client_version>
<![CDATA[
<stderr_txt>

2019-12-20 08:08:13 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 09:28:14 (70060): Status Report: Job Duration: '360000.000000'
2019-12-20 09:28:14 (70060): Status Report: Elapsed Time: '282013.342487'
2019-12-20 09:28:14 (70060): Status Report: CPU Time: '211607.743651'
2019-12-20 10:20:55 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 10:21:05 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 10:21:55 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 10:22:05 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 10:50:49 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 10:50:59 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 10:52:59 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 10:53:09 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 11:10:02 (70060): Status Report: Job Duration: '360000.000000'
2019-12-20 11:10:02 (70060): Status Report: Elapsed Time: '288013.342487'
2019-12-20 11:10:02 (70060): Status Report: CPU Time: '216121.039782'
2019-12-20 12:23:11 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 12:23:21 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 12:51:12 (70060): Status Report: Job Duration: '360000.000000'
2019-12-20 12:51:12 (70060): Status Report: Elapsed Time: '294013.342487'
2019-12-20 12:51:12 (70060): Status Report: CPU Time: '220622.869840'
2019-12-20 13:21:12 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 13:21:22 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 13:22:02 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 13:22:12 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 13:22:21 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 13:22:41 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 13:23:01 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 13:23:11 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 13:23:21 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 13:23:31 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 13:24:43 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 13:24:53 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 13:25:23 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 13:25:33 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 13:28:44 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 13:28:54 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 13:58:15 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 13:58:25 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 14:23:17 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 14:23:27 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 14:23:36 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 14:23:46 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 14:34:42 (70060): Status Report: Job Duration: '360000.000000'
2019-12-20 14:34:42 (70060): Status Report: Elapsed Time: '300013.830986'
2019-12-20 14:34:42 (70060): Status Report: CPU Time: '225155.744097'
2019-12-20 14:37:02 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 14:37:12 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 15:07:01 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 15:07:11 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 15:19:17 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 15:19:27 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 15:19:37 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 15:19:47 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 15:21:57 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 15:22:07 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 15:22:18 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 15:22:38 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 15:22:58 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 15:23:08 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 15:23:39 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 15:23:49 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 15:52:01 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 15:52:12 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 16:17:51 (70060): Status Report: Job Duration: '360000.000000'
2019-12-20 16:17:51 (70060): Status Report: Elapsed Time: '306014.702486'
2019-12-20 16:17:51 (70060): Status Report: CPU Time: '229693.142382'
2019-12-20 16:22:43 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 16:23:04 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 16:23:23 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 16:23:33 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 16:50:53 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 16:51:03 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 16:52:54 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 16:53:04 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 16:58:55 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 16:59:05 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 17:06:58 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 17:07:08 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 17:37:00 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 17:37:50 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 17:52:35 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 17:52:45 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 18:01:36 (70060): Status Report: Job Duration: '360000.000000'
2019-12-20 18:01:36 (70060): Status Report: Elapsed Time: '312014.702486'
2019-12-20 18:01:36 (70060): Status Report: CPU Time: '234130.668828'
2019-12-20 18:07:28 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 18:07:39 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 18:38:20 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 18:38:30 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 19:02:18 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 19:12:09 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 19:13:47 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 19:18:16 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 19:22:50 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 19:23:01 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 19:23:32 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 19:23:42 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 19:26:02 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 19:26:13 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 19:26:22 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 19:26:32 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 19:33:31 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 19:36:32 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 19:37:08 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 19:37:18 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 19:37:38 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 19:37:48 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 19:47:50 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 19:51:00 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 19:53:23 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 19:53:34 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 19:53:35 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 20:24:39 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 20:24:44 (70060): Guest Log: 67:48:31.860212 timesync vgsvcTimeSyncWorker: Radical host time change: 1 887 931 000 000ns (HostNow=1 576 844 683 936 000 000 ns HostLast=1 576 842 796 005 000 000 ns)

2019-12-20 20:24:57 (70060): Guest Log: 67:48:41.865523 timesync vgsvcTimeSyncWorker: Radical guest time change: 2 717 589 876 000ns (GuestNow=1 576 844 693 965 059 000 ns GuestLast=1 576 841 976 375 183 000 ns fSetTimeLastLoop=true )

2019-12-20 20:25:25 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 20:25:35 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 20:31:01 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 20:42:40 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 20:42:58 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 20:43:09 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 20:48:18 (70060): Status Report: Job Duration: '360000.000000'
2019-12-20 20:48:18 (70060): Status Report: Elapsed Time: '318014.751989'
2019-12-20 20:48:18 (70060): Status Report: CPU Time: '238533.313450'
2019-12-20 20:48:31 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 20:51:33 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 20:52:02 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 20:52:12 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 20:52:21 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 20:52:31 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 20:54:03 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 20:54:13 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 20:59:44 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 20:59:54 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 21:04:36 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 21:04:46 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 21:06:58 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 21:07:07 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 21:07:17 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 21:07:27 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 21:13:39 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 21:17:26 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 21:19:31 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 21:19:41 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 21:20:41 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 21:20:51 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 21:22:01 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 21:22:12 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 21:22:22 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 21:22:32 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 21:22:42 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 21:22:52 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 21:23:02 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 21:23:12 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 21:24:23 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 21:24:33 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 21:25:03 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 21:25:13 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 21:27:04 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 21:27:14 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 21:30:05 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 21:30:15 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 21:34:28 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 21:34:39 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 21:36:59 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 21:37:09 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 21:37:20 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 21:37:30 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 21:40:30 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 21:40:40 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 21:49:34 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 21:49:44 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 21:51:54 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 21:52:05 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 21:52:15 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 21:52:25 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 21:54:06 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 21:54:16 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 21:59:59 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 22:00:08 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 22:03:24 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 22:08:37 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 22:19:34 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 22:19:45 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 22:21:55 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 22:22:05 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 22:22:15 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 22:22:25 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 22:22:35 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 22:22:45 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 22:24:16 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 22:24:26 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 22:24:41 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 22:27:44 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 22:34:28 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 22:34:39 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 22:35:09 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 22:35:19 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 22:35:29 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 22:35:39 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 22:35:49 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 22:35:59 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 22:36:09 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 22:36:19 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 22:36:29 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 22:36:39 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 22:36:49 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 22:37:09 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 22:37:19 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 22:37:30 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 22:37:39 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 22:37:49 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 22:37:59 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 22:38:09 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 22:38:19 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 22:38:29 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 22:38:40 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 22:38:51 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 22:39:01 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 22:39:11 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 22:39:21 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 22:39:31 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 22:44:51 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 22:45:01 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 22:49:34 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 22:49:44 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 22:50:44 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 22:50:54 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 22:51:55 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 22:52:06 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 22:52:16 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 22:52:26 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 22:53:57 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 22:54:07 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 22:54:28 (70060): Status Report: Job Duration: '360000.000000'
2019-12-20 22:54:28 (70060): Status Report: Elapsed Time: '324015.197491'
2019-12-20 22:54:28 (70060): Status Report: CPU Time: '242935.474469'
2019-12-20 22:58:03 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 23:06:50 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 23:07:18 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 23:07:28 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 23:19:33 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 23:19:43 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 23:22:14 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 23:22:24 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 23:22:34 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 23:22:44 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 23:24:16 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 23:24:26 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 23:34:40 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 23:34:50 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 23:37:01 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 23:37:11 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 23:37:21 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 23:37:31 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 23:49:36 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 23:49:46 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 23:52:06 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 23:52:16 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-20 23:52:26 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-20 23:52:36 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 00:00:09 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 00:00:19 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 00:04:31 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 00:04:41 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 00:07:01 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 00:07:11 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 00:07:21 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 00:07:31 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 00:08:11 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 00:08:21 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 00:09:13 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 00:09:23 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 00:14:15 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 00:14:25 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 00:14:35 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 00:14:45 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 00:14:55 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 00:15:05 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 00:15:15 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 00:15:25 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 00:15:34 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 00:15:44 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 00:19:36 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 00:19:46 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 00:22:07 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 00:22:17 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 00:22:57 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 00:23:07 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 00:34:33 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 00:34:43 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 00:44:17 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 00:44:27 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 00:49:37 (70060): Status Report: Job Duration: '360000.000000'
2019-12-21 00:49:37 (70060): Status Report: Elapsed Time: '330015.678490'
2019-12-21 00:49:37 (70060): Status Report: CPU Time: '247363.188451'
2019-12-21 00:49:40 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 00:49:50 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 00:52:22 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 00:52:32 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 01:00:25 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 01:00:35 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 01:04:37 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 01:04:47 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 01:06:59 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 01:07:09 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 01:07:28 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 01:07:38 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 01:19:33 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 01:19:43 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 01:22:25 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 01:22:35 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 01:22:45 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 01:22:54 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 01:23:45 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 01:23:55 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 01:34:36 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 01:34:46 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 01:35:46 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 01:35:56 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 01:36:06 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 01:36:16 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 01:36:26 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 01:36:37 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 01:37:06 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 01:37:16 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 01:37:26 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 01:37:36 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 01:45:33 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 01:45:43 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 01:49:04 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 01:49:15 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 01:49:34 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 01:49:44 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 01:51:45 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 01:51:55 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 01:52:05 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 01:52:15 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 01:53:08 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 01:53:18 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 01:53:28 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 01:53:38 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 01:54:19 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 01:54:29 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 02:00:32 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 02:00:42 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 02:04:45 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 02:04:53 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 02:07:44 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 02:07:54 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 02:19:47 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 02:19:57 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 02:22:48 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 02:22:58 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 02:23:08 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 02:23:18 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 02:34:44 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 02:34:54 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 02:36:15 (70060): Status Report: Job Duration: '360000.000000'
2019-12-21 02:36:15 (70060): Status Report: Elapsed Time: '336015.678490'
2019-12-21 02:36:15 (70060): Status Report: CPU Time: '251788.656019'
2019-12-21 02:37:04 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 02:37:14 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 02:37:23 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 02:37:34 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 02:49:40 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 02:49:50 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 02:52:51 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 02:53:01 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 02:54:53 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 02:55:03 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 03:04:39 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 03:04:49 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 03:07:04 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 03:07:14 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 03:19:40 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 03:19:50 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 03:22:01 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 03:22:11 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 03:22:21 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 03:22:31 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 03:22:41 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 03:22:51 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 03:24:59 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 03:25:09 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 03:34:31 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 03:34:41 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 03:37:02 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 03:37:12 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 03:37:22 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 03:37:32 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 03:49:38 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 03:49:48 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 03:52:00 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 03:52:10 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 03:52:20 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 03:52:30 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 03:55:20 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 03:55:30 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 04:00:53 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 04:01:03 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 04:04:34 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 04:04:44 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 04:07:05 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 04:07:15 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 04:19:40 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 04:19:50 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 04:21:04 (70060): Status Report: Job Duration: '360000.000000'
2019-12-21 04:21:04 (70060): Status Report: Elapsed Time: '342015.732990'
2019-12-21 04:21:04 (70060): Status Report: CPU Time: '256239.816952'
2019-12-21 04:23:01 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 04:23:11 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 04:34:36 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 04:34:46 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 04:35:57 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 04:36:07 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 04:36:48 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 04:36:58 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 04:37:07 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 04:37:17 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 04:37:27 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 04:37:37 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 04:37:47 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 04:38:07 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 04:38:17 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 04:38:27 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 04:38:37 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 04:38:47 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 04:38:58 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 04:39:08 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 04:39:18 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 04:39:28 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 04:39:38 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 04:39:48 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 04:40:58 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 04:41:08 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 04:41:28 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 04:41:38 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 04:41:48 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 04:41:58 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 04:42:19 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 04:42:29 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 04:42:39 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 04:42:49 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 04:45:40 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 04:45:50 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 04:46:10 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 04:46:20 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 04:46:50 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 04:47:08 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 04:47:21 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 04:47:30 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 04:47:41 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 04:47:51 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 04:48:00 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 04:48:10 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 04:48:20 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 04:48:30 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 04:48:40 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 04:48:50 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 04:49:00 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 04:49:10 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 04:49:21 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 04:49:31 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 04:49:41 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 04:49:51 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 04:50:01 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 04:50:11 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 04:50:21 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 04:50:31 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 04:50:41 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 04:50:51 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 04:51:42 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 04:51:52 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 04:52:02 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 04:52:12 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 04:52:22 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 04:52:32 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 04:52:52 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 04:53:02 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 04:53:42 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 04:53:52 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 04:54:02 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 04:54:12 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 04:54:22 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 04:54:32 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 04:54:52 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 04:55:02 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 04:55:12 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 04:55:22 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 04:55:32 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 04:55:42 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 04:55:52 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 04:56:02 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 04:56:12 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 04:56:22 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 04:57:02 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 04:57:12 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 04:57:53 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 04:58:03 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 04:58:13 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 04:58:23 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 04:58:43 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 04:58:53 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 05:01:04 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 05:01:15 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 05:04:35 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 05:04:45 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 05:07:16 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 05:07:26 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 05:19:44 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 05:19:53 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 05:21:44 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 05:21:54 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 05:22:04 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 05:22:14 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 05:22:24 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 05:22:34 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 05:24:56 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 05:25:06 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 05:25:26 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 05:25:36 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 05:25:55 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 05:26:05 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 05:26:56 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 05:27:06 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 05:34:39 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 05:34:49 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 05:37:00 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 05:37:10 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 05:37:20 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 05:37:30 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 05:49:36 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 05:49:46 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 05:52:18 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 05:52:28 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 05:52:37 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 05:52:48 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 05:55:19 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 05:55:29 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 06:01:22 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 06:01:32 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 06:04:41 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 06:04:51 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 06:07:02 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 06:07:12 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 06:07:22 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 06:07:32 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 06:14:38 (70060): Status Report: Job Duration: '360000.000000'
2019-12-21 06:14:38 (70060): Status Report: Elapsed Time: '348016.066990'
2019-12-21 06:14:38 (70060): Status Report: CPU Time: '260684.441443'
2019-12-21 06:19:39 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 06:19:49 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 06:22:51 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 06:23:01 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 06:23:11 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 06:23:21 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 06:34:36 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 06:34:46 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 06:37:46 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 06:37:56 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 06:49:42 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 06:49:52 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 06:52:03 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 06:52:13 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 06:56:06 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 06:56:16 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 07:01:38 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 07:01:48 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 07:04:39 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 07:04:49 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 07:07:30 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 07:07:40 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 07:17:55 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 07:18:05 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 07:18:14 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 07:18:25 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 07:18:35 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 07:18:45 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 07:18:55 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 07:19:05 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 07:19:15 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 07:19:25 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 07:19:35 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 07:19:45 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 07:20:14 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 07:20:25 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 07:20:35 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 07:20:45 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 07:20:55 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 07:21:05 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 07:21:15 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 07:21:25 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 07:21:35 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 07:21:46 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 07:21:55 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 07:22:06 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 07:22:16 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 07:22:26 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 07:22:35 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 07:22:46 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 07:22:56 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 07:23:06 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 07:23:16 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 07:23:26 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 07:23:36 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 07:23:46 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 07:23:56 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 07:24:06 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 07:24:16 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 07:24:26 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 07:24:35 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 07:24:46 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 07:24:56 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 07:25:06 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 07:25:16 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 07:25:26 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 07:25:36 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 07:25:46 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 07:25:56 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 07:26:06 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 07:26:17 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 07:26:27 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 07:26:36 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 07:26:46 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 07:27:37 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 07:27:47 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 07:27:57 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 07:28:07 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 07:34:40 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 07:34:50 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 07:36:10 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 07:36:21 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 07:37:10 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 07:37:20 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 07:37:30 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 07:37:40 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 07:38:00 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 07:38:11 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 07:38:41 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 07:38:51 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 07:49:47 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 07:49:56 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 07:52:38 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 07:52:48 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 07:56:19 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 07:56:29 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 08:01:52 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 08:02:02 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 08:04:17 (70060): Status Report: Job Duration: '360000.000000'
2019-12-21 08:04:17 (70060): Status Report: Elapsed Time: '354016.983490'
2019-12-21 08:04:17 (70060): Status Report: CPU Time: '265157.816919'
2019-12-21 08:04:44 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 08:04:54 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 08:08:05 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 08:08:15 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 08:16:09 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 08:16:19 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 08:19:41 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 08:19:51 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 08:22:02 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 08:22:12 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 08:22:21 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 08:22:31 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 08:24:12 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 08:24:22 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 08:26:03 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 08:26:14 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 08:34:35 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 08:34:45 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 08:37:07 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 08:37:17 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 08:37:27 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 08:37:37 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 08:38:31 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 08:38:41 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 08:43:54 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 08:44:04 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 08:49:37 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 08:49:47 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 08:52:18 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 08:52:28 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 08:52:38 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 08:52:48 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 08:56:08 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 08:56:19 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 09:02:11 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 09:02:21 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 09:04:40 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 09:04:50 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 09:07:11 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 09:07:21 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 09:07:31 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 09:07:41 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 09:09:36 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 09:12:38 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 09:19:36 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 09:19:46 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 09:23:07 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 09:23:17 (70060): VM state change detected. (old = 'Paused', new = 'Running')
2019-12-21 09:26:48 (70060): VM state change detected. (old = 'Running', new = 'Paused')
2019-12-21 09:26:5

The record stopped here.

Cheers
8) Message boards : Number crunching : Something is wrong managed!!! (Message 20539)
Posted 26 Sep 2008 by droople
Post:

I just think its unfair for the people who finish the WU before the deadline but only a bit later than other people. Not all the computers are running and connecting to Internet and crunching LHC@Home 24 hours.


It wasnt very long ago that I joined this project, and I must admit that initially I felt the same way you do. It just seemed totally unfair that after waiting SO long for WUs to crunch, that when I finally get some, it seems that most get aborted before my clients barely get started on them.

I dont feel that way any longer. Why, you might ask? Well, not long ago, while I was in the process of composing a post for a new thread(not unlike the first in this thread), I had an epiphany. I suddenly realized that, it really shouldnt matter to me how may WUs I crunch and get credit for. The important thing is that the work just gets done period. It doesnt matter if my total credit is 1 or 1 million. For me, its not a contest, because the only \"winner\" should be the LHC and the scientists who have devoted their lives to the science.

By participating, I am offering my available resources to the project. If they are used, great! If they are not, thats OK too. Either way, I am still participating and feel like I am a part of something very important thats much bigger than me.

Leevis


Hi Leevis

I understand what you mean, as long as you can contribute something to the project, you are satisfied.

But please don\'t forget crunching needs electricity, and in Australia, the power stations burn coal(85%). LHC@home aborted the results, this means some CO2 was generated for nothing.

It\'s not sustainable.

Cheers


They are canceling only redundant results that have not started crunching. If they haven\'t started then next to 0 CPU time and electricity has been spent on them. It\'s the tasks that are redundant but don\'t get canceled because the host has already started them... those are the results that are wasting CPU time and electricity. If hosts would contact the server more often then more redundant tasks would get canceled. The project managers can direct hosts to contact the server more frequently but they are not doing so.

There are BOINC options you can set on your end (the host end) which will tend to cause your computer to contact the server more often. More frequent contact increases the odds of your computer receiving a cancel order before it starts crunching a redundant task rather than after.


So you mean if I start to crunch the WU before connect with the server, the WU will be completed without cancelling even when during the crunching, BOINC communicates with the server?

Cheers
9) Message boards : Number crunching : Something is wrong managed!!! (Message 20536)
Posted 26 Sep 2008 by droople
Post:

I just think its unfair for the people who finish the WU before the deadline but only a bit later than other people. Not all the computers are running and connecting to Internet and crunching LHC@Home 24 hours.


It wasnt very long ago that I joined this project, and I must admit that initially I felt the same way you do. It just seemed totally unfair that after waiting SO long for WUs to crunch, that when I finally get some, it seems that most get aborted before my clients barely get started on them.

I dont feel that way any longer. Why, you might ask? Well, not long ago, while I was in the process of composing a post for a new thread(not unlike the first in this thread), I had an epiphany. I suddenly realized that, it really shouldnt matter to me how may WUs I crunch and get credit for. The important thing is that the work just gets done period. It doesnt matter if my total credit is 1 or 1 million. For me, its not a contest, because the only \"winner\" should be the LHC and the scientists who have devoted their lives to the science.

By participating, I am offering my available resources to the project. If they are used, great! If they are not, thats OK too. Either way, I am still participating and feel like I am a part of something very important thats much bigger than me.

Leevis


Hi Leevis

I understand what you mean, as long as you can contribute something to the project, you are satisfied.

But please don\'t forget crunching needs electricity, and in Australia, the power stations burn coal(85%). LHC@home aborted the results, this means some CO2 was generated for nothing.

It\'s not sustainable.

Cheers
10) Message boards : Number crunching : Something is wrong managed!!! (Message 20534)
Posted 25 Sep 2008 by droople
Post:
THx for all replies.

Sure is better to speed-up the server-side; when 3 sucess results are returned, then the other work is not necessary.

But you can do this by sending out only 3 WU !!
(or 4 , for assurance, when one fails)


The problem with LHC tasks is that the first 3 Success returns frequently fail to match. Frequently, even a 4th Success result fails to match. And frequently 1 or more of the first 3 returns Compute Error. That happens so frequently that they find it is faster just to send 5 results at the beginning, in other words, to use Initial Replication = 5 (IR = 5).

More precisely, it was faster to use the IR = 5 strategy when they were using the old server code. In fact, that was the only speed up strategy possible with the old server code. The new server code they recently installed has new features and capabilities that make possible new strategys where maybe they can have IR = 3 and complete the batch of work just as quickly as with IR = 5. But that strategy also includes implementing some as yet undetermined combination of these new features:

1) limiting crunchers to having no more than X tasks per CPU in their cache, to spread the work out over more hosts
2) the server keeping a list of fast reliable hosts to which additional tasks will be sent if 1 or more of the first 3 tasks fails or the first 3 don\\\'t achieve quorum.
3) possibly reducing the deadline
4) making BOINC client version 5.8.16 a minimum requirement to participate in the project
5) other new features I have forgotten at the moment

Using those new features, other projects that were forced to use IR = 5 a few years ago have found they are now able to reduce IR to 3 and still finish the batch quickly but they likely had to rewrite some/all of their verifier/assimilator/feeder/transitioner scripts to accomplish all that. That takes time and money. We can only hope that LHC will find a way to incorporate some of those new features into their strategy, if they can. Canceling redundant tasks is a good start.




I just think it's unfair for the people who finish the WU before the deadline but only a bit later than other people. Not all the computers are running and connecting to Internet and crunching LHC@Home 24 hours.
11) Message boards : Number crunching : Task was reported too late to validate (Message 19964)
Posted 1 Sep 2008 by droople
Post:
Finally servers accepting results. But credit don't granted because of "Task was reported too late to validate". :(
All tasks was finished more than week ago.
Why report deadline wasn't increased?


I have the same problem.



©2024 CERN