LHC@home

Not sure how to troubleshoot CMS task errors like this https://lhcathome.cern.ch/lhcathome/result.php?resultid=331542649
which shows as 'Error while computing' in my task list.

I'm running BOINC Manager 7.16.19 and VirtualBox 6.1.28 on an Intel Mac Mini macOS 11.6.1
I think these errors started since I went from macOS 11.5.x to 11.6.x, but I have also been moving my BOINC Data folder to a different SSD (now back to its original SSD)

Sayling

This seems to be related to macOS 11.16.x

I came across this thread on Ask Different https://apple.stackexchange.com/questions/408154/macos-big-sur-virtualbox-error-the-virtual-machine-has-terminated-unexpectedly-d with a similar 'VirtualBox Error the virtual machine has terminated unexpectedly during startup with exit code 1 (0x1)' as I had experienced. I'm testing out the suggested solution (run csrutil, uninstall and reinstall VirtualBox) and it seems to be hopeful. My single CMS task has been running for 55mins where it was stopping after 10s before.

Sayling

Just to report my CMS task https://lhcathome.cern.ch/lhcathome/result.php?resultid=331660008 completed and validated.
Sayling

Just saw this lines in your logfile:

2021-11-02 08:31:20 (2996): Guest Log: [DEBUG] Proxy 127.0.0.1:3128 is set by BOINC but can't be contacted
.
.
.
2021-11-02 08:31:35 (2996): Guest Log: [INFO] 2.7.2.0 http://s1cern-cvmfs.openhtc.io DIRECT

This happens if you forward the wrong IP from your BOINC client to the CMS VM.
127.a.b.c or "localhost" must not be used since it is not the same outside and inside a VM.
See here:
https://lhcathome.cern.ch/lhcathome/forum_thread.php?id=5742&postid=45593
If your local name resolution correctly resolves name_of_your_proxy_box -> IP_of_your_proxy_box, use name_of_your_proxy_box in BOINC's proxy form.
You may instead use the local network IP of your proxy box, e.g. 192.168.f.g.

Thanks. I will change my BOINC manager proxy setting to the machine IP address as it is a fixed value assigned by my router.

And my recent tasks don't show that proxy error anymore.

The DEBUG message is only shown when there's a problem.
It's fine as long as the log looks like in this example:

2021-11-03 22:16:51 (83132): Guest Log: [INFO] Got a proxy from the local BOINC client
2021-11-03 22:16:51 (83132): Guest Log: [INFO] Will use it for CVMFS and Frontier
2021-11-03 22:16:51 (83132): Guest Log: [INFO] Reloading and probing the CVMFS configuration
2021-11-03 22:17:07 (83132): Guest Log: [INFO] Probing /cvmfs/cvmfs-config.cern.ch... OK
2021-11-03 22:17:07 (83132): Guest Log: [INFO] Probing /cvmfs/grid.cern.ch... OK
2021-11-03 22:17:14 (83132): Guest Log: [INFO] Probing /cvmfs/cms.cern.ch... OK
2021-11-03 22:17:15 (83132): Guest Log: [INFO] Probing /cvmfs/oasis.opensciencegrid.org... OK
2021-11-03 22:17:15 (83132): Guest Log: [INFO] Probing /cvmfs/singularity.opensciencegrid.org... OK
2021-11-03 22:17:15 (83132): Guest Log: [INFO] Probing /cvmfs/cms-ib.cern.ch... OK
2021-11-03 22:17:15 (83132): Guest Log: [INFO] Excerpt from "cvmfs_config stat": VERSION HOST PROXY
2021-11-03 22:17:15 (83132): Guest Log: [INFO] 2.7.2.0 http://s1ral-cvmfs.openhtc.io http://192.168.1.142:3128

Beside that a few thousand lines should be added to your proxy log within the first 20 minutes of a fresh task and when a running task gets a fresh subtask.

Today the computers made errors by computing CMS Tasks
I think that the origin of the problem desn't came from my computers but it is possible that the origin was CERN
I prepared my computers to avoid overload of the system
at the least control it showed a use of the processors of 100% but the use inside of the boinc manager is only 70% of processors
I will try to reduce the number of used processors inside of the boinc manager

I hope an incresed stability of the system

Sorry that i couldn't control further because i was very busy inside of my city

Sorry for desagrement

Best regards

Guy

I had some tasks fetched at around 09:00UTC 05Nov21 which failed with X509 errors - for example:

2021-11-05 09:15:25 (13849): Guest Log: [INFO] Reading volunteer information
2021-11-05 09:15:26 (13849): Guest Log: [INFO] Requesting an X509 credential from LHC@home
2021-11-05 09:15:26 (13849): Guest Log: [INFO] Requesting an X509 credential from vLHC@home-dev
2021-11-05 09:15:57 (13849): Guest Log: [INFO] Requesting an X509 credential from LHC@home
2021-11-05 09:15:57 (13849): Guest Log: [INFO] Requesting an X509 credential from vLHC@home-dev
2021-11-05 09:16:28 (13849): Guest Log: [INFO] Requesting an X509 credential from LHC@home
2021-11-05 09:16:29 (13849): Guest Log: [INFO] Requesting an X509 credential from vLHC@home-dev
2021-11-05 09:16:59 (13849): Guest Log: [INFO] Requesting an X509 credential from LHC@home
2021-11-05 09:17:00 (13849): Guest Log: [INFO] Requesting an X509 credential from vLHC@home-dev
2021-11-05 09:17:29 (13849): Guest Log: [INFO] Requesting an X509 credential from LHC@home
2021-11-05 09:17:30 (13849): Guest Log: [INFO] Requesting an X509 credential from vLHC@home-dev
2021-11-05 09:18:00 (13849): Guest Log: [INFO] Requesting an X509 credential from LHC@home
2021-11-05 09:18:01 (13849): Guest Log: [INFO] Requesting an X509 credential from vLHC@home-dev
2021-11-05 09:18:32 (13849): Guest Log: [DEBUG] % Total % Received % Xferd Average Speed Time Time Time Current
2021-11-05 09:18:32 (13849): Guest Log: Dload Upload Total Spent Left Speed
2021-11-05 09:18:32 (13849): Guest Log: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
2021-11-05 09:18:32 (13849): Guest Log: 100 531 100 531 0 0 1053 0 --:--:-- --:--:-- --:--:-- 1055
2021-11-05 09:18:32 (13849): Guest Log: [DEBUG]
2021-11-05 09:18:32 (13849): Guest Log: ERROR: Couldn't read proxy from: /tmp/x509up_u0
2021-11-05 09:18:32 (13849): Guest Log: globus_credential: Error reading proxy credential
2021-11-05 09:18:32 (13849): Guest Log: globus_credential: Error reading proxy credential: Couldn't read PEM from bio
2021-11-05 09:18:32 (13849): Guest Log: OpenSSL Error: pem_lib.c:707: in library: PEM routines, function PEM_read_bio: no start line
2021-11-05 09:18:32 (13849): Guest Log: Use -debug for further information.
2021-11-05 09:18:32 (13849): Guest Log: [ERROR] Could not get an x509 credential
2021-11-05 09:18:32 (13849): Guest Log: [ERROR] The x509 proxy creation failed.
2021-11-05 09:18:32 (13849): Guest Log: [DEBUG] Volunteer: Sayling Low (612456)
2021-11-05 09:18:33 (13849): Guest Log: [INFO] Shutting Down.
2021-11-05 09:19:03 (13849): Guest Log: [INFO] CMS application starting. Check log files.

These ran for quite short times (200-300s).
but I have a task fetched at about 22:50UTC 05Nov21 which is still running now. I am hopeful the problem has been fixed.
Sayling

This problem is caused by a crashed credential service at CERN.
Unfortunately it happens from time to time.
Volunteers can't do anything but to wait until that service is back again.

I'm getting a lot of computation errors for CMS vBox64 tasks on my Win10 host.
https://lhcathome.cern.ch/lhcathome/results.php?hostid=10687301&offset=0&show_names=0&state=6&appid=

I'm not sure where to start in troubleshooting this. My other Linux based hosts are doing ok (aside from sporadic comms issues with my FW and SNORT).
I'm running BOINC client 7.16.11 with vBox 6.1.12. Below is the app_config file from my Win10 host and the output of one of the failed tasks below that.

<app_config>
<!--
  <app>
    <name>ATLAS</name>
    <fraction_done_exact/>
  </app>
  <app_version>
    <app_name>ATLAS</app_name>
    <plan_class>vbox64_mt_mcore_atlas</plan_class>
    <avg_ncpus>4.0</avg_ncpus>
    <cmdline>>--nthreads 4 --memory_size_mb 3800</cmdline>
  </app_version>

  <app_version>
    <app_name>ATLAS</app_name>
    <plan_class>vbox64_mt_mcore_atlas</plan_class>
    <avg_ncpus>8.0</avg_ncpus>
    <cmdline>>--nthreads 8 --memory_size_mb 5000</cmdline>
  </app_version>
-->
<!--
  <app>
    <name>Theory</name>
    <fraction_done_exact/>
  </app>
  <app_version>
    <app_name>Theory</app_name>
    <plan_class>vbox64_theory</plan_class>
    <avg_ncpus>1.0</avg_ncpus>
    <cmdline>--nthreads 1</cmdline>
  </app_version>
-->
  <app>
    <name>CMS</name>
    <fraction_done_exact/>
  </app>
  <app_version>
    <app_name>CMS</app_name>
    <plan_class>vbox64</plan_class>
    <avg_ncpus>1.0</avg_ncpus>
    <cmdline>--nthreads 1 --memory_size_mb 2048</cmdline>
  </app_version>

</app_config>

Here is the task output from one of the failed tasks from my Win10 host.

<core_client_version>7.16.11</core_client_version>
<![CDATA[
<message>
The global filename characters, * or ?, are entered incorrectly or too many global filename characters are specified.
 (0xd0) - exit code 208 (0xd0)</message>
<stderr_txt>
2021-11-06 11:31:28 (16296): Detected: vboxwrapper 26202
2021-11-06 11:31:28 (16296): Detected: BOINC client v7.16.11
2021-11-06 11:31:28 (16296): Detected: VirtualBox VboxManage Interface (Version: 6.1.12)
2021-11-06 11:31:29 (16296): Detected: Heartbeat check (file: 'heartbeat' every 1200.000000 seconds)
2021-11-06 11:31:29 (16296): Successfully copied 'init_data.xml' to the shared directory.
2021-11-06 11:31:30 (16296): Create VM. (boinc_05d65e3058a799c6, slot#9)
2021-11-06 11:31:31 (16296): Setting Memory Size for VM. (2048MB)
2021-11-06 11:31:31 (16296): Setting CPU Count for VM. (1)
2021-11-06 11:31:31 (16296): Setting Chipset Options for VM.
2021-11-06 11:31:31 (16296): Setting Boot Options for VM.
2021-11-06 11:31:32 (16296): Setting Network Configuration for NAT.
2021-11-06 11:31:32 (16296): Enabling VM Network Access.
2021-11-06 11:31:32 (16296): Disabling USB Support for VM.
2021-11-06 11:31:32 (16296): Disabling COM Port Support for VM.
2021-11-06 11:31:33 (16296): Disabling LPT Port Support for VM.
2021-11-06 11:31:33 (16296): Disabling Audio Support for VM.
2021-11-06 11:31:33 (16296): Disabling Clipboard Support for VM.
2021-11-06 11:31:33 (16296): Disabling Drag and Drop Support for VM.
2021-11-06 11:31:34 (16296): Adding storage controller(s) to VM.
2021-11-06 11:31:34 (16296): Adding virtual disk drive to VM. (vm_image.vdi)
2021-11-06 11:31:34 (16296): Adding VirtualBox Guest Additions to VM.
2021-11-06 11:31:34 (16296): Adding network bandwidth throttle group to VM. (Defaulting to 1024GB)
2021-11-06 11:31:35 (16296): forwarding host port 49866 to guest port 80
2021-11-06 11:31:35 (16296): Enabling remote desktop for VM.
2021-11-06 11:31:35 (16296): Required extension pack not installed, remote desktop not enabled.
2021-11-06 11:31:35 (16296): Enabling shared directory for VM.
2021-11-06 11:31:36 (16296): Starting VM using VBoxManage interface. (boinc_05d65e3058a799c6, slot#9)
2021-11-06 11:31:39 (16296): Successfully started VM. (PID = '15392')
2021-11-06 11:31:39 (16296): Reporting VM Process ID to BOINC.
2021-11-06 11:31:39 (16296): Guest Log: BIOS: VirtualBox 6.1.12
2021-11-06 11:31:39 (16296): Guest Log: CPUID EDX: 0x178bfbff
2021-11-06 11:31:39 (16296): Guest Log: BIOS: ata0-0: PCHS=16383/16/63 LCHS=1024/255/63
2021-11-06 11:31:39 (16296): VM state change detected. (old = 'poweredoff', new = 'running')
2021-11-06 11:31:39 (16296): Detected: Web Application Enabled (http://localhost:49866)
2021-11-06 11:31:39 (16296): Preference change detected
2021-11-06 11:31:39 (16296): Setting CPU throttle for VM. (100%)
2021-11-06 11:31:40 (16296): Setting checkpoint interval to 600 seconds. (Higher value of (Preference: 600 seconds) or (Vbox_job.xml: 600 seconds))
2021-11-06 11:31:41 (16296): Guest Log: BIOS: Boot : bseqnr=1, bootseq=0032
2021-11-06 11:31:41 (16296): Guest Log: BIOS: Booting from Hard Disk...
2021-11-06 11:31:43 (16296): Guest Log: BIOS: KBD: unsupported int 16h function 03
2021-11-06 11:31:43 (16296): Guest Log: BIOS: AX=0305 BX=0000 CX=0000 DX=0000 
2021-11-06 11:32:03 (16296): Guest Log: vgdrvHeartbeatInit: Setting up heartbeat to trigger every 2000 milliseconds
2021-11-06 11:32:03 (16296): Guest Log: vboxguest: misc device minor 56, IRQ 20, I/O port d020, MMIO at 00000000f0400000 (size 0x400000)
2021-11-06 11:32:04 (16296): Guest Log: VBoxService 5.2.6 r120293 (verbosity: 0) linux.amd64 (Jan 15 2018 14:51:00) release log
2021-11-06 11:32:04 (16296): Guest Log: 00:00:00.000085 main     Log opened 2021-11-06T15:32:03.999422000Z
2021-11-06 11:32:04 (16296): Guest Log: 00:00:00.000168 main     OS Product: Linux
2021-11-06 11:32:04 (16296): Guest Log: 00:00:00.000193 main     OS Release: 4.14.232-19.cernvm.x86_64
2021-11-06 11:32:04 (16296): Guest Log: 00:00:00.000215 main     OS Version: #1 SMP Fri Apr 30 17:12:25 CEST 2021
2021-11-06 11:32:04 (16296): Guest Log: 00:00:00.000247 main     Executable: /usr/sbin/VBoxService
2021-11-06 11:32:04 (16296): Guest Log: 00:00:00.000247 main     Process ID: 2153
2021-11-06 11:32:04 (16296): Guest Log: 00:00:00.000248 main     Package type: LINUX_64BITS_GENERIC
2021-11-06 11:32:04 (16296): Guest Log: 00:00:00.001532 main     5.2.6 r120293 started. Verbose level = 0
2021-11-06 11:32:13 (16296): Guest Log: [INFO] Mounting the shared directory
2021-11-06 11:32:13 (16296): Guest Log: [INFO] Shared directory mounted, enabling vboxmonitor
2021-11-06 11:32:13 (16296): Guest Log: [INFO] Sourcing essential functions from /cvmfs/grid.cern.ch
2021-11-06 11:32:13 (16296): Guest Log: [INFO] Testing connection to cern.ch
2021-11-06 11:32:13 (16296): Guest Log: [INFO] Testing connection to VCCS
2021-11-06 11:32:13 (16296): Guest Log: [INFO] Testing connection to HTCondor
2021-11-06 11:32:13 (16296): Guest Log: [INFO] Testing connection to WMAgent
2021-11-06 11:32:14 (16296): Guest Log: [INFO] Testing connection to Frontier
2021-11-06 11:32:14 (16296): Guest Log: [INFO] Got a proxy from the local BOINC client
2021-11-06 11:32:14 (16296): Guest Log: [INFO] Will use it for CVMFS and Frontier
2021-11-06 11:32:14 (16296): Guest Log: [INFO] Reloading and probing the CVMFS configuration
2021-11-06 11:32:18 (16296): Guest Log: [INFO] Probing /cvmfs/cvmfs-config.cern.ch... OK
2021-11-06 11:32:18 (16296): Guest Log: [INFO] Probing /cvmfs/grid.cern.ch... OK
2021-11-06 11:32:20 (16296): Guest Log: [INFO] Probing /cvmfs/oasis.opensciencegrid.org... OK
2021-11-06 11:32:20 (16296): Guest Log: [INFO] Probing /cvmfs/singularity.opensciencegrid.org... OK
2021-11-06 11:32:20 (16296): Guest Log: [INFO] Probing /cvmfs/cms-ib.cern.ch... OK
2021-11-06 11:32:20 (16296): Guest Log: [INFO] Probing /cvmfs/cms.cern.ch... OK
2021-11-06 11:32:20 (16296): Guest Log: [INFO] Excerpt from "cvmfs_config stat": VERSION HOST PROXY
2021-11-06 11:32:20 (16296): Guest Log: [INFO] 2.7.2.0 http://s1bnl-cvmfs.openhtc.io http://192.168.150.1:3128
2021-11-06 11:32:20 (16296): Guest Log: [INFO] Reading volunteer information
2021-11-06 11:32:21 (16296): Guest Log: [INFO] Requesting an X509 credential from LHC@home
2021-11-06 11:32:22 (16296): Guest Log: [INFO] CMS application starting. Check log files.
2021-11-06 11:52:22 (16296): Guest Log: [ERROR] glidein exited with return value 1.
2021-11-06 11:52:22 (16296): Guest Log: [DEBUG] Volunteer: scotth (787857)
2021-11-06 11:52:22 (16296): Guest Log: [INFO] Shutting Down.
2021-11-06 11:52:52 (16296): VM Completion File Detected.
2021-11-06 11:52:52 (16296): VM Completion Message: glidein exited with return value 1.
.
2021-11-06 11:52:52 (16296): Powering off VM.
2021-11-06 11:52:52 (16296): Successfully stopped VM.
2021-11-06 11:52:52 (16296): Deregistering VM. (boinc_05d65e3058a799c6, slot#9)
2021-11-06 11:52:52 (16296): Removing network bandwidth throttle group from VM.
2021-11-06 11:52:53 (16296): Removing VM from VirtualBox.
11:52:58 (16296): called boinc_finish(208)

</stderr_txt>
]]>

Some ideas:

Some of the logs show different vbox/BOINC versions.
Did you upgrade/downgrade something and forgot to reboot?

<message>
The global filename characters, * or ?, are entered incorrectly or too many global filename characters are specified.
(0xd0) - exit code 208 (0xd0)</message>
Where is the BOINC directory located?
Does it lead to a too long expanded path name?

This CMS-Task was running successfull:
2021-11-02 11:39:45 (1556): Detected: BOINC client v7.16.20
2021-11-02 11:39:46 (1556): Detected: VirtualBox VboxManage Interface (Version: 6.1.26)
Was there a change of Boinc or Virtualbox?
https://lhcathome.cern.ch/lhcathome/results.php?hostid=10687301&offset=0&show_names=0&state=4&appid=
Are there yellow triangle shown in Virtualbox VM-Filemanager?

I did downgrade the BOINC and vBox versions while attempting to troubleshoot the runaway task downloads I was encountering. I found, through the Rosetta@home forum, that BOINC has issues with calculating task queue depth with the <max_concurrent> or <project_max_concurrent> flags in project app_config files. I thought it could have been the upgrade of BOINC and vBox I did a while back, so I downgraded them. I've also removed those settings from my Win10 host app_config.

I have rebooted since then and just now had my client upload more failed results to LHC@home. This task is one of the ones that failed just in the last 15-20 minutes.
https://lhcathome.cern.ch/lhcathome/result.php?result_name=CMS_3108593_1635827713.939210_0
I've also restricted the project from getting new tasks for the time being until I can get this fixed.

R/S
Scott

The main BOINC directory is at

C:\Program Files\BOINC

with the data being at

C:\ProgramData\BOINC

Virtual box is at

C:\Users\Scott\.VirtualBox

Looking into the vBox logs, I do see that it's pointing out the differing versions and lack of permissions to the vm storage. Could it not have liked the software downgrade? I'll download the current versions again and reinstall them.

03:59:21.738008          Saving settings file "C:\ProgramData\BOINC\slots\9\boinc_00ca13701b3d268d\boinc_00ca13701b3d268d.vbox" with version "1.16-windows"
03:59:21.886440          ERROR [COM]: aRC=E_FAIL (0x80004005) aIID={85632c68-b5bb-4316-a900-5eb28d3413df} aComponent={SessionMachine} aText={This machine does not have any snapshots}, preserve=false aResultDetail=0
03:59:21.926647          Saving settings file "C:\ProgramData\BOINC\slots\9\boinc_00ca13701b3d268d\boinc_00ca13701b3d268d.vbox" with version "1.16-windows"
03:59:22.180939          Saving settings file "C:\Users\Scott\.VirtualBox\VirtualBox.xml" with version "1.12-windows"
03:59:28.134972          ERROR [COM]: aRC=VBOX_E_OBJECT_NOT_FOUND (0x80bb0001) aIID={d0a0163f-e254-4e5b-a1f2-011cf991c38d} aComponent={VirtualBoxWrap} aText={Could not find a registered machine named 'boinc_c84b3e5bbd6d461e'}, preserve=false aResultDetail=0
03:59:28.389054          ERROR [COM]: aRC=E_ACCESSDENIED (0x80070005) aIID={ad47ad09-787b-44ab-b343-a082a3f2dfb1} aComponent={MediumWrap} aText={The object is not ready}, preserve=false aResultDetail=0
03:59:28.725563          ERROR [COM]: aRC=E_ACCESSDENIED (0x80070005) aIID={ad47ad09-787b-44ab-b343-a082a3f2dfb1} aComponent={MediumWrap} aText={The object is not ready}, preserve=false aResultDetail=0
03:59:28.893063          ERROR [COM]: aRC=E_ACCESSDENIED (0x80070005) aIID={ad47ad09-787b-44ab-b343-a082a3f2dfb1} aComponent={MediumWrap} aText={The object is not ready}, preserve=false aResultDetail=0
03:59:29.647620          ERROR [COM]: aRC=E_ACCESSDENIED (0x80070005) aIID={ad47ad09-787b-44ab-b343-a082a3f2dfb1} aComponent={MediumWrap} aText={The object is not ready}, preserve=false aResultDetail=0
03:59:30.054998          Saving settings file "C:\Users\Scott\.VirtualBox\VirtualBox.xml" with version "1.12-windows"
03:59:30.161742          Saving settings file "C:\ProgramData\BOINC\slots\8\boinc_c84b3e5bbd6d461e\boinc_c84b3e5bbd6d461e.vbox" with version "1.16-windows"
03:59:30.172140          Saving settings file "C:\Users\Scott\.VirtualBox\VirtualBox.xml" with version "1.12-windows"
03:59:30.418205          Saving settings file "C:\ProgramData\BOINC\slots\8\boinc_c84b3e5bbd6d461e\boinc_c84b3e5bbd6d461e.vbox" with version "1.16-windows"
03:59:30.703352          Saving settings file "C:\ProgramData\BOINC\slots\8\boinc_c84b3e5bbd6d461e\boinc_c84b3e5bbd6d461e.vbox" with version "1.16-windows"
03:59:30.962257          Saving settings file "C:\ProgramData\BOINC\slots\8\boinc_c84b3e5bbd6d461e\boinc_c84b3e5bbd6d461e.vbox" with version "1.16-windows"
03:59:31.220827          Saving settings file "C:\ProgramData\BOINC\slots\8\boinc_c84b3e5bbd6d461e\boinc_c84b3e5bbd6d461e.vbox" with version "1.16-windows"
03:59:31.493533          Saving settings file "C:\ProgramData\BOINC\slots\8\boinc_c84b3e5bbd6d461e\boinc_c84b3e5bbd6d461e.vbox" with version "1.16-windows"
03:59:31.769940          Saving settings file "C:\ProgramData\BOINC\slots\8\boinc_c84b3e5bbd6d461e\boinc_c84b3e5bbd6d461e.vbox" with version "1.16-windows"
03:59:32.026155          Saving settings file "C:\ProgramData\BOINC\slots\8\boinc_c84b3e5bbd6d461e\boinc_c84b3e5bbd6d461e.vbox" with version "1.16-windows"
03:59:32.284644          Saving settings file "C:\ProgramData\BOINC\slots\8\boinc_c84b3e5bbd6d461e\boinc_c84b3e5bbd6d461e.vbox" with version "1.16-windows"
03:59:32.544079          Saving settings file "C:\ProgramData\BOINC\slots\8\boinc_c84b3e5bbd6d461e\boinc_c84b3e5bbd6d461e.vbox" with version "1.16-windows"
03:59:32.800604          Saving settings file "C:\ProgramData\BOINC\slots\8\boinc_c84b3e5bbd6d461e\boinc_c84b3e5bbd6d461e.vbox" with version "1.16-windows"
03:59:33.059791          Saving settings file "C:\ProgramData\BOINC\slots\8\boinc_c84b3e5bbd6d461e\boinc_c84b3e5bbd6d461e.vbox" with version "1.16-windows"
03:59:33.316373          Saving settings file "C:\ProgramData\BOINC\slots\8\boinc_c84b3e5bbd6d461e\boinc_c84b3e5bbd6d461e.vbox" with version "1.16-windows"
03:59:33.577000          Saving settings file "C:\ProgramData\BOINC\slots\8\boinc_c84b3e5bbd6d461e\boinc_c84b3e5bbd6d461e.vbox" with version "1.16-windows"
03:59:33.833723          Saving settings file "C:\ProgramData\BOINC\slots\8\boinc_c84b3e5bbd6d461e\boinc_c84b3e5bbd6d461e.vbox" with version "1.16-windows"
03:59:34.091140          ERROR [COM]: aRC=VBOX_E_OBJECT_NOT_FOUND (0x80bb0001) aIID={85632c68-b5bb-4316-a900-5eb28d3413df} aComponent={SessionMachine} aText={No storage device attached to device slot 0 on port 0 of controller 'Hard Disk Controller'}, preserve=false aResultDetail=0
03:59:34.091339          ERROR [COM]: aRC=E_ACCESSDENIED (0x80070005) aIID={ad47ad09-787b-44ab-b343-a082a3f2dfb1} aComponent={MediumWrap} aText={The object is not ready}, preserve=false aResultDetail=0
03:59:34.995533          ERROR [COM]: aRC=E_ACCESSDENIED (0x80070005) aIID={ad47ad09-787b-44ab-b343-a082a3f2dfb1} aComponent={MediumWrap} aText={The object is not ready}, preserve=false aResultDetail=0
03:59:35.100846          ERROR [COM]: aRC=VBOX_E_OBJECT_NOT_FOUND (0x80bb0001) aIID={85632c68-b5bb-4316-a900-5eb28d3413df} aComponent={SessionMachine} aText={No storage device attached to device slot 0 on port 0 of controller 'Hard Disk Controller'}, preserve=false aResultDetail=0
03:59:35.101101          Saving settings file "C:\Users\Scott\.VirtualBox\VirtualBox.xml" with version "1.12-windows"
03:59:35.117080          Saving settings file "C:\ProgramData\BOINC\slots\8\boinc_c84b3e5bbd6d461e\boinc_c84b3e5bbd6d461e.vbox" with version "1.16-windows"
03:59:35.349780          ERROR [COM]: aRC=VBOX_E_OBJECT_NOT_FOUND (0x80bb0001) aIID={85632c68-b5bb-4316-a900-5eb28d3413df} aComponent={SessionMachine} aText={No storage device attached to device slot 0 on port 1 of controller 'Hard Disk Controller'}, preserve=false aResultDetail=0
03:59:35.350066          ERROR [COM]: aRC=VBOX_E_OBJECT_NOT_FOUND (0x80bb0001) aIID={85632c68-b5bb-4316-a900-5eb28d3413df} aComponent={SessionMachine} aText={No storage device attached to device slot 0 on port 1 of controller 'Hard Disk Controller'}, preserve=false aResultDetail=0
03:59:35.353458          Saving settings file "C:\ProgramData\BOINC\slots\8\boinc_c84b3e5bbd6d461e\boinc_c84b3e5bbd6d461e.vbox" with version "1.16-windows"
03:59:35.605351          Saving settings file "C:\ProgramData\BOINC\slots\8\boinc_c84b3e5bbd6d461e\boinc_c84b3e5bbd6d461e.vbox" with version "1.16-windows"
03:59:35.866400          Saving settings file "C:\ProgramData\BOINC\slots\8\boinc_c84b3e5bbd6d461e\boinc_c84b3e5bbd6d461e.vbox" with version "1.16-windows"
03:59:36.375597          Saving settings file "C:\ProgramData\BOINC\slots\8\boinc_c84b3e5bbd6d461e\boinc_c84b3e5bbd6d461e.vbox" with version "1.16-windows"
03:59:36.630528          ERROR [COM]: aRC=VBOX_E_OBJECT_NOT_FOUND (0x80bb0001) aIID={85632c68-b5bb-4316-a900-5eb28d3413df} aComponent={SessionMachine} aText={No storage device attached to device slot 1 on port 0 of controller 'Hard Disk Controller'}, preserve=false aResultDetail=0
03:59:36.630634          ERROR [COM]: aRC=VBOX_E_OBJECT_NOT_FOUND (0x80bb0001) aIID={85632c68-b5bb-4316-a900-5eb28d3413df} aComponent={SessionMachine} aText={No storage device attached to device slot 1 on port 0 of controller 'Hard Disk Controller'}, preserve=false aResultDetail=0
03:59:36.631122          ERROR [COM]: aRC=VBOX_E_OBJECT_NOT_FOUND (0x80bb0001) aIID={85632c68-b5bb-4316-a900-5eb28d3413df} aComponent={SessionMachine} aText={No storage device attached to device slot 1 on port 1 of controller 'Hard Disk Controller'}, preserve=false aResultDetail=0
03:59:36.631188          ERROR [COM]: aRC=VBOX_E_OBJECT_NOT_FOUND (0x80bb0001) aIID={85632c68-b5bb-4316-a900-5eb28d3413df} aComponent={SessionMachine} aText={No storage device attached to device slot 1 on port 1 of controller 'Hard Disk Controller'}, preserve=false aResultDetail=0
03:59:36.635096          ERROR [COM]: aRC=E_FAIL (0x80004005) aIID={85632c68-b5bb-4316-a900-5eb28d3413df} aComponent={SessionMachine} aText={This machine does not have any snapshots}, preserve=false aResultDetail=0
03:59:36.886577          Launched VM: 79687136 pid: 10988 (0x2aec) frontend: headless name: boinc_c84b3e5bbd6d461e
03:59:37.339774          ERROR [COM]: aRC=VBOX_E_OBJECT_NOT_FOUND (0x80bb0001) aIID={d0a0163f-e254-4e5b-a1f2-011cf991c38d} aComponent={VirtualBoxWrap} aText={Could not find a registered machine named 'boinc_0665d7a71cb3cdc3'}, preserve=false aResultDetail=0
03:59:37.591751          ERROR [COM]: aRC=E_ACCESSDENIED (0x80070005) aIID={ad47ad09-787b-44ab-b343-a082a3f2dfb1} aComponent={MediumWrap} aText={The object is not ready}, preserve=false aResultDetail=0
03:59:37.929053          ERROR [COM]: aRC=E_ACCESSDENIED (0x80070005) aIID={ad47ad09-787b-44ab-b343-a082a3f2dfb1} aComponent={MediumWrap} aText={The object is not ready}, preserve=false aResultDetail=0
03:59:38.100938          ERROR [COM]: aRC=E_ACCESSDENIED (0x80070005) aIID={ad47ad09-787b-44ab-b343-a082a3f2dfb1} aComponent={MediumWrap} aText={The object is not ready}, preserve=false aResultDetail=0

When you downgrade, be sure to reboot after installation. What's about your extensionpack of Virtualbox?
You can deinstall Virtualbox and reboot your Windows. After this you can install a clean Installation of Virtualbox.
When the trouble is always, on Boinc-homepage is a combination of Boinc/Virtualbox for Installation.
From this point you can do a upgrade of Boinc and/or Virtualbox.
https://boinc.berkeley.edu/download_all.php

I removed vBox 6.1.12, rebooted, reinstalled 6.1.12 and the issue persists. Since the vBox logs still show notices for a version difference from 6.1.12 and 6.1.16, I uninstalled 6.1.12 and installed 6.1.16. Rebooting and ensuring all the system files were gone.

I do not have any extension packs configured for use. Do I need to configure that? I've previously read that the vBox extension pack was not needed. Is that incorrect?

I'm waiting for a few tasks to download to see if the issue persists in the vBox logs. If so, I'll open a new thread under the Windows board.

Yes you need to install the extensionpack from your installed Virtualbox-Version or higher.

I do not have any extension packs configured for use. Do I need to configure that? I've previously read that the vBox extension pack was not needed. Is that incorrect? .

well, you don't need to install the extension pack necessarily. Virtual Box also runs without the extension pack. However, the extension pack gives you the chance to "look inside" a VM task while running, thus giving you some more information.
There is no need to configure the extension pack. Just install it. From then on, in the BOINC manager, on the left hand side, the button "show VM console" is no longer greyed out, and you can click on it.
Once opened, with the keys F1, F2 and F3 you can retrieve various information about the running task. What exactly you see, varies between the sub-tasks. So in ATLAS you see different things than in Theory and in CMS. Find out yourself, it's pretty interesting and informative.

https://lhcathome.cern.ch/lhcathome/results.php?hostid=10408749&offset=0&show_names=0&state=6&appid=
Have a crash on one PC two days ago. After restarting all CMS-Tasks, they doesn't starting correct and finished with Error.
2022-03-30 21:42:46 (11896): Status Report: Job Duration: '64800.000000'
2022-03-30 21:42:46 (11896): Status Report: Elapsed Time: '12000.000000'
2022-03-30 21:42:46 (11896): Status Report: CPU Time: '11682.312500'
2022-03-31 04:51:54 (5528): Detected: vboxwrapper 26202
2022-03-31 04:51:54 (5528): Detected: BOINC client v7.16.20
2022-03-31 04:52:03 (5528): Error in guest additions for VM: -2147024809
Command:
VBoxManage -q list systemproperties
Output:
VBoxManage.exe: error: Failed to create the VirtualBox object!
VBoxManage.exe: error: Cannot register the hard disk 'S:\ProgramData\BOINC\slots\1\vm_image.vdi' {7db93f2c-e3c3-41b6-acdb-54f59c57e05d} because a hard disk 'S:\ProgramData\BOINC\slots\1\vm_image.vdi' with UUID {f38d867d-442c-4d84-b8a3-932556102fad} already exists
VBoxManage.exe: error: Details: code E_INVALIDARG (0x80070057), component VirtualBoxWrap, interface IVirtualBox

2022-03-31 04:52:03 (5528): Detected: VirtualBox VboxManage Interface (Version: 6.1.28)
2022-03-31 04:52:09 (5528): Error in host info for VM: -2147024809
Command:
VBoxManage -q list hostinfo
Output:
VBoxManage.exe: error: Failed to create the VirtualBox object!
VBoxManage.exe: error: Cannot register the hard disk 'S:\ProgramData\BOINC\slots\1\vm_image.vdi' {7db93f2c-e3c3-41b6-acdb-54f59c57e05d} because a hard disk 'S:\ProgramData\BOINC\slots\1\vm_image.vdi' with UUID {f38d867d-442c-4d84-b8a3-932556102fad} already exists
VBoxManage.exe: error: Details: code E_INVALIDARG (0x80070057), component VirtualBoxWrap, interface IVirtualBox

2022-03-31 04:52:09 (5528): WARNING: Communication with VM Hypervisor failed.
2022-03-31 04:52:09 (5528): ERROR: VBoxManage list hostinfo failed
04:52:09 (5528): called boinc_finish(1)