Message boards : Theory Application : Poor CPU utilization on High Core Count CPUs during Theory Application
Message board moderation

To post messages, you must log in.

AuthorMessage
groberts

Send message
Joined: 24 Oct 19
Posts: 1
Credit: 867,329
RAC: 0
Message 40292 - Posted: 27 Oct 2019, 0:14:46 UTC

I am running an AMD Ryzen Threadripper 2950X (16c/32t) and works great when running all other tasks, except the Theory Application, as the CPU is being fully utilized. However, when running the Theory Application, the BIONC scheduler is allocated all 32 threads to this task and is only using around 10-15% of the entire CPU. This is very sub-optimal as the rest of the CPU should be allocated to running other tasks like SixTrack, etc. Unfortunately, the BIONC application doesn't allow the user to redefine the number of threads a given Application should use. For example, it would be nice to configure each Theory Application to use only 4 threads each, SixTrack Applications to use only 1 thread each, etc.

The problem is not that there is not enough RAM as I have plenty of RAM not being utilized during these applications.

Has anyone found a workaround to this for high core-count CPUs?
ID: 40292 · Report as offensive     Reply Quote
computezrmle
Volunteer moderator
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 15 Jun 08
Posts: 1439
Credit: 74,001,195
RAC: 119,064
Message 40296 - Posted: 27 Oct 2019, 8:02:28 UTC - in response to Message 40292.  

You may set the max # cores at the preferences page as low as possible.
Pitfall: This conflicts with ATLAS if you run it concurrently as a setting of n would allow you to download only n ATLAS tasks.
(ATLAS is the bad guy in this scenario)

To give a more detailed answer it would be required to examine the logs.
Hence you may make your computers visible for other users, also on the preferences page linked above.
ID: 40296 · Report as offensive     Reply Quote
Toby Broom
Volunteer moderator

Send message
Joined: 27 Sep 08
Posts: 597
Credit: 371,400,015
RAC: 28,836
Message 40297 - Posted: 27 Oct 2019, 10:23:55 UTC

I use for everything 1 core then I can get 100% load, the problem is then you need lots of RAM, my computer have 64GB or more. You could run 2 cores to save some RAM as you don't need much more ram for more cores.

Also I found that VirtualBox crashes if you run 1:1 threads:WU, so you have to do some tweaking to get the load OK

The maximum number of ATLAS WU is 12 at once.
ID: 40297 · Report as offensive     Reply Quote
Guy PF Masevaux

Send message
Joined: 9 Sep 19
Posts: 32
Credit: 2,856,470
RAC: 132
Message 40310 - Posted: 29 Oct 2019, 12:00:37 UTC - in response to Message 40292.  

If you have an AMD Ryzen threadripper, then i want to know what you made for running atlas without Failure?


I installed yeti's cheklist and it is impossible to run atlas on my AMD Ryzen 7 2700X
I can run six track and Nothing else
Something must be wrong but i do not know what?
I have no problem with intel skylake processor with yeti's cheklist

My AMD Ryzen have 32 GB Ram and all my other computers have 16 Gb RAM
Also i tested the AMD with or without Hyper V and at everytime i have Failure. Also with Theory running.
I think there exist a bug.
But AMD running exist in each applys:


CMS Simulation
Plateforme
Version
Créé
Calcul moyen
Microsoft Windows running on an AMD x86_64 or Intel EM64T CPU
49.00 (vbox64)
26 Mar 2019, 8:56:04 UTC
267 GigaFLOPS
Linux running on an AMD x86_64 or Intel EM64T CPU
49.00 (vbox64)
26 Mar 2019, 9:00:17 UTC
244 GigaFLOPS
Intel 64-bit Mac OS 10.5 or later
49.00 (vbox64)
26 Mar 2019, 8:55:02 UTC
17 GigaFLOPS
Theory Simulation
Plateforme
Version
Créé
Calcul moyen
Microsoft Windows (98 or later) running on an Intel x86-compatible CPU
263.50 (vbox32)
8 Oct 2017, 10:47:42 UTC
310 GigaFLOPS
Microsoft Windows running on an AMD x86_64 or Intel EM64T CPU
263.98 (vbox64_mt_mcore)
16 Jul 2019, 14:18:50 UTC
9,667 GigaFLOPS
Linux running on an AMD x86_64 or Intel EM64T CPU
263.98 (vbox64_mt_mcore)
16 Jul 2019, 14:19:04 UTC
4,782 GigaFLOPS
Intel 64-bit Mac OS 10.5 or later
263.98 (vbox64_mt_mcore)
16 Jul 2019, 14:19:16 UTC
358 GigaFLOPS
ATLAS Simulation
Plateforme
Version
Créé
Calcul moyen
Microsoft Windows running on an AMD x86_64 or Intel EM64T CPU
2.00 (vbox64_mt_mcore_atlas)
9 Oct 2019, 8:30:17 UTC
3,128 GigaFLOPS
Linux running on an AMD x86_64 or Intel EM64T CPU
2.00 (vbox64_mt_mcore_atlas)
9 Oct 2019, 8:30:50 UTC
806 GigaFLOPS
Linux running on an AMD x86_64 or Intel EM64T CPU
2.73 (native_mt) (beta test)
17 Oct 2019, 12:21:07 UTC
27,575 GigaFLOPS
Intel 64-bit Mac OS 10.5 or later
2.00 (vbox64_mt_mcore_atlas)
9 Oct 2019, 8:31:15 UTC
138 GigaFLOPS

I hope an Answer

Guy PFLIEGER
ID: 40310 · Report as offensive     Reply Quote
Guy PF Masevaux

Send message
Joined: 9 Sep 19
Posts: 32
Credit: 2,856,470
RAC: 132
Message 40311 - Posted: 29 Oct 2019, 12:18:49 UTC - in response to Message 40310.  

this is a sample of errors with AMD Ryzen
For errors exist usually answers.


Nom
Theory_1729189_1572308367.834984_0
Unité de travail (WU)
125694646
Créé
29 Oct 2019, 0:19:29 UTC
Envoyé
29 Oct 2019, 5:52:44 UTC
Date limite de rapport
29 Nov 2019, 5:52:44 UTC
Reçu
29 Oct 2019, 5:54:21 UTC


2019-10-29 06:52:50 (14168):
Command: VBoxManage -q showvminfo "boinc_1ad8e5bfb0f350ca" --machinereadable
Exit Code: -2135228415
Output:
VBoxManage.exe: error: Could not find a registered machine named 'boinc_1ad8e5bfb0f350ca'
VBoxManage.exe: error: Details: code VBOX_E_OBJECT_NOT_FOUND (0x80bb0001), component VirtualBoxWrap, interface IVirtualBox, callee IUnknown
VBoxManage.exe: error: Context: "FindMachine(Bstr(VMNameOrUuid).raw(), machine.asOutParam())" at line 2621 of file VBoxManageInfo.cpp

VBoxManage.exe: error: Not in a hypervisor partition (HVP=0) (VERR_NEM_NOT_AVAILABLE).
VBoxManage.exe: error: AMD-V is disabled in the BIOS (or by the host OS) (VERR_SVM_DISABLED)
VBoxManage.exe: error: Details: code E_FAIL (0x80004005), component ConsoleWrap, interface IConsole

This machine does not have any snapshots
2019-10-29 06:53:03 (14168):
Command: VBoxManage -q bandwidthctl "boinc_1ad8e5bfb0f350ca" remove "boinc_1ad8e5bfb0f350ca_net"
Exit Code: 0
Output:
VBoxManage.exe: error: Bandwidth groups cannot be deleted while the VM is running


I cross my fingers
Sorry, but i must go to médicine between 13h20 to 15h30 French time

Good afternoon everybody
Guy PFLIEGER
ID: 40311 · Report as offensive     Reply Quote
Guy PF Masevaux

Send message
Joined: 9 Sep 19
Posts: 32
Credit: 2,856,470
RAC: 132
Message 40312 - Posted: 29 Oct 2019, 12:27:09 UTC - in response to Message 40311.  

I Just received an answer: they are changing the Windows of the buro at the hospital and they can not receive us until next time.

My wife confirmed it

Guy PFLIEGER
ID: 40312 · Report as offensive     Reply Quote
Crystal Pellet
Volunteer moderator
Volunteer tester

Send message
Joined: 14 Jan 10
Posts: 942
Credit: 6,295,475
RAC: 1,091
Message 40314 - Posted: 29 Oct 2019, 14:30:41 UTC - in response to Message 40310.  

My AMD Ryzen have 32 GB Ram and all my other computers have 16 Gb RAM
Also i tested the AMD with or without Hyper V and at everytime i have Failure. Also with Theory running.
I think there exist a bug.
But AMD running exist in each applys:

In one of your Theory results:
VBoxManage.exe: error: Not in a hypervisor partition (HVP=0) (VERR_NEM_NOT_AVAILABLE).
VBoxManage.exe: error: AMD-V is disabled in the BIOS (or by the host OS) (VERR_SVM_DISABLED)
VBoxManage.exe: error: Details: code E_FAIL (0x80004005), component ConsoleWrap, interface IConsole

2019-10-29 07:05:23 (1860): VM failed to start.
2019-10-29 07:05:23 (1860): Could not start
2019-10-29 07:05:23 (1860): ERROR: VM failed to start
ID: 40314 · Report as offensive     Reply Quote
Guy PF Masevaux

Send message
Joined: 9 Sep 19
Posts: 32
Credit: 2,856,470
RAC: 132
Message 40327 - Posted: 30 Oct 2019, 7:27:43 UTC - in response to Message 40314.  

Yes!
This was the answer!
I searched on internet how to modify this problem and i found a detailled answer
I made the change like explained on the website
Then i modified the boot
Now the virtualization is fully recognized and the hyper V too
I installed the latest virtual box and the latest extension pack
I cleaned the computer and optimized the registry
Then I restarted
Now the computer needs a lot less memory and is able to work at full power

Greeting You!

Guy PFLIEGER
ID: 40327 · Report as offensive     Reply Quote
Guy PF Masevaux

Send message
Joined: 9 Sep 19
Posts: 32
Credit: 2,856,470
RAC: 132
Message 40338 - Posted: 30 Oct 2019, 12:40:32 UTC - in response to Message 40327.  

The virtualisation of amd ryzen is now activated
I can send the proof to a mail box of cern

Guy PFLIEGER
ID: 40338 · Report as offensive     Reply Quote
computezrmle
Volunteer moderator
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 15 Jun 08
Posts: 1439
Credit: 74,001,195
RAC: 119,064
Message 40339 - Posted: 30 Oct 2019, 13:01:18 UTC - in response to Message 40338.  

Your computer page
https://lhcathome.cern.ch/lhcathome/show_host_detail.php?hostid=10611285
mentions:
Virtualization 	Virtualbox (6.0.12) installed, CPU does not have hardware virtualization support

Did you recently contact the project server to update that record?

In addition you mentioned in an earlier post:
Now the virtualization is fully recognized and the hyper V too

Did you really mean Microsoft's Hyper-V hypervisor is activated?

If yes I would be surprised if any vbox task would be able to run as
- it looks like hardware virtualization is still not activated
- Hyper-V must be switched off as it conflicts with VirtualBox
ID: 40339 · Report as offensive     Reply Quote

Message boards : Theory Application : Poor CPU utilization on High Core Count CPUs during Theory Application


©2020 CERN