Message boards : Number crunching : Vbox error that I have never seen before
Message board moderation

To post messages, you must log in.

1 · 2 · Next

AuthorMessage
greg_be

Send message
Joined: 28 Dec 08
Posts: 318
Credit: 4,148,677
RAC: 2,010
Message 36089 - Posted: 27 Jul 2018, 16:58:51 UTC
Last modified: 27 Jul 2018, 17:03:06 UTC

A bunch of Atlas and Theory tasks failed with this error.

https://lhcathome.cern.ch/lhcathome/result.php?resultid=200977168

2018-07-23 12:33:48 (5176): vboxwrapper (7.7.26196): starting
2018-07-23 12:33:49 (5176): Error creating VirtualBox instance! rc = 0x80004002
2018-07-23 12:33:49 (5176): Feature: Checkpoint interval offset (221 seconds)
2018-07-23 12:33:49 (5176): Detected: VirtualBox VboxManage Interface (Version: 5.2.16)

then more blah blah blah and then this
2018-07-23 12:33:57 (5176): Hardware acceleration CPU extensions not detected. Disabling VirtualBox hardware acceleration support.
2018-07-23 12:33:57 (5176): Hardware acceleration failed with previous execution. Disabling VirtualBox hardware acceleration support.
2018-07-23 12:33:57 (5176): ERROR: Invalid configuration. VM type requires acceleration but the current configuration cannot support it.
2018-07-23 12:33:57 (5176):
NOTE: VirtualBox has reported an improperly configured virtual machine. It was configured to require
hardware acceleration for virtual machines, but your processor does not support the required feature.
Please report this issue to the project so that it can be addresssed.
Error Code: ERR_CPU_VM_EXTENSIONS_DISABLED

I checked that Virtual Machine or whatever in the BIOS and that is enabled. It has been since I just had to get a new SSD with windows a few days earlier.
So what is going on that VM can not run from theory and atlas tasks?
ID: 36089 · Report as offensive     Reply Quote
computezrmle
Volunteer moderator
Volunteer developer
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 15 Jun 08
Posts: 2386
Credit: 222,902,029
RAC: 138,039
Message 36090 - Posted: 27 Jul 2018, 17:26:24 UTC - in response to Message 36089.  

Quick checks:
- VT-x/AMD-v is enabled in the BIOS (you wrote that you already checked it)
- <p_vm_extensions_disabled> is absent or set to "0" in your client_state.xml
- Other hypervisors are disabled (new windows)
- Are you sure you really want to run your VMs with 8 cores each?
- Did all your VMs start concurrently
- Check if your VirtualBox environment needs to be cleaned


You may find more checks in Yeti's checklist:
https://lhcathome.cern.ch/lhcathome/forum_thread.php?id=4161&postid=29359
ID: 36090 · Report as offensive     Reply Quote
greg_be

Send message
Joined: 28 Dec 08
Posts: 318
Credit: 4,148,677
RAC: 2,010
Message 36091 - Posted: 27 Jul 2018, 18:35:52 UTC - in response to Message 36090.  

p_vm_extensions_disabled was set to 1 changed to 0
possible VBox extensions were not installed, downloaded and installed.
I run wide open with 8 cores for all tasks from all projects that need them.
This is more a crunching machine with background web browsing on occasion.
The setup works.

Will see if everything runs ok now. Have to chew through a bunch of Sixtrack stuff first.
ID: 36091 · Report as offensive     Reply Quote
computezrmle
Volunteer moderator
Volunteer developer
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 15 Jun 08
Posts: 2386
Credit: 222,902,029
RAC: 138,039
Message 36092 - Posted: 27 Jul 2018, 18:57:32 UTC - in response to Message 36091.  

This computer should be able to run 2 ATLAS VMs concurrently using a 4-core setup.
A 4-core setup is much more efficient than an 8-core setup.

In case of Theory multicore should run as efficient as singlecore but there's on big risk:
If you get a longrunner like sherpa close to the end of the 12h design limit it will use only 1 core (+1 core used by auxiliary apps).
The rest of your cores will remain idle for up to 6h.

Suggestion:
To be most efficient you may use a 4-core setup for ATLAS and a 1-core or 2-core setup for Theory.
ID: 36092 · Report as offensive     Reply Quote
Erich56

Send message
Joined: 18 Dec 15
Posts: 1686
Credit: 100,344,392
RAC: 101,798
Message 36095 - Posted: 27 Jul 2018, 19:47:05 UTC - in response to Message 36092.  

...To be most efficient you may use a 4-core setup for ATLAS ...
hm, I remember having seen some diagram long time ago on which the 4-core setup for ATLAS was indeed shown as the most efficient one.

However, meanwhile I have come across several statements here in various places in the forum to the effect that a 1-core setup is the most efficient one, followed by a 2-core setup, next then 4-core (interestingly, NOT 3-core).

No idea what is correct now ...
ID: 36095 · Report as offensive     Reply Quote
computezrmle
Volunteer moderator
Volunteer developer
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 15 Jun 08
Posts: 2386
Credit: 222,902,029
RAC: 138,039
Message 36096 - Posted: 27 Jul 2018, 20:07:06 UTC - in response to Message 36095.  

... 1-core setup is the most efficient one, followed by a 2-core setup ...

Right.

This is a special case as greg_be runs an 8-core setup on a host with only 16GB RAM.
Each 1-core/2-core setup would need 4800MB per core, thus only 3/6 cores could be used in total.

Let's see if 1 8-core or 2 4-core VMs deliver valid results.
ID: 36096 · Report as offensive     Reply Quote
greg_be

Send message
Joined: 28 Dec 08
Posts: 318
Credit: 4,148,677
RAC: 2,010
Message 36097 - Posted: 27 Jul 2018, 22:49:57 UTC - in response to Message 36096.  

I'm not keeping track...but you guys tell me what looks better.
My 8 cores vs the 4 core stuff you guys run.
Thing is...is there a way to limit ATLAS tasks to just 4 cores?
I really haven't explored this much.
It will be awhile before I get new LHC stuff as I have to work through 10 more 6 track tasks first.
ID: 36097 · Report as offensive     Reply Quote
bronco

Send message
Joined: 13 Apr 18
Posts: 443
Credit: 8,438,885
RAC: 0
Message 36098 - Posted: 28 Jul 2018, 0:18:06 UTC - in response to Message 36097.  

Thing is...is there a way to limit ATLAS tasks to just 4 cores?

Yes. There are 2 ways and I believe the gurus are recommending to use both methods.
First, on your preferences webpage set "max #CPUs" to 4. Normally that would be adequate but for ATLAS tasks the server under-estimates the amount of RAM required per task which frequently causes critical problems. So to get the client to use the correct amount of RAM you also need to create an app_config.xml file. This post gives an example app_config.xml as well as the path to the folder in which it should be created. Other posts in that thread tell you how to locate your BOINC data folder and how to make the client read and verify the file. Be sure to get BOINC to verify it.

Remember that is just an example app_config for a 2-core per task setup. Yours will be slightly different if you want to use 4 cores per task. I defer now to the gurus to tell you the precise content you need for your situation.
ID: 36098 · Report as offensive     Reply Quote
maeax

Send message
Joined: 2 May 07
Posts: 2071
Credit: 156,085,957
RAC: 104,411
Message 36099 - Posted: 28 Jul 2018, 5:16:12 UTC - in response to Message 36095.  
Last modified: 28 Jul 2018, 5:21:16 UTC

...To be most efficient you may use a 4-core setup for ATLAS ...
hm, I remember having seen some diagram long time ago on which the 4-core setup for ATLAS was indeed shown as the most efficient one.

However, meanwhile I have come across several statements here in various places in the forum to the effect that a 1-core setup is the most efficient one, followed by a 2-core setup, next then 4-core (interestingly, NOT 3-core).

No idea what is correct now ...

Have a 3-Core running on a Quad-Core with native Linux SL69 and a CPU-Usage from 286% up to of 289% (300% are max.)
https://lhcathome.cern.ch/lhcathome/results.php?hostid=10496403
ID: 36099 · Report as offensive     Reply Quote
computezrmle
Volunteer moderator
Volunteer developer
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 15 Jun 08
Posts: 2386
Credit: 222,902,029
RAC: 138,039
Message 36100 - Posted: 28 Jul 2018, 7:43:44 UTC - in response to Message 36099.  

Have a 3-Core running on a Quad-Core with native Linux SL69 and a CPU-Usage from 286% up to of 289% (300% are max.)
https://lhcathome.cern.ch/lhcathome/results.php?hostid=10496403

Good point.
1-core, 2-core and 4-core setups are mostly used to explain the principles.
3-core and other setups are "forgotten ones" although they may work better on some computers.

At the end it's up to the user to find out what performs best on a distinct system.
ID: 36100 · Report as offensive     Reply Quote
greg_be

Send message
Joined: 28 Dec 08
Posts: 318
Credit: 4,148,677
RAC: 2,010
Message 36108 - Posted: 29 Jul 2018, 12:25:34 UTC - in response to Message 36098.  

If you change preferences for LHC in Boinc, you altering ALL processes of LHC.
So is there a data file where ATLAS only tasks can be limited to 4 cores?

On a different note, BOINC locks up and system freezes after that when running 100% 8 cores. So went down to 99% of total cpu's and it dropped to 7 cores. I forgot that I had done something like this in the past to stabilize the system.
ID: 36108 · Report as offensive     Reply Quote
computezrmle
Volunteer moderator
Volunteer developer
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 15 Jun 08
Posts: 2386
Credit: 222,902,029
RAC: 138,039
Message 36109 - Posted: 29 Jul 2018, 13:55:25 UTC - in response to Message 36108.  

If you change preferences for LHC in Boinc, you altering ALL processes of LHC.
So is there a data file where ATLAS only tasks can be limited to 4 cores?

Do you mean the preferences on the website?
Then, yes there's only 1 setting that affects ATLAS multicore as well as Theory multicore for all computers using the same venue.
#cores and VM's RAM usage can be overwritten locally with the settings in app_config.xml.
See other threads for examples, e.g. https://lhcathome.cern.ch/lhcathome/forum_thread.php?id=4763&postid=35989#35989


... locks up and system freezes after that when running 100% 8 cores ...

That's what I meant here:
https://lhcathome.cern.ch/lhcathome/forum_thread.php?id=4774&postid=36096
"... Let's see if 1 8-core or 2 4-core VMs deliver valid results ..."


... So went down to 99% of total cpu's and it dropped to 7 cores.

This is normal BOINC client behaviour.
With this setting BOINC should not run any 8-core task or more than 1 4-core tasks concurrently.


Conclusion:
You may either follow maeax's comment and try out a 3-core setup without app_config.xml.
This would leave your 7th core free for another project (or a SixTrack).
Total RAM requirement: 10.6 GB

Or you may try a 1-core/2-core setup with app_config.xml to configure at least 4800MB per ATLAS VM.
This should be stable running 2 VMs concurrently and may be stable running 3 VMs.
ID: 36109 · Report as offensive     Reply Quote
bronco

Send message
Joined: 13 Apr 18
Posts: 443
Credit: 8,438,885
RAC: 0
Message 36110 - Posted: 29 Jul 2018, 14:13:08 UTC - in response to Message 36108.  
Last modified: 29 Jul 2018, 14:15:12 UTC

too late
ID: 36110 · Report as offensive     Reply Quote
greg_be

Send message
Joined: 28 Dec 08
Posts: 318
Credit: 4,148,677
RAC: 2,010
Message 36111 - Posted: 29 Jul 2018, 14:21:21 UTC - in response to Message 36098.  

If you change preferences for LHC in Boinc, you altering ALL processes of LHC.
So is there a data file where ATLAS only tasks can be limited to 4 cores?
I had a look at the pages you referenced, but I don't find the files mentioned or the directory mentioned.

On a different note, BOINC locks up and system freezes after that when running 100% 8 cores. So went down to 99% of total cpu's and it dropped to 7 cores. I forgot that I had done something like this in the past to stabilize the system.
ID: 36111 · Report as offensive     Reply Quote
bronco

Send message
Joined: 13 Apr 18
Posts: 443
Credit: 8,438,885
RAC: 0
Message 36112 - Posted: 29 Jul 2018, 14:34:19 UTC - in response to Message 36111.  

So is there a data file where ATLAS only tasks can be limited to 4 cores?
Yes, there is if you create it. But it's not as if BOINC creates it for you when you install BOINC and then you customize it later to suit your needs. You have to create it from scratch using a text editor. The name is app_config.xml. When you feel comfortable with that info we can move on to the folder where you create app_config.xml and other details.
ID: 36112 · Report as offensive     Reply Quote
Erich56

Send message
Joined: 18 Dec 15
Posts: 1686
Credit: 100,344,392
RAC: 101,798
Message 36113 - Posted: 29 Jul 2018, 14:41:17 UTC - in response to Message 36109.  

...Or you may try a 1-core/2-core setup with app_config.xml to configure at least 4800MB per ATLAS VM.
This should be stable running 2 VMs concurrently and may be stable running 3 VMs.
except that if in the web preferences the number of cores for ATLAS is set to 2, then only 2 tasks can be downloaded (and processed)..

This (kind of) error was pointed out and discussed here several months ago.
ID: 36113 · Report as offensive     Reply Quote
greg_be

Send message
Joined: 28 Dec 08
Posts: 318
Credit: 4,148,677
RAC: 2,010
Message 36115 - Posted: 29 Jul 2018, 17:37:45 UTC - in response to Message 36112.  

Just tell me step by step what to do.
I have BOINC spread out over 2 drives.
SSD carries the program files and a external drive carries the data files.
I just recovered from a SSD failure. Lifetime was up on it as was the data read/write.
So now a new SSD.
ID: 36115 · Report as offensive     Reply Quote
greg_be

Send message
Joined: 28 Dec 08
Posts: 318
Credit: 4,148,677
RAC: 2,010
Message 36116 - Posted: 29 Jul 2018, 17:39:55 UTC - in response to Message 36113.  

Can ATLAS handle 4 cores and be stable?
As far as memory goes, I don't think I have ever run out of memory.

For stability purposes on BOINC I have had to cut back to 7 cores (99% of total CPU's).
8 cores it would freeze.
ID: 36116 · Report as offensive     Reply Quote
bronco

Send message
Joined: 13 Apr 18
Posts: 443
Credit: 8,438,885
RAC: 0
Message 36117 - Posted: 29 Jul 2018, 18:35:40 UTC - in response to Message 36115.  

Just tell me step by step what to do.

Meh. Been there done that. Mostly what happens is that after I spend hours typing up goof-proof directions some pr**k purposely pretends to be unable to follow the directions (or maybe there are people who are actually that thick and don't have to pretend). They come back with "it didn't work, what do I do?" I spend hours asking questions trying to find out where they went wrong but they never give a straight answer and never follow corrective measures because their primary objective is to lead me on a wild goose chase rather than solve a problem. So... the info is all there in the links previously mentioned. If you are earnest and if you have the ability then you can figure it out from that. And you can ask pointed questions about specific things you don't understand. If you can't figure it out then you are in over your head. If so then Happy Crunching but please disable ATLAS and possibly other VBox apps, stick to Sixtrack and don't select "Send other work if there is no work for your selected apps".
ID: 36117 · Report as offensive     Reply Quote
greg_be

Send message
Joined: 28 Dec 08
Posts: 318
Credit: 4,148,677
RAC: 2,010
Message 36118 - Posted: 29 Jul 2018, 21:12:41 UTC - in response to Message 36117.  

hey, no need to get nasty with me.
Just because other aholes drag you on and on fine,but I am not that type and I am offended by your attitude. Take it elsewhere. I'm not the average ahole.

What I was not understanding in the other posts was that the files were not already in existence.
So if the information is already in other posts, then save me some grief and just point out the links in order.
The threads jump around a bit.

My set up is not the normal set up either.
If you would get off your firebrand and read what I said, my setup is split on two drives after suffering a SSD failure. So programs files are on the SSD and data files are on the external.
And then within the the LHC program files folder, I did find something that related to ATLAS, but the text did not match what the linked information provided.

Look, if you don't want to help, then don't get your panties in a wad. Just say you don't want to help. Maybe someone else has the patience to explain, rather than get all volcano like. GEES...get a freaking grip.
ID: 36118 · Report as offensive     Reply Quote
1 · 2 · Next

Message boards : Number crunching : Vbox error that I have never seen before


©2024 CERN