Message boards : LHC@home Science : Problem with virtual box settings/configuration
Message board moderation

To post messages, you must log in.

AuthorMessage
Iscaran

Send message
Joined: 10 May 17
Posts: 4
Credit: 104,546
RAC: 1
Message 32484 - Posted: 21 Sep 2017, 7:33:16 UTC

Hello All,

recently I upgrade to the new BOINC client and decided to also use the virtual box installation (Host-System is a Win7 x64).

BOINC is set to run only when system is idle and recently I am running in to issues that waking the system up again is often hardly possible.

Now I find that the created virtual boxes (VB) always start with 3500MB of memory setting. This is a problem because the Host only has 4GB total RAM and thus when I try to re-awaken the system it takes like 30min before it actually gets "usable" again as it seems the VB is eating up all the memory and it takes that long to "clean" it up.

When I manage to set the running VB to 2048MB memory allocation then wait until the system goes into BOINC-state and try to re-awaken it works fast (within a few seconds).

Now the problem is - that whenever a new task is applied it creates a new VB and it has AGAIN the default setting of 3500 MB RAM.

I already edited the projects xml e.g. \BOINC_Data\projects\lhcathome.cern.ch_lhcathome\ATLAS_2017_01_09.xml

so it reads 2048 MB
<memory_size_mb>2048</memory_size_mb>

But still whenever e.g. an ATLAS task is startet the VB starts with 3500MB allocation...

So my question is, how and where can I change the default value with which VBs for the calculation task are created. And how can I make sure the setting is not overridden somehow ?

I hope somebody can help me with this issue, otherwise I will have to go back to the defaul client with no VB.
ID: 32484 · Report as offensive     Reply Quote
Harri Liljeroos
Avatar

Send message
Joined: 28 Sep 04
Posts: 423
Credit: 22,582,281
RAC: 5,692
Message 32487 - Posted: 21 Sep 2017, 11:28:22 UTC - in response to Message 32484.  

You shouldn't change the memory setting for Atlas tasks. They need that much memory to be crunched successfully. Instead you should select in the preferences on the web site to crunch only Theory (600 MB) or LHCb (2400 MB) or sixtrack (no VB, about 450 MB) tasks.
ID: 32487 · Report as offensive     Reply Quote
Iscaran

Send message
Joined: 10 May 17
Posts: 4
Credit: 104,546
RAC: 1
Message 32488 - Posted: 21 Sep 2017, 11:51:25 UTC - in response to Message 32487.  
Last modified: 21 Sep 2017, 11:52:19 UTC

But shouldnt the BOINC client automatically detect that the VBox setting is too small and therefore NOT load such tasks on a machine like this ?

I mean I have set global pereferences properly before in the BOINC client.
For example I have defined that calculation stops when mouse is used and that the system is allowed to take up to 90% of the System RAM. (which would be 3600MB maximum).
This works pretty fine and the system returns into a responsive "workable" condition within a few seconds after moving the mouse.

Now with VB installed however there are no special global preferences for the VBox and it seems that when the system creates a VBox according to the global preferences it seems to overstretch the system somehow.

When I then try to return the system into "workable" condition when there is a default VB task by moving the mouse it can take up to 30minutes to respond.

When I wait that long and then open up the Vbox manager and edit the value of memory to be used into e.g. 2048MB instead of the 3500MB which are there. Then the returning into normal state is again "normal" (within a few seconds).

But OK I can disable ATLAS and LHCb tasks then completely if they require that large amounts of Memory for VBox.

Also there is nowhere an information on how much RAM which tasks require...could be helpful to add this to the selection in the account management or in the global settings ?
ID: 32488 · Report as offensive     Reply Quote
Harri Liljeroos
Avatar

Send message
Joined: 28 Sep 04
Posts: 423
Credit: 22,582,281
RAC: 5,692
Message 32489 - Posted: 21 Sep 2017, 12:55:50 UTC - in response to Message 32488.  
Last modified: 21 Sep 2017, 13:05:34 UTC

I agree that the memory handling of Virtual Box and Boinc are not optimal and need improving. Maybe something can be done when the configuration files are created. Best to report this on the message board for the Atlas application.
[edit]There are a lot of discussions there about Atlas memory consumption, like in this thread: https://lhcathome.cern.ch/lhcathome/forum_thread.php?id=4308[/edit]

Have you gone through Yeti's Atlas & VB checklist? https://lhcathome.cern.ch/lhcathome/forum_thread.php?id=4161

There are no "official" developers for Boinc anymore, everything is nowadays done by volunteer users. If you want changes for Boinc, please report your problems in Boinc message board: http://boinc.berkeley.edu/dev/forum_index.php
ID: 32489 · Report as offensive     Reply Quote
Iscaran

Send message
Joined: 10 May 17
Posts: 4
Credit: 104,546
RAC: 1
Message 32491 - Posted: 21 Sep 2017, 13:39:30 UTC

Thank you.
No I didnt see Yetis post first. From reading through it I learn that LHC and ATLAS Task might be too large for a system with that limited amount of RAM as I have.

For the time now I have disabled LHC and Atlas tasks - since if the numbers are correct they would require >2GB RAM in the virtual box.

The Problem seems to be then that the BOINC Manager seems to be unable to properly limit the VBox RAM size to a reasonable amount(and thus would automatically skip task which are too large to fit in the box).
That way it should no longer happen that the system is not responding within a reasonable time after mouse movement.
ID: 32491 · Report as offensive     Reply Quote
Harri Liljeroos
Avatar

Send message
Joined: 28 Sep 04
Posts: 423
Credit: 22,582,281
RAC: 5,692
Message 32492 - Posted: 21 Sep 2017, 14:00:06 UTC - in response to Message 32491.  
Last modified: 21 Sep 2017, 14:00:35 UTC

Boinc does not limit an application's memory to anything, it just adds up the memory reports of different applications and compares that to the value of memory limit set in your preferences. When a new task is about to start it decides if that application still fits in the memory. I think that Boinc doesn't even check the free memory but only relies to the settings i.e. if Boinc is allowed to use 70% of memory it will try to use it whether any memory is actually free or not. And to make things worse the used memory that an application reports to Boinc isn't what the application is actually using, only an estimate made by the server/programmer/scientist who has submitted the application/task.

If I am wrong about what I wrote above, please somebody correct me.
ID: 32492 · Report as offensive     Reply Quote
Iscaran

Send message
Joined: 10 May 17
Posts: 4
Credit: 104,546
RAC: 1
Message 32493 - Posted: 21 Sep 2017, 14:40:06 UTC - in response to Message 32492.  
Last modified: 21 Sep 2017, 14:46:06 UTC

No it seems correct. As that is probably what happens with the LHC, ATLAS task in the VB.

Although the task itself is maybe reported as say 2500MB due to the fact it requires running in the VB it is "expanded" to something larger than that.

At the same time my BOINC is set to utilize maximum 90% RAM when idle. System has 4GB so 90% = 3600MB.

Now it decides 2500MB LHC Task is <3600MB Limit and kicks it off in the VB...but the VB which gets created "automatically" sizes to 3500MB (+ maybe some more excess) which then causes the system to cripple down as it has to shuffle data from and too the HDD Swap file all the time when trying to reawake.

So as I see it I can either "cripple" my BOINC by using a lower total RAM setting (say 50% instead of 90%) to avoid the BOINC manager to start too large virtual boxes which exceed the RAM limitation.
Or I choose not to run ATLAS and LHC tasks at all. Which seems to be the smarter option as the other task then can access 90% of RAM instead of just 50%.

BUT:
I think I tried using a global memory limit of as low as 50% already and still it kicked off virtual boxes which were set to 3500MB of RAM.
Anyway to know for sure I will try that today over night...lets see if I can get the system back tomorrow morning.

Also to consider:

WHY the virtual boxes still get created with 3500MB when I EDIT the .xml to use only 2048MB ?

EDIT: I created also a question on this on the BOINC board as you suggested:
http://boinc.berkeley.edu/dev/forum_thread.php?id=11866&postid=81522#81522
ID: 32493 · Report as offensive     Reply Quote
Harri Liljeroos
Avatar

Send message
Joined: 28 Sep 04
Posts: 423
Credit: 22,582,281
RAC: 5,692
Message 32494 - Posted: 21 Sep 2017, 15:29:59 UTC - in response to Message 32493.  

The memory reported to Boinc is in init_data.xml which I think is downloaded with the task. When a task has been started you can find it in one of the slots folders where runtime data is kept. In that file there is a tag <rsc_memory_bound> which value is sent to Boinc. If you want to affect the memory Virtual Machine is started with you should create an app_config.xml file where this can be done. But as I mentioned before, if you reduce the memory below what the application needs, the task will not produce any relevant results. It may end very fast or it may run several hours without using the CPU hardly at all, anyway the result is useless. You may even get the task validated and get credits for it, but the science is not being served.

How much of memory is actually used by the task is not straight forward to find out. Windows doesn't show it in the task manager but in the VM VirtualBox Manager you can see how much is reserved (System, Base Memory) for the Virtual Machine where the task is running. The actual memory can be found inside the Virtual Machine on Alt+F3 which should show TOP. There is more decription about this in the thread in Atlas message board I linked previously. (I don't really know my way around Linux.)
ID: 32494 · Report as offensive     Reply Quote

Message boards : LHC@home Science : Problem with virtual box settings/configuration


©2020 CERN