Message boards : ATLAS application : three 1-core tasks possible with 16GB RAM?
Message board moderation

To post messages, you must log in.

AuthorMessage
Erich56

Send message
Joined: 18 Dec 15
Posts: 1686
Credit: 100,350,374
RAC: 101,608
Message 42602 - Posted: 24 May 2020, 7:09:41 UTC

Did anyone ever succeed in running three 1-core tasks on a system with 16GB RAM?
I set my app_config.xml accordingly, but only 2 tasks are running concurrently.

(I know that I could set the system to run one 3-core task. I was just curious)
ID: 42602 · Report as offensive     Reply Quote
Crystal Pellet
Volunteer moderator
Volunteer tester

Send message
Joined: 14 Jan 10
Posts: 1268
Credit: 8,421,616
RAC: 2,139
Message 42603 - Posted: 24 May 2020, 7:43:32 UTC - in response to Message 42602.  

When you have BOINC allowed to use the most of your memory it should be possible.
Maybe you did not set max CPU's to 1 to in your preferences.
When that's higher, BOINC will reserve more memory for your ATLAS tasks and the 3rd one is waiting for memory.
ID: 42603 · Report as offensive     Reply Quote
Erich56

Send message
Joined: 18 Dec 15
Posts: 1686
Credit: 100,350,374
RAC: 101,608
Message 42604 - Posted: 24 May 2020, 7:49:28 UTC
Last modified: 24 May 2020, 7:54:40 UTC

thanks, Crystal Pellet, for your reply.

You are right, max CPUs was set higher than 1 in the preferences.
I will correct this now and see what happens after new tasks will be downloaded with this new setting.

BTW, can I see somewhere how much RAM BOINC is reserving for a given task? Obviously, that must be different (i.e. higher) to the 3.900MB that I have set in the app_config.xml.

EDIT: I just changed the preferences to # of tasks: 3, and # of cores: 1
And by doing this, I seem to be caught in the known trap - this setting would let me download only 1 task, not 3 :-(
ID: 42604 · Report as offensive     Reply Quote
Crystal Pellet
Volunteer moderator
Volunteer tester

Send message
Joined: 14 Jan 10
Posts: 1268
Credit: 8,421,616
RAC: 2,139
Message 42605 - Posted: 24 May 2020, 8:46:54 UTC - in response to Message 42604.  

BTW, can I see somewhere how much RAM BOINC is reserving for a given task? Obviously, that must be different (i.e. higher) to the 3.900MB that I have set in the app_config.xml.
That's the same formula: 3000MB + ncpus*900MB.
So when it was 8 or No Limit BOINC reserves 10200MB for 1 task not knowing what you configured in app_config.

EDIT: I just changed the preferences to # of tasks: 3, and # of cores: 1
And by doing this, I seem to be caught in the known trap - this setting would let me download only 1 task, not 3 :-(
AAAhh, the same old server bug. I don't understand that it can't be fixed.
So when you would set 3 cores BOINC would reserve 5700MB and you would get 3 tasks, but 16GB is still too less.
Only hanky panky helps here:
Load a number of Atlas-tasks and don't let start them. Shutdown BOINC client and edit client_state.xml
Change <rsc_memory_bound>xxxxxxxxxxxxxx</rsc_memory_bound> in the ATLAS workunits to a lower number.
ID: 42605 · Report as offensive     Reply Quote
computezrmle
Volunteer moderator
Volunteer developer
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 15 Jun 08
Posts: 2386
Credit: 222,913,744
RAC: 138,135
Message 42606 - Posted: 24 May 2020, 8:51:37 UTC - in response to Message 42604.  

The project server calculates 2 memory values, both based on the well known RAM formula.


memory_size_mb is used when a fresh VM is initialized.
This value can be modified using an app_config.xml

rsc_memory_bound is used by the BOINC client to decide if there's enough free RAM to start a task.
This value can't be modified by the client.


If at your web preferences #cores (per task) is set to unlimited or a high value and you run a 1-core setup it may result in a situation where the BOINC client thinks too much RAM is already in use.
ID: 42606 · Report as offensive     Reply Quote
computezrmle
Volunteer moderator
Volunteer developer
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 15 Jun 08
Posts: 2386
Credit: 222,913,744
RAC: 138,135
Message 42607 - Posted: 24 May 2020, 8:55:03 UTC - in response to Message 42605.  

... the same old server bug. I don't understand that it can't be fixed.

I agree.
Very old, very nasty and it has been reported very often, e.g. by Erich.
ID: 42607 · Report as offensive     Reply Quote
Erich56

Send message
Joined: 18 Dec 15
Posts: 1686
Credit: 100,350,374
RAC: 101,608
Message 42608 - Posted: 24 May 2020, 9:27:10 UTC - in response to Message 42605.  

Only hanky panky helps here:
Load a number of Atlas-tasks and don't let start them. Shutdown BOINC client and edit client_state.xml
Change <rsc_memory_bound>xxxxxxxxxxxxxx</rsc_memory_bound> in the ATLAS workunits to a lower number.
okay, thanks for the hint. While 2 tasks are currently running (and I don't want to kill them) I took a look at the client_state.xml. It shows the following value:

<rsc_memory_bound>5976883200.000000</rsc_memory_bound>

I guess, for future tasks, changing this value to about 4 GB should work - do you people agree?
ID: 42608 · Report as offensive     Reply Quote
Crystal Pellet
Volunteer moderator
Volunteer tester

Send message
Joined: 14 Jan 10
Posts: 1268
Credit: 8,421,616
RAC: 2,139
Message 42610 - Posted: 24 May 2020, 10:09:44 UTC - in response to Message 42608.  

I took a look at the client_state.xml. It shows the following value:

<rsc_memory_bound>5976883200.000000</rsc_memory_bound>

I guess, for future tasks, changing this value to about 4 GB should work - do you people agree?
Yes that's right.
When the server would work well with the setting of 1 CPUs, the rsc_memory_bound in bytes would be ‭4089446400‬.
When BOINC is allowed to use (almost) 100% of your memory, 3 ATLAS-tasks should be running.
ID: 42610 · Report as offensive     Reply Quote
Erich56

Send message
Joined: 18 Dec 15
Posts: 1686
Credit: 100,350,374
RAC: 101,608
Message 42612 - Posted: 24 May 2020, 18:47:41 UTC - in response to Message 42610.  

I followed the instructions given above (setting the value, to make is simple, to straight 4000000000.00000) and it works with 3 tasks simultaneously - all 3 of them startet right away.
At this point, after about 1 1/2 hours, the total RAM usage is shown as 14.46GB. So I'll see how it develops during the following night (if I am unlucky, it will go up more and eventually crash; if I am lucky, the tasks will be finished tomorrow morning).

Of course, the procedure of changing the rsc_memory_bound value is rather tedious :-(
ID: 42612 · Report as offensive     Reply Quote

Message boards : ATLAS application : three 1-core tasks possible with 16GB RAM?


©2024 CERN