Message boards : ATLAS application : How can I run more than 2 4-core tasks ?
Message board moderation

To post messages, you must log in.

1 · 2 · Next

AuthorMessage
peterfilla

Send message
Joined: 2 Jan 11
Posts: 23
Credit: 5,986,899
RAC: 0
Message 29097 - Posted: 7 Mar 2017, 13:37:01 UTC

I have a 12/24 CPU with 64 GB Mem; in the old ATLAS project I could run up to 6 4-core tasks without any problems; app_config etc. all OK.
ID: 29097 · Report as offensive     Reply Quote
Crystal Pellet
Volunteer moderator
Volunteer tester

Send message
Joined: 14 Jan 10
Posts: 1280
Credit: 8,496,817
RAC: 2,374
Message 29099 - Posted: 7 Mar 2017, 14:59:56 UTC
Last modified: 7 Mar 2017, 15:00:50 UTC

I tested it for you with a simulated 24-core machine running 6 Theory tasks each using 4 cores.

App_version part of app_config.xml for the Theory is

 <app_version>
  <app_name>Theory</app_name>
  <plan_class>vbox64</plan_class>
  <avg_ncpus>4.000000</avg_ncpus>
  <cmdline>--memory_size_mb 1280</cmdline>
 </app_version>


For ATLAS it should be:

 <app_version>
  <app_name>ATLAS</app_name>
  <plan_class>vbox64_mt_mcore_atlas</plan_class>
  <avg_ncpus>4.000000</avg_ncpus>
  <cmdline>--memory_size_mb 5400</cmdline>
 </app_version>


In your preferences you don't have to set # of CPUs. Just keep it 1.
ID: 29099 · Report as offensive     Reply Quote
peterfilla

Send message
Joined: 2 Jan 11
Posts: 23
Credit: 5,986,899
RAC: 0
Message 29100 - Posted: 7 Mar 2017, 15:53:45 UTC

Not working . . . I changed the app_config: not working; I rebooted: not working; I changed my preferences on the server: not working.

At the beginning I got 4 tasks, 2 started and 2 are waiting, and this is now the same situation.
ID: 29100 · Report as offensive     Reply Quote
Crystal Pellet
Volunteer moderator
Volunteer tester

Send message
Joined: 14 Jan 10
Posts: 1280
Credit: 8,496,817
RAC: 2,374
Message 29102 - Posted: 7 Mar 2017, 17:35:27 UTC
Last modified: 7 Mar 2017, 17:36:48 UTC

Could you try it with Theory?

Mind that those tasks will run between 12 and 18 hours.

Edit: Maybe you could post your whole app_config.xml . . .
ID: 29102 · Report as offensive     Reply Quote
peterfilla

Send message
Joined: 2 Jan 11
Posts: 23
Credit: 5,986,899
RAC: 0
Message 29103 - Posted: 7 Mar 2017, 17:51:11 UTC

I will try it . . . when ATLAS finished.
My app_config is as your advice (with "app_config" at start and end).
ID: 29103 · Report as offensive     Reply Quote
peterfilla

Send message
Joined: 2 Jan 11
Posts: 23
Credit: 5,986,899
RAC: 0
Message 29105 - Posted: 7 Mar 2017, 20:03:07 UTC

Now I am running 6 4-core tasks from theory using the given theory-app_config - and habe no problems . . . perhaps ATLAS has a restriction ???
ID: 29105 · Report as offensive     Reply Quote
Crystal Pellet
Volunteer moderator
Volunteer tester

Send message
Joined: 14 Jan 10
Posts: 1280
Credit: 8,496,817
RAC: 2,374
Message 29106 - Posted: 7 Mar 2017, 20:40:58 UTC - in response to Message 29105.  

Now I am running 6 4-core tasks from theory using the given theory-app_config - and habe no problems . . . perhaps ATLAS has a restriction ???

Sorry, I can't test that for you. I've 'only' 16GB of RAM.
ID: 29106 · Report as offensive     Reply Quote
peterfilla

Send message
Joined: 2 Jan 11
Posts: 23
Credit: 5,986,899
RAC: 0
Message 29107 - Posted: 7 Mar 2017, 20:51:13 UTC

Thank you very much for helping !!!
ID: 29107 · Report as offensive     Reply Quote
BananaSieg

Send message
Joined: 24 Dec 14
Posts: 3
Credit: 21,664,260
RAC: 0
Message 29114 - Posted: 8 Mar 2017, 19:12:25 UTC
Last modified: 8 Mar 2017, 19:14:33 UTC

Hello, it's working for me with the app_config written as explained by Atlas@home, BUT I modified the section "app_name" from ATLAS_MCORE to ATLAS and "plan_class" to vbox64_mt_mcore_atlas:

http://atlasathome.cern.ch/forum_thread.php?id=517#4729

This is my app_config:


<app_config>
<app_version>
<app_name>ATLAS</app_name>
<avg_ncpus>4.000000</avg_ncpus>
<plan_class>vbox64_mt_mcore_atlas</plan_class>
<cmdline>--memory_size_mb 5300</cmdline>
</app_version>
</app_config>
ID: 29114 · Report as offensive     Reply Quote
peterfilla

Send message
Joined: 2 Jan 11
Posts: 23
Credit: 5,986,899
RAC: 0
Message 29115 - Posted: 8 Mar 2017, 19:50:38 UTC

Hello ! I made the same modifications, but set the memory_size to 5400 (last advice) - but : I can only run 2 tasks at the same time, the third task say "waiting for memory".

Haw many tasks are you running ?
ID: 29115 · Report as offensive     Reply Quote
David Cameron
Project administrator
Project developer
Project scientist

Send message
Joined: 13 May 14
Posts: 387
Credit: 15,314,184
RAC: 0
Message 29126 - Posted: 9 Mar 2017, 13:12:26 UTC - in response to Message 29115.  

I have checked the configurations of ATLAS and LHC and there should be no difference on how many parallel WU you can run.

If you get "waiting for memory" you may be suffering from the BOINC scheduler bug where BOINC assumes each WU will use all available cores and so thinks it needs more memory than it really does.

For example if you have 8 cores and 16GB RAM and setup app_config.xml to run 2 4-core tasks, BOINC will think you need 2*9.4GB=18.8GB to run them both and so will only allow you to run one task.
ID: 29126 · Report as offensive     Reply Quote
peterfilla

Send message
Joined: 2 Jan 11
Posts: 23
Credit: 5,986,899
RAC: 0
Message 29128 - Posted: 9 Mar 2017, 13:45:15 UTC

I have a 12/24-core CPU with 64 GB RAM.

2*13.4GB=26.8GB; 2*25.4GB=50.8GB; ???

But the memory_size in the app_config should overrule this . . .

And in my actual running test there are 2 ATLAS tasks and 3 Theory task running . . so far, I will go on with testing.
ID: 29128 · Report as offensive     Reply Quote
David Cameron
Project administrator
Project developer
Project scientist

Send message
Joined: 13 May 14
Posts: 387
Credit: 15,314,184
RAC: 0
Message 29131 - Posted: 9 Mar 2017, 14:26:29 UTC - in response to Message 29128.  

But the memory_size in the app_config should overrule this . . .


Yes, it does when starting up the VM. It allocates the memory in app_config to the image. But the scheduling algorithm in BOINC client ignores app_config and assumes that each task will use the full number of cores available. You can see some discussion on this in this ATLAS@Home thread
ID: 29131 · Report as offensive     Reply Quote
BananaSieg

Send message
Joined: 24 Dec 14
Posts: 3
Credit: 21,664,260
RAC: 0
Message 29134 - Posted: 9 Mar 2017, 15:47:28 UTC - in response to Message 29115.  
Last modified: 9 Mar 2017, 15:47:45 UTC

Effectively, I get 2 tasks (max) at the same time because i have an 8 threads CPU... So I can't test with your parameters.
ID: 29134 · Report as offensive     Reply Quote
peterfilla

Send message
Joined: 2 Jan 11
Posts: 23
Credit: 5,986,899
RAC: 0
Message 29145 - Posted: 9 Mar 2017, 17:52:13 UTC

Now I have much more information to continue testing - thanks to all !!!
ID: 29145 · Report as offensive     Reply Quote
Erich56

Send message
Joined: 18 Dec 15
Posts: 1689
Credit: 103,920,970
RAC: 121,840
Message 29372 - Posted: 18 Mar 2017, 8:49:58 UTC - in response to Message 29114.  
Last modified: 18 Mar 2017, 8:50:17 UTC

Hello, it's working for me with the app_config written as explained by Atlas@home, BUT I modified the section "app_name" from ATLAS_MCORE to ATLAS and "plan_class" to vbox64_mt_mcore_atlas:

http://atlasathome.cern.ch/forum_thread.php?id=517#4729

This is my app_config:


<app_config>
<app_version>
<app_name>ATLAS</app_name>
<avg_ncpus>4.000000</avg_ncpus>
<plan_class>vbox64_mt_mcore_atlas</plan_class>
<cmdline>--memory_size_mb 5300</cmdline>
</app_version>
</app_config>


I tried this app_config.xml, however, after BOINC reads it, I receive the message

... Your app_config.xml file refers to an unknown application 'ATLAS'. Known applications: 'sixtrack'

ID: 29372 · Report as offensive     Reply Quote
Crystal Pellet
Volunteer moderator
Volunteer tester

Send message
Joined: 14 Jan 10
Posts: 1280
Credit: 8,496,817
RAC: 2,374
Message 29374 - Posted: 18 Mar 2017, 9:57:25 UTC - in response to Message 29372.  

I tried this app_config.xml, however, after BOINC reads it, I receive the message

... Your app_config.xml file refers to an unknown application 'ATLAS'. Known applications: 'sixtrack'


This is normal as long as you did not receive an ATLAS-task.
After you have received the first task, that message will disappear.
Have you checked "Run test applications?" in your preferences?
Be aware that there are some long runners (10x the normal run time) among the tasks.
ID: 29374 · Report as offensive     Reply Quote
Erich56

Send message
Joined: 18 Dec 15
Posts: 1689
Credit: 103,920,970
RAC: 121,840
Message 29376 - Posted: 18 Mar 2017, 10:48:42 UTC - in response to Message 29374.  

This is normal as long as you did not receive an ATLAS-task.
After you have received the first task, that message will disappear.
Have you checked "Run test applications?" in your preferences?
Be aware that there are some long runners (10x the normal run time) among the tasks.


thanks for the Information. As soon as the one ATLAS Task which is still running under the ATLAS Project gets finished, I'll try to download from the LHC@home Project.
Re your question: I have NOT checked "run test applications".

One final question from my side: from what I see in the LHC@home settings, there seems to be no way to check/uncheck either single-core or multicore tasks, as it was the case in the settings of the ATLAS Project. How can I make this determination now?
ID: 29376 · Report as offensive     Reply Quote
gyllic

Send message
Joined: 9 Dec 14
Posts: 202
Credit: 2,533,875
RAC: 0
Message 29379 - Posted: 18 Mar 2017, 10:54:53 UTC - in response to Message 29376.  
Last modified: 18 Mar 2017, 10:56:27 UTC

One final question from my side: from what I see in the LHC@home settings, there seems to be no way to check/uncheck either single-core or multicore tasks, as it was the case in the settings of the ATLAS Project. How can I make this determination now?

There are no dedicated single core tasks any more, only multicore here. but you can run multicore tasks with only one core with app_config, then you have something similar like single core. You can also define the Number of CPUs to one in lhc@home preferences, and then your multicore task will only use one core.
ID: 29379 · Report as offensive     Reply Quote
Erich56

Send message
Joined: 18 Dec 15
Posts: 1689
Credit: 103,920,970
RAC: 121,840
Message 29380 - Posted: 18 Mar 2017, 11:08:02 UTC - in response to Message 29379.  

There are no dedicated single core tasks any more, only multicore here. but you can run multicore tasks with only one core with app_config, then you have something similar like single core. You can also define the Number of CPUs to one in lhc@home preferences, and then your multicore task will only use one core.

this is bad news for one of my machines on which I run ATLAS tasks, i.e. the Notebook which has only 4GB RAM. On this, I can run single-core tasks only; I had tried to run multicore-tasks (defining by an app_config to run only one task on one core), however, I ran out of RAM by doing this;
as then discussed in the message board, a multicore task requires more than 4 GB RAM for the initial Setup (even if during processing, it does NOT require that much).
So, after several efforts to get such a multicore task to run on the Notebook, I gave up and returned to run single-core tasks, which works fine. Unfortunately, this may no longer be possible now :-(
ID: 29380 · Report as offensive     Reply Quote
1 · 2 · Next

Message boards : ATLAS application : How can I run more than 2 4-core tasks ?


©2024 CERN