Message boards : Theory Application : Theory not utilizing all cores
Message board moderation

To post messages, you must log in.

Previous · 1 · 2 · 3

AuthorMessage
bronco

Send message
Joined: 13 Apr 18
Posts: 443
Credit: 8,438,885
RAC: 0
Message 36789 - Posted: 20 Sep 2018, 15:03:21 UTC - in response to Message 36786.  

Hi all, one of my machines has 16c/32t AMD TR1950X and I've also noticed that the new 263.70 multicore app severely under-utilizes the CPU. What's even worse, all tasks I've watched failed at about halfway of the computation. So naturally, I want to try to limit the number of cores as suggested here, but limiting them globally via the web interface seems rather cumbersome to me.


It's worse than cumbersome, it's downright confusing and misleading due to the fact the "Max cores" setting is mislabeled.

Isn't there a way to limit them for Theory via app_config.xml file, like it can be done for ATLAS?

app_config.xml is what all the smart kids are using. Here's what I use but don't just copy 'n paste it into your setup because it needs to be tweaked for your situation and needs.
The following allows a maximum of 2 concurrently running LHCb tasks and maximum of 2 concurrently running Theory tasks due to the <max_concurrent> elements.
Adjust the --nthreads parameter in the <cmdline> element to determine the number of cores/threads.

<app_config>

  <app>
    <name>Theory</name>
      <max_concurrent>2</max_concurrent>
  </app>
  <app_version>
    <app_name>Theory</app_name>
    <avg_ncpus>1.0</avg_ncpus>
    <plan_class>vbox64_mt_mcore</plan_class>
    <cmdline>--nthreads 1 --memory_size_mb 750</cmdline>
  </app_version>
  
  <app>
    <name>LHCb</name>
    <max_concurrent>2</max_concurrent>
  </app>
  <app_version>
    <app_name>LHCb</app_name>
    <avg_ncpus>1.0</avg_ncpus>
    <plan_class>vbox64_mt_mcore_lhcb</plan_class>
    <cmdline>--nthreads 1 --memory_size_mb 2048</cmdline>
  </app_version>

</app_config>
ID: 36789 · Report as offensive     Reply Quote
Pavel Hanak

Send message
Joined: 5 Mar 06
Posts: 13
Credit: 30,870,563
RAC: 83
Message 36791 - Posted: 20 Sep 2018, 16:07:14 UTC - in response to Message 36789.  

Awesome, thanks! The memory limit is fixed to 750 MB for Theory? There is no need to adjust it with the number of CPUs/threads, like in ATLAS?
ID: 36791 · Report as offensive     Reply Quote
Crystal Pellet
Volunteer moderator
Volunteer tester

Send message
Joined: 14 Jan 10
Posts: 1268
Credit: 8,421,637
RAC: 1,939
Message 36793 - Posted: 20 Sep 2018, 16:26:37 UTC - in response to Message 36791.  

Awesome, thanks! The memory limit is fixed to 750 MB for Theory? There is no need to adjust it with the number of CPUs/threads, like in ATLAS?

730MB is the default RAM for a single core.
When you want to run more jobs (threads) in the Theory VM, add 100MB for each thread.
ID: 36793 · Report as offensive     Reply Quote
bronco

Send message
Joined: 13 Apr 18
Posts: 443
Credit: 8,438,885
RAC: 0
Message 36794 - Posted: 20 Sep 2018, 19:31:33 UTC - in response to Message 36793.  
Last modified: 20 Sep 2018, 19:32:59 UTC

Here's an update. Note it has the "formula" for calculating the memory size included as a comment for Theory and LHCb, because we forget.

<app_config>

  <app>
    <name>Theory</name>
    <max_concurrent>2</max_concurrent>
  </app>
  <app_version>
    <app_name>Theory</app_name>
    <avg_ncpus>1.0</avg_ncpus>
    <plan_class>vbox64_mt_mcore</plan_class>
    <!-- memory for 1 thread is 730 MB, add 100 MB for each additional thread -->
    <cmdline>--nthreads 1 --memory_size_mb 730</cmdline>
  </app_version>
  
  <app>
    <name>LHCb</name>
    <max_concurrent>2</max_concurrent>
  </app>
  <app_version>
    <app_name>LHCb</app_name>
    <avg_ncpus>1.0</avg_ncpus>
    <plan_class>vbox64_mt_mcore_lhcb</plan_class>
    <!-- memory for 1 thread is 2048 MB, add 1300 MB for each additional thread -->
    <cmdline>--nthreads 1 --memory_size_mb 2048</cmdline>
  </app_version>

</app_config>
ID: 36794 · Report as offensive     Reply Quote
Profile denjoR

Send message
Joined: 9 Dec 16
Posts: 1
Credit: 14,571,142
RAC: 0
Message 36795 - Posted: 20 Sep 2018, 20:26:09 UTC

its nothing special about this. the theorie, atlas and so on VMs get diffrent task via the internet. so sometimes the Vm is in idle, maybe to get work or running a less compute intensive task.
you cant compare a vm to a normal workunit task like sixtrack or apps von other boinc projects. everythin is fine.

workunits fail if the system is offline! so if you dont have a stable internet connection dont crunch it ;)
ID: 36795 · Report as offensive     Reply Quote
Previous · 1 · 2 · 3

Message boards : Theory Application : Theory not utilizing all cores


©2024 CERN