Message boards : ATLAS application : app_config.xml parameters question
Message board moderation

To post messages, you must log in.

AuthorMessage
AndreyOR

Send message
Joined: 8 Dec 19
Posts: 27
Credit: 3,034,172
RAC: 1,355
Message 46784 - Posted: 18 May 2022, 6:38:33 UTC

What is the difference between the following 2 parameters in the app_version section of app_config.xml file, especially as it pertains to multithread apps (LHC ATLAS, MilkyWay N-Body Simulation)?
<app_version>
   <avg_ncpus>x</avg_ncpus>
   <cmdline>--nthreads x</cmdline>
</app_version>
ID: 46784 · Report as offensive     Reply Quote
computezrmle
Volunteer moderator
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 15 Jun 08
Posts: 2043
Credit: 153,623,260
RAC: 152,767
Message 46786 - Posted: 18 May 2022, 6:51:10 UTC - in response to Message 46784.  

See the BOINC manual:
https://boinc.berkeley.edu/wiki/Client_configuration#Project-level_configuration

avg_ncpus
the number of CPU instances (possibly fractional) used by the app version.
This is used by the BOINC client to calculate various values (work fetch, computer usage, ...)

cmdline
command-line parameters to pass to the program.
The program needs to understand this.
Program can be a scientific app or a wrapper, e.g. vboxwrapper.
ID: 46786 · Report as offensive     Reply Quote
AndreyOR

Send message
Joined: 8 Dec 19
Posts: 27
Credit: 3,034,172
RAC: 1,355
Message 46789 - Posted: 18 May 2022, 7:36:12 UTC - in response to Message 46786.  

I'm familiar with that page and have read it before but it doesn't clarify things much. I understand the difference between cmdline and avg_ncpus in general. I'm specifically wondering about <cmdline>--nthreads x</cmdline> not just cmdline in general. --nthreads in cmdline and avg_ncpus seems to be specifying the same thing and thus seem redundant. However, I've seen people post their app_config files with both entries and I couldn't see why. Would you ever use both in the same app_config and if so under what circumstances? Also, how does one know what cmdline parameters a given program understands?
ID: 46789 · Report as offensive     Reply Quote
computezrmle
Volunteer moderator
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 15 Jun 08
Posts: 2043
Credit: 153,623,260
RAC: 152,767
Message 46790 - Posted: 18 May 2022, 8:05:36 UTC - in response to Message 46789.  

I understand the difference between cmdline and avg_ncpus

Sorry to say that:
It looks like you don't understand the difference.

As explained in the BOINC manual cmdline passes parameters to the program that is called.
BOINC does not care about what you pass.
It is up to the developers of that program how/if those parameters are understood.

how does one know what cmdline parameters a given program understands?

Under normal circumstances you would call "program {--help|-h|/h"}.
You could also look into the program's manual.
Example:
vboxwrapper's command line parameters can be seen here (usage section):
https://github.com/BOINC/boinc/blob/master/samples/vboxwrapper/vboxwrapper.cpp


Again:
In case of app_config.xml
avg_ncpus controls the BOINC client and not the program that is called.
cmdline controls the program that is called and not the BOINC client.


In case of vboxwrapper a missing cmdline entry automatically results in default values that are passed by the BOINC client.
Those can be taken from app_config.xml or from a couple of other configuration sources.
It's up to the BOINC project developers to set useful values there.


If you set different values for avg_ncpus and nthreads this may lead to unwanted side-effects, typically overloaded systems, work fetch trouble and wrong credit calculation.
ID: 46790 · Report as offensive     Reply Quote
AndreyOR

Send message
Joined: 8 Dec 19
Posts: 27
Credit: 3,034,172
RAC: 1,355
Message 46793 - Posted: 18 May 2022, 20:56:35 UTC - in response to Message 46790.  

Thank you for explaining some more. It seems my suspicion was right that you wouldn't use avg_ncpus and --nthreads command line parameter for the same app. It'll either be redundant or detrimental (if different values are used). avg_ncpus would be the way to control thread usage of multithread apps. I was curious and was able to find a list of command line paramaters for MilkyWay N-Body Simulation but couldn't find them for LHC ATLAS (native). Could you provide a link? Thank you.
ID: 46793 · Report as offensive     Reply Quote
captainjack

Send message
Joined: 21 Jun 10
Posts: 36
Credit: 5,718,661
RAC: 9,035
Message 46794 - Posted: 18 May 2022, 22:36:35 UTC

Can't guarantee that this is perfect, but it works for me and doesn't throw any errors in the Event Log.

<app_config>
  <app>
      <name>ATLAS</name>
          <cpu_usage>5</cpu_usage>
  </app>
  <app_version>
      <app_name>ATLAS</app_name>
        <plan_class>native_mt</plan_class>
           <avg_ncpus>5</avg_ncpus>
           <cmdline>--nthreads 5</cmdline>
  </app_version>
</app_config>
ID: 46794 · Report as offensive     Reply Quote
computezrmle
Volunteer moderator
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 15 Jun 08
Posts: 2043
Credit: 153,623,260
RAC: 152,767
Message 46796 - Posted: 19 May 2022, 6:08:56 UTC - in response to Message 46793.  

Could you provide a link?

ATLAS native calls a wrapper script as main program.
The current version of that script is ATLAS_run_atlas_2.87_x86_64-pc-linux-gnu.
It can be found in your projects directory.
ID: 46796 · Report as offensive     Reply Quote
computezrmle
Volunteer moderator
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 15 Jun 08
Posts: 2043
Credit: 153,623,260
RAC: 152,767
Message 46797 - Posted: 19 May 2022, 6:14:10 UTC - in response to Message 46794.  

  <app>
      <name>ATLAS</name>
          <cpu_usage>5</cpu_usage>
  </app>

This part of your app_config.xml will be ignored by the BOINC client.
It should be removed.

The manual states:
cpu_usage The number of CPU instances (possibly fractional) used by GPU versions of this app.

ATLAS doesn't have a GPU version.
In addition, if you use it for an app that does <cpu_usage> must be enclosed by <gpu_versions> ... </gpu_versions>.
ID: 46797 · Report as offensive     Reply Quote
AndreyOR

Send message
Joined: 8 Dec 19
Posts: 27
Credit: 3,034,172
RAC: 1,355
Message 46798 - Posted: 19 May 2022, 6:43:36 UTC - in response to Message 46794.  

captainjack, yes, I'd expect your app_config to work. There seem to be some redundant/unnecessary entries though. Since you're only trying to modify the native ATLAS version you'd only need the app_version portion of the app_config. In addition, <cmdline>--nthreads 5</cmdline> is redundant since you're already specifying that you want to use 5 CPUs via avg_ncpus. I also believe that the app section is ignored since it's incomplete. Try the following. It's cleaner, shorter and so if you want to change things you're less likely to make an accidental mistake.
<app_config>
    <app_version>
      <app_name>ATLAS</app_name>
      <plan_class>native_mt</plan_class>
      <avg_ncpus>5</avg_ncpus>
    </app_version>
</app_config>
ID: 46798 · Report as offensive     Reply Quote
AndreyOR

Send message
Joined: 8 Dec 19
Posts: 27
Credit: 3,034,172
RAC: 1,355
Message 46799 - Posted: 19 May 2022, 7:05:52 UTC - in response to Message 46796.  

I checked that file and it seems like the only argument that's available is --nthreads. I thought that some time ago when I was trying to figure out how to set up native ATLAS I saw the usage of --memory_size parameter in the forums. I added it to the app_config file (can't remember what made me think I needed it) but it seems like it's invalid and is just ignored. I'll have to delete it then. Was it ever used in the past?
ID: 46799 · Report as offensive     Reply Quote
computezrmle
Volunteer moderator
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 15 Jun 08
Posts: 2043
Credit: 153,623,260
RAC: 152,767
Message 46800 - Posted: 19 May 2022, 7:34:59 UTC - in response to Message 46799.  

--memory_size parameter ... Was it ever used in the past?

I don't think it has ever been used for ATLAS native.
It is mentioned very often due to the fact that it is used by vboxwrapper for years.
Most people just copy/paste those options without thinking about whether it makes sense.
ID: 46800 · Report as offensive     Reply Quote
Toby Broom
Volunteer moderator

Send message
Joined: 27 Sep 08
Posts: 730
Credit: 503,545,068
RAC: 274,969
Message 46803 - Posted: 19 May 2022, 16:54:22 UTC - in response to Message 46800.  
Last modified: 19 May 2022, 16:54:37 UTC

I use the --memory_size_mb one, since I force atlas to use 1 WU and unlimited on the web settings this tells BOINC each WU use 10 GB, since BOINC thinks its using 10 GB anyway I just let it go with 10 GB.
ID: 46803 · Report as offensive     Reply Quote
AndreyOR

Send message
Joined: 8 Dec 19
Posts: 27
Credit: 3,034,172
RAC: 1,355
Message 46805 - Posted: 19 May 2022, 20:01:47 UTC - in response to Message 46803.  

It seems like you're using the VBox version of ATLAS not native. According to a post above, that argument is valid in the VBox version not native though.
ID: 46805 · Report as offensive     Reply Quote

Message boards : ATLAS application : app_config.xml parameters question


©2022 CERN