Message boards :
Number crunching :
Memory requirements for LHC applications
Message board moderation
Author | Message |
---|---|
Send message Joined: 22 Oct 07 Posts: 27 Credit: 808,821 RAC: 0 |
I'm trying to assemble an authoritative list of memory requirement formulas for the multithreaded app. The bottom of this post is what I have so far. Please let me know of any additions for corrections. When complete, perhaps someone with the appropriate authority can make this a pinned post. Thanks, ++PLS ======================================================================= LHC application multithreaded memory requirements Single threaded applications are not listed here. App: ATLAS Forumla: 3900 + 900 * nCPU, megabyte Command line: --memory_size_mb megabytes Source: Yeti setup guide, version 3 App: CMS App: LHCb Formula: 2048 + 1300 * nCPU, megabytes Command line: --memory_size_mb megabytes Source: LHCb Application board, message 37105 App: Theory Formula: 730 + 100 * nCPU, megabytes Command line: --memory_size_mb megabytes Source: Number crunching, message 37161 |
Send message Joined: 13 Apr 18 Posts: 443 Credit: 8,438,885 RAC: 0 |
Your formula for Theory is wrong. The mistake is VERY subtle. Please read carefully and think. Theory requires 730 MB for the first task plus 100 MB for each additional task. I call that the word expression and I think we agree it is correct. The problem is where you translate the word expression into an algebraic expression. It translates into 630 + 100 * nCPUs. Your algebraic expression adds an additional 100 MB for nCPUs > 1. It's not a huge error. But if you make the same error for ATLAS and other apps then it becomes a huge error > 1 GB. If you don't believe me then work out RAM for a 3 CPU task using both methods (the word expression and your algebraic equation) and note that you get 2 different results. |
Send message Joined: 14 Jan 10 Posts: 1411 Credit: 9,433,926 RAC: 11,615 |
The 'Command line: --memory_size_mb' in app_config.xml is ignored by the Theory vbox64_mt_mcore applicfation. It will use the setting coming from the server: 630MB + nCPUs * 100MB. |
Send message Joined: 13 Apr 18 Posts: 443 Credit: 8,438,885 RAC: 0 |
App: ATLAS You seem to have misquoted Yeti. In item #6 of Yeti setup guide, version 3 it says 3,0 GB + 0,9 GB * number of cores. Again you are adding the amount for each additional core twice. |
Send message Joined: 13 Apr 18 Posts: 443 Credit: 8,438,885 RAC: 0 |
The 'Command line: --memory_size_mb' in app_config.xml is ignored by the Theory vbox64_mt_mcore applicfation.Do ATLAS and LHCb also ignore the --memory_size_mb command line arg? |
Send message Joined: 14 Jan 10 Posts: 1411 Credit: 9,433,926 RAC: 11,615 |
Do ATLAS and LHCb also ignore the --memory_size_mb command line arg? ATLAS and CMS are obeying the settings from app_config.xml. I'm not sure about LHCb. I did not run LHCb lately, cause I think the application is not doing something useful at the moment. However I'll test it and report here. |
Send message Joined: 14 Jan 10 Posts: 1411 Credit: 9,433,926 RAC: 11,615 |
Do ATLAS and LHCb also ignore the --memory_size_mb command line arg? LHCb is also using the memory settings, one could have configured in an app_config.xml. |
Send message Joined: 15 Jun 08 Posts: 2520 Credit: 251,913,483 RAC: 128,200 |
The way an app_config.xml works is that the local client simply overwrites the parameter values sent by the server with the corresponding values from your app_config.xml. In case of Theory you may check if you still use the old singletask plan_class. |
Send message Joined: 14 Jan 10 Posts: 1411 Credit: 9,433,926 RAC: 11,615 |
The way an app_config.xml works is that the local client simply overwrites the parameter values sent by the server with the corresponding values from your app_config.xml. I was using the right plan_class, but did not retested a divergent MB-size for Theory, cause I was still in the belief that the server setting for Theory could still not be overruled (as it was in the past). |
Send message Joined: 15 Jun 08 Posts: 2520 Credit: 251,913,483 RAC: 128,200 |
I forgot to mention: I'm using a slightly higher RAM setting for Theory (to spend a bit more cache inside the VM) and I can confirm that it works as expected. |
Send message Joined: 22 Oct 07 Posts: 27 Credit: 808,821 RAC: 0 |
Corrections are incorporated. I couldn't find requirements for CMS, does anyone have them? ===================================== LHC application multithreaded memory requirements Single threaded applications are not listed here. App: ATLAS Forumla: 3000 + 900 * nCPU, megabyte Command line: --memory_size_mb megabytes Source: Yeti setup guide, version 3 App: CMS App: LHCb Formula: 2048 + 1300 * nCPU, megabytes Command line: --memory_size_mb megabytes Source: LHCb Application board, message 37105 App: Theory Formula: 630 + 100 * nCPU, megabytes Command line: not needed, memory is computed by server Source: Number crunching, message 37193 |
Send message Joined: 16 Jul 05 Posts: 24 Credit: 35,251,537 RAC: 0 |
Corrections are incorporated. I think your LHCb formula is wrong - it doesn't match what I posted in the other thread https://lhcathome.cern.ch/lhcathome/forum_thread.php?id=4867&postid=37168, and the latter matches what I see running. Also, your list is for VirtualBox jobs, not multi-threaded - multi-threaded native Atlas seems to have more frugal requirements. |
Send message Joined: 13 Apr 18 Posts: 443 Credit: 8,438,885 RAC: 0 |
Looking good. Sorry no info for CMS You may as well include "plan class" as it is required in app_config.xml Your App: LHCb looks to be correct so just add the plan class to get: App: LHCb Plan class: vbox64_mt_mcore_lhcb Formula: 2048 + 1300 * nCPU, megabytes Command line: --memory_size_mb megabytes Source: LHCb Application board, message 37105 Your App: ATLAS looks to be correct for the VBox version so just add the plan class to get: App: ATLAS Plan class: vbox64_mt_mcore_atlas Forumla: 3000 + 900 * nCPU, megabyte Command line: --memory_size_mb megabytes Source: Yeti setup guide, version 3 A new section for ATLAS native: App: ATLAS Plan class: native_mt Formula: 2100 + ? * nCPU (verify this line) Command line: --memory_size_mb megabytes Source: ??? Theory is more complicated as there are 32 bit and 64 bit versions, one of them ignores the memory parameter from app_config.xml while the other does not, not sure which is which. App: Theory Plan class: vbox32 Formula: ? Command line: Source: ??? App: Theory Plan class: vbox64_mt_mcore Formula: 630 + 100 * nCPU, megabytes Command line: Source: ??? |
Send message Joined: 15 Jun 08 Posts: 2520 Credit: 251,913,483 RAC: 128,200 |
The current default RAM requirements are as follows Theory vbox32 (only singlecore) 320 (in megabytes) Theory vbox64 (multicore) 630 + 100 * #cpus (in megabytes) LHCb vbox64 (multicore) 748 + 1300 * #cpus (in megabytes) based on the old singlecore app that required 2 GB CMS vbox64 (old singlecore is currently not available, multicore is still under development) 2048 (in megabytes) ATLAS vbox64 3000 + 900 * #cpus (in megabytes) ATLAS native roughly 2 GB per thread + RAM required by auxiliary apps like CVMFS RAM that is allocated by a native app can be returned to the OS. Thus those requirements are less critical than vbox requirements which can't be returned. All default RAM settings are more or less suggestions based on experience. VMs with less RAM configured via an app_config.xml may be able to run successfully. On the other hand there may be future jobs with higher requirements. |
Send message Joined: 14 Jan 10 Posts: 1411 Credit: 9,433,926 RAC: 11,615 |
Theory vbox32 (only singlecore) That's how it is set in the project file Theory32_2017_01_25.xml, but the VM is created with 'only' 256MB. You may set a different RAM-size in app_config.xml and also could run 2 or more jobs concurrently inside the VM. E.g. <app_version> <app_name>Theory</app_name> <plan_class>vbox32</plan_class> <avg_ncpus>2</avg_ncpus> <cmdline>--nthreads 2</cmdline> <cmdline>--memory_size_mb 384</cmdline> </app_version> |
Send message Joined: 13 Apr 18 Posts: 443 Credit: 8,438,885 RAC: 0 |
Sixtrack has 2 plan classes, avx and sse2. What is the memory requirement for each plan class? |
Send message Joined: 15 Jun 08 Posts: 2520 Credit: 251,913,483 RAC: 128,200 |
Sixtrack has 2 plan classes, avx and sse2. What is the memory requirement for each plan class? The size of the (linux) app itself is roughly 7 MB. AVX and SSE2 are fed from the same task pool. Thus the RAM requirements are the same. The recent SixTrack tasks over at LHC-dev had a peak working set size of roughly 350 MB each. This may vary depending on the task's input data. Thus I would not recommend to set a RAM value via app_config.xml. |
Send message Joined: 22 Oct 07 Posts: 27 Credit: 808,821 RAC: 0 |
Thank you all. This has become far more interesting than I thought it would be. Here is the current file, incorporating these changes. I'm not listing the sixtrack entries. It seem the native jobs really don't need assistance in managing memory, whereas VBox requires specifying memory size when the VM is created. ====================================== LHC application multithreaded memory requirements, Single threads applications, VirtualBox or not, are not mentioned here. App: ATLAS VirtualBox64 Forumla: 3000 + 900 * nCPU, megabyte Plan class: vbox64_mt_mcore_atlas Command line: --memory_size_mb megabytes Source: Yeti setup guide, version 3 App: ATLAS native (not VirtualBox) Formula 2100 + 2000 * nCPU, megabytes Plan class: native_mt Command line: --memory_size_mb megabytes Source: Number Cruching message 37225 App: CMS App: LHCb VirtualBox64 Formula: 748 + 1300 * nCPU, megabytes Plan class: vbox64_mt_mcore_lhcb Command line: --memory_size_mb megabytes Source: LHCb Application board, message 37105 App: Theory VirtualBox64 Formula: 630 + 100 * nCPU, megabytes Plan class: vbox64_mt_mcore Command line: not needed, memory is computed by server Source: Number crunching, message 37193 App: Theory VirtualBox32 Formula: 256 + 64 * nCPU, megabytes Plan class: vbox32 Command line: --memory_size_mb megabytes Source: Number crunching message 37229 |
Send message Joined: 13 Apr 18 Posts: 443 Credit: 8,438,885 RAC: 0 |
App: ATLAS native (not VirtualBox) The formula is wrong. It should be 100 + 2000 * nCPU. Also it might be worth mentioning it's Linux only else the unwary might try to run it on Windows or Mac. App: ATLAS native (not VirtualBox, Linux only) Formula 100 + 2000 * nCPU, megabytes Plan class: native_mt Command line: --memory_size_mb megabytes Source: Number Cruching message 37225 |
Send message Joined: 9 Dec 14 Posts: 202 Credit: 2,533,875 RAC: 0 |
The formula is wrong. It should be 100 + 2000 * nCPU.The formula 100 + 2000*nCPU is also wrong. I don't know the correct one but since my PC with 6GB RAM can crunch a 4 core native ATLAS task without any problem, the mentioned formula can't be correct (according to the formula the native task would need 100 + 2000*4 which is 8100MB on a 6000MB RAM machine which obviously would not work). It is also possible to crunch two 2-core native ATLAS tasks concurrently with 6GB RAM, so the correct formula has to be another one. |
©2024 CERN