log in

Feedback for multicore theory wus


Advanced search

Message boards : Theory Application : Feedback for multicore theory wus

Author Message
PHILIPPE
Send message
Joined: 24 Jul 16
Posts: 58
Credit: 103,726
RAC: 386
Message 30260 - Posted: 9 May 2017, 17:12:53 UTC

Usually , a volunteer can choose his way of crunching following 3 main criterias :
1°) credits earned
2°) cpu efficiency --> reduce idle time between jobs
3°) ram optimization --> adjust the behaviour of computer (smoothy or not responsible)

Folowing the computezrmle's advice ,
i decided to watch the behaviour of multi-core for the theory work units.
I wanted to know how it was possible to run them on this bedtest :
a small host ( 4 cpus ,less than 3.7 GBytes Ram memory, windows 10 ).
(100 % use cpu max , 90 % use memory max)

No other projects present.

I used this linear formula :
Ram memory VM in MBytes = 370 + 260 * n , where n is the number of cores used in the vm.

1-core : 630 MB ram
2-core : 890 MB ram
3-core : 1150 MB ram
4-core : 1410 MB ram


The setting of the ram memory is more difficult than expected because the lower value of ram free ,
inside the vm ,is not just after the beginning of the job ,
but often when the process "cc1plus" appears for few seconds at the beginning or at the end of the jobs.

4 cases :

4-core wu: Not possible because i need to open my browser during the tests.
in such condition ,one of the 4 jobs inside the vm failed.
Orphean processes appeared , either pythia process without rivetvm , either rivetvm without pythia.
The reliability of the 4-core is lower than the single core.
A service launched in background by windows or other apps would have probably produced the same effect.
(but maybe on linux , it would work , if daemons are under control).
Netherveless , ram memory was enough during the test of some jobs during a partial run of the wu.
Ram improvement / core : (630 - 1410/4)/630 = 44 %
Cpu efficiency : not calculated


3-core wu: possible , its interest is to enable normal use of my computer ( 3 cpus busy , 1 idle).
I can browse without any trouble.
Ram improvement / core : (630 - 1150/3)/630 = 39 %
Cpu efficiency :

Test 1 : Task 137020841 :
78618 / 3 / 27274 = 96 %
Test 2 : Task 137102831 :
70807 / 3 / 28141 = 84 %
Test 3 : Task 137178516 :
93493 / 3 / 33485 = 93 %


2-core wu: possible and more resilient, i used the 4 cores simultaneously (2 * 2-core wu) and browsing without any trouble
Ram improvement / core : (630 - 890/2)/630 = 29 %
Cpu efficiency :

Test 1 : Task 137300541 :
45767 / 2 / 29559 = 77 %
Test 2 : Task 137300507 :
72010 / 2 / 47434 = 76 %
Test 3 : Task 137474119
83597 / 2 / 47268 = 88 %
Test 4 : Task 137497053
62842 / 2 / 35075 = 89 %

1-core wu:by default
Ram improvement / core : 0%
Cpu efficiency :

Test 1 : Task 137928981 :
16810 / 1 / 24077 = 70 %
Test 2 : Task 137923944 :
41168 / 1 / 49179 = 84 %
Test 3 : Task 137915263 :
39054 / 1 / 50821 = 77 %
Test 4 : Task
37420 / 1 / 40763 = 92 %
----------------------------------------------------------------------------------------------------------------------------------------------------------------
4 * 1-core wus at the same times , and i can't browse.
3 * 1-core wus at the same times , and it's possible.
Why ?
4 * 630 = 2520 MB ram used (out of 90 % * 3.7 GB ram = 3 330 MB ram)--> 3330-2520= 810 MB ram for OS
3 * 630 = 1890 MB ram used (out of 90 % * 3.7 GB ram = 3 330 MB ram)--> 3330-1890=1440 MB ram for OS and browser.
In windows task manager , i see firefox needs 200 MB ram , so i can only spend (3 330 - 810 - 200 = ) 2320 MB ram for boinc projects.
that is to say only 2320 / 4 = 580 MB by cpu if it is shared equally.

To conclude :
If i want to browse during running theory wus with a maximum of jobs, it's better to use
3 * 1-core with little and medium browsing (630 * 3 = 1890 MB ram used),
2 * 2-core with little ans slow browsing (890 * 2 = 1780 MB ram used),
1 * 3-core with big and quick browsing (1460 * 1 = 1460 MB ram used).
And if i avoid to browse , the best choice is 2 * 2-core.

Hope it may help someone having the same hardware configuration and troubles using his computer.
But each host is different , each user , too...
You have to play with the 2 parameters max#cpu and max#job in your web preference
and modify the app_config.xml file according to the parameters chosen.

These choices are only for small hosts running only theory wus which have the slower ram footprint , next sixtrack.
With other sub-projects , i can run only 1-core LHCb or only 1-core CMS but no Atlas Tasks.

By the way , credits are given in relation with cpu times for multicores , not with elapsed times * number of core used.
So you earn a bit less credits than with 1-core .(It depends on cpu efficiency).
(To evaluate cpu efficiency ,
for elapsed times , i look deep inside the logs to take the hour of the first event, and the hour of the last event written, less period shutdown times,
for cpu times , i get the value in the last phrase where appears cpu times inside the logs.) Values reported in task list are slightly different.

computezrmle
Send message
Joined: 15 Jun 08
Posts: 347
Credit: 3,397,624
RAC: 3,695
Message 30262 - Posted: 9 May 2017, 18:33:43 UTC - in response to Message 30260.

Well done!

It can be very annoying to run long-lasting tests on a nearly saturated host.
Your numbers will help other users to optimise their setup.

Message boards : Theory Application : Feedback for multicore theory wus