Message boards : ATLAS application : 3-core task crunches slower than 2-core task - why so?
Message board moderation

To post messages, you must log in.

AuthorMessage
Erich56

Send message
Joined: 18 Dec 15
Posts: 1688
Credit: 103,119,319
RAC: 127,180
Message 36435 - Posted: 15 Aug 2018, 11:21:44 UTC

When recently trying out various options as to what would be the optimal ATLAS crunching configuration on my Windows10 machine with a 6+6HT core processor (Intel i7-4930k @3.6GHz), RAM 32GB, I made the following strange observation:

First I was crunching four 2-core tasks concurrently. Then three 3-core tasks concurrently.

First surprise: when opening VM console_2, I saw that the processing times for each job was even a little bit higher with the 3-core tasks (~ 650secs vis-a-vis ~550secs with the 2-core tasks).
Second surprise (or, actually: no surprise): total runtime for each task of the both versions was about the same (~11 hours), CPU time of course was about 1 1/2 times higher for the 3-core tasks.

And last, but not least: the credit points earned for a 3-core task were about half of what it was for a 2-core task.

In all cases, no other projects/tasks were running, no other programs (except the Antivirus tool). Total CPU load as shown in the Windows Task Manager was ~ 73% when running the four 2-core tasks, and ~ 82% when running the three 3-core tasks.

Does anyone have a logical explanation for this strange behaviour?
ID: 36435 · Report as offensive     Reply Quote
Toby Broom
Volunteer moderator

Send message
Joined: 27 Sep 08
Posts: 804
Credit: 650,261,501
RAC: 251,948
Message 36442 - Posted: 15 Aug 2018, 16:27:15 UTC

I think this is due to the extra shuffling the OS has to do it make the tasks a little slower each time, the ATLAS team tested it with there many core servers and saw the same. You are doing more work overall but each task gets a little less efficient. You gain some though when the tasks overlap as one can be prepping while the other is running for example.

For the credit I think you get more for less time so the more cores the longer it takes as its 3xthe time for a 3 core, therefore the more cores the lower the credit?


I run 1core on all my machine except one as there is a limit of 50WU at once and that machine has more than 50cores so if I choose the 2 core option I can get 100WU
ID: 36442 · Report as offensive     Reply Quote
Erich56

Send message
Joined: 18 Dec 15
Posts: 1688
Credit: 103,119,319
RAC: 127,180
Message 36448 - Posted: 15 Aug 2018, 19:17:18 UTC - in response to Message 36442.  

I think this is due to the extra shuffling the OS has to do it make the tasks a little slower each time
well, I would understand "a little slower" - but in this case the difference is tremendous.
Why would I waste 3 cores per task if I can achieve the same computation result with 2 cores?
It I were not limited with 32GB RAM, I would now try eight 1-core tasks in order to find out how this relates to the 2-core and 3-core operation.
ID: 36448 · Report as offensive     Reply Quote

Message boards : ATLAS application : 3-core task crunches slower than 2-core task - why so?


©2024 CERN