Message boards : ATLAS application : 3-core task crunches slower than 2-core task - why so?
Message board moderation

To post messages, you must log in.

AuthorMessage
Erich56

Send message
Joined: 18 Dec 15
Posts: 1821
Credit: 118,941,268
RAC: 21,804
Message 36435 - Posted: 15 Aug 2018, 11:21:44 UTC

When recently trying out various options as to what would be the optimal ATLAS crunching configuration on my Windows10 machine with a 6+6HT core processor (Intel i7-4930k @3.6GHz), RAM 32GB, I made the following strange observation:

First I was crunching four 2-core tasks concurrently. Then three 3-core tasks concurrently.

First surprise: when opening VM console_2, I saw that the processing times for each job was even a little bit higher with the 3-core tasks (~ 650secs vis-a-vis ~550secs with the 2-core tasks).
Second surprise (or, actually: no surprise): total runtime for each task of the both versions was about the same (~11 hours), CPU time of course was about 1 1/2 times higher for the 3-core tasks.

And last, but not least: the credit points earned for a 3-core task were about half of what it was for a 2-core task.

In all cases, no other projects/tasks were running, no other programs (except the Antivirus tool). Total CPU load as shown in the Windows Task Manager was ~ 73% when running the four 2-core tasks, and ~ 82% when running the three 3-core tasks.

Does anyone have a logical explanation for this strange behaviour?
ID: 36435 · Report as offensive     Reply Quote
Toby Broom
Volunteer moderator

Send message
Joined: 27 Sep 08
Posts: 850
Credit: 692,823,409
RAC: 68,497
Message 36442 - Posted: 15 Aug 2018, 16:27:15 UTC

I think this is due to the extra shuffling the OS has to do it make the tasks a little slower each time, the ATLAS team tested it with there many core servers and saw the same. You are doing more work overall but each task gets a little less efficient. You gain some though when the tasks overlap as one can be prepping while the other is running for example.

For the credit I think you get more for less time so the more cores the longer it takes as its 3xthe time for a 3 core, therefore the more cores the lower the credit?


I run 1core on all my machine except one as there is a limit of 50WU at once and that machine has more than 50cores so if I choose the 2 core option I can get 100WU
ID: 36442 · Report as offensive     Reply Quote
Erich56

Send message
Joined: 18 Dec 15
Posts: 1821
Credit: 118,941,268
RAC: 21,804
Message 36448 - Posted: 15 Aug 2018, 19:17:18 UTC - in response to Message 36442.  

I think this is due to the extra shuffling the OS has to do it make the tasks a little slower each time
well, I would understand "a little slower" - but in this case the difference is tremendous.
Why would I waste 3 cores per task if I can achieve the same computation result with 2 cores?
It I were not limited with 32GB RAM, I would now try eight 1-core tasks in order to find out how this relates to the 2-core and 3-core operation.
ID: 36448 · Report as offensive     Reply Quote
Profile rbpeake

Send message
Joined: 17 Sep 04
Posts: 105
Credit: 32,824,862
RAC: 88
Message 50369 - Posted: 10 Jun 2024, 0:09:03 UTC - in response to Message 36448.  

I think this is due to the extra shuffling the OS has to do it make the tasks a little slower each time
well, I would understand "a little slower" - but in this case the difference is tremendous.
Why would I waste 3 cores per task if I can achieve the same computation result with 2 cores?
It I were not limited with 32GB RAM, I would now try eight 1-core tasks in order to find out how this relates to the 2-core and 3-core operation.

Do these observations remain valid currently? In other words, the fewer cores you use the more efficient is the process?
Thanks.
Regards,
Bob P.
ID: 50369 · Report as offensive     Reply Quote
Crystal Pellet
Volunteer moderator
Volunteer tester

Send message
Joined: 14 Jan 10
Posts: 1422
Credit: 9,484,585
RAC: 1,266
Message 50372 - Posted: 10 Jun 2024, 7:09:43 UTC - in response to Message 50369.  
Last modified: 10 Jun 2024, 7:13:11 UTC

Do these observations remain valid currently? In other words, the fewer cores you use the more efficient is the process?.
Multi-core is much faster, because the 400 events (atm) are divided by the number of threads.
ID: 50372 · Report as offensive     Reply Quote
Profile Yeti
Volunteer moderator
Avatar

Send message
Joined: 2 Sep 04
Posts: 455
Credit: 201,268,029
RAC: 6,930
Message 50373 - Posted: 10 Jun 2024, 8:38:11 UTC - in response to Message 50369.  

Do these observations remain valid currently? In other words, the fewer cores you use the more efficient is the process?
Thanks.

This is more from a theoretical sight. Each WU needs a startup-Sequence, that run's only on 1 Core. At the End of the WU you will get Idle-Cores until the last thread is finished.

In the past it has proven, that mid-Core configurations are best. I have preferred to run 4-Core-Tasks, it may vary depending on your personal needs


Supporting BOINC, a great concept !
ID: 50373 · Report as offensive     Reply Quote
Emmanuel Mar
Avatar

Send message
Joined: 9 Feb 09
Posts: 25
Credit: 2,449,651
RAC: 3,783
Message 50418 - Posted: 18 Jun 2024, 0:11:59 UTC
Last modified: 18 Jun 2024, 0:31:52 UTC

My experience with Atlas and other work with xeon w3680(130w) ddr3 1300 and i5 12600 (65w) ddr5 5600 both using Atlas, in both cases 8 threads or 4 cores were used, on the I5 12600(year2023) the tasks are completed in 3 hours and score 3000/4000 on the xeon w3680 (year 2009) the time doubles and triples the score is lower despite completing the task.

The energy expenditure per hour of the Xeon w3680 is double compared to the current generation for the same threads and if it met the same deadline, but it takes 2 to 3 times to complete the task.Mermory DDR3 1300 vs DDR5 5200,lot step up.

intel xeon w3680(130w) and i5 12600(65w) =12 theads/6 cores

16 years apart and that xeon already worked at the LHC at the time.2012

w3680 1 task 10 hours= 130w x 10=1300w
i5 12600 1 task 3 hours= 65w x 3=210w
Big energy difference, big performance difference and more tasks in the same day completed with much less electricity consumption.
16 years apart and that xeon already worked at the LHC at the time.2009 to 2012
ID: 50418 · Report as offensive     Reply Quote

Message boards : ATLAS application : 3-core task crunches slower than 2-core task - why so?


©2024 CERN