Message boards : Number crunching : Max # CPUs vs projects?
Message board moderation

To post messages, you must log in.

AuthorMessage
Toby Broom
Volunteer moderator

Send message
Joined: 27 Sep 08
Posts: 700
Credit: 444,765,736
RAC: 188,239
Message 28340 - Posted: 4 Jan 2017, 17:43:15 UTC

I changed my max number of CPU's to 2 from no limit and was expecting to get some dual core WU's. The LHCb ones that came down are single core. Are any of the projects enabled for multi core?
ID: 28340 · Report as offensive     Reply Quote
Crystal Pellet
Volunteer moderator
Volunteer tester

Send message
Joined: 14 Jan 10
Posts: 1055
Credit: 6,654,350
RAC: 1,567
Message 28341 - Posted: 4 Jan 2017, 17:51:15 UTC

Not for this project, but only for the Development for Virtual LHC@home project
ID: 28341 · Report as offensive     Reply Quote
Jesse Viviano

Send message
Joined: 12 Feb 14
Posts: 71
Credit: 1,807,236
RAC: 1,215
Message 28342 - Posted: 4 Jan 2017, 17:55:27 UTC - in response to Message 28340.  

ATLAS@home at https://lhcathome.cern.ch/ATLAS/ provides up to 8-core tasks.
ID: 28342 · Report as offensive     Reply Quote
Toby Broom
Volunteer moderator

Send message
Joined: 27 Sep 08
Posts: 700
Credit: 444,765,736
RAC: 188,239
Message 28344 - Posted: 4 Jan 2017, 18:07:20 UTC

Thanks, I tried some other settings to see if I get any multicore ones. Maybe CMS or Theory.

I have some Atlas ones crunching on my computer already.

I have 0.1% problems on how best to fill the 56 cores on my best PC :D

I assume the ones from dev project will get moved over when stable.
ID: 28344 · Report as offensive     Reply Quote
Profile Laurence
Project administrator
Project developer

Send message
Joined: 20 Jun 14
Posts: 348
Credit: 237,918
RAC: 0
Message 28347 - Posted: 4 Jan 2017, 19:59:05 UTC - in response to Message 28344.  

We will aim to add the ATLAS application here in January and this is now multicore by default. Once this has been done and the Theory, LHCb and CMS multicore version have been verified in the dev project, we can add them here.
ID: 28347 · Report as offensive     Reply Quote
Toby Broom
Volunteer moderator

Send message
Joined: 27 Sep 08
Posts: 700
Credit: 444,765,736
RAC: 188,239
Message 28351 - Posted: 5 Jan 2017, 6:53:58 UTC

Thanks Laurence, I'm running 2 instances of BOINC on the same machine at the moment this allows me to run more than 24 WU at once.
ID: 28351 · Report as offensive     Reply Quote
maeax

Send message
Joined: 2 May 07
Posts: 1364
Credit: 41,215,531
RAC: 26,119
Message 28606 - Posted: 22 Jan 2017, 10:34:23 UTC
Last modified: 22 Jan 2017, 11:00:26 UTC

Max. runtime of LHC 18 hours!

Multi-Core Tasks were finishing each task in the own CPU.

If the last Task is sherpa or Herwig++, the other CPU's must wait, until the last CPU is coming to the end of the Task.

Multi-Core is useful if TASKS for each CPU have nearby the same runtime.

LHC-Tasks have a great runtime difference.

Any idea?

Edit: Have for LHC Multi-Core Task in dev a difference of the last CPU finishing from 1.5 hours.
ID: 28606 · Report as offensive     Reply Quote
Crystal Pellet
Volunteer moderator
Volunteer tester

Send message
Joined: 14 Jan 10
Posts: 1055
Credit: 6,654,350
RAC: 1,567
Message 28608 - Posted: 22 Jan 2017, 12:15:25 UTC - in response to Message 28606.  

Multi-Core is useful if TASKS for each CPU have nearby the same runtime.

LHC-Tasks have a great runtime difference.

Any idea?

Edit: Have for LHC Multi-Core Task in dev a difference of the last CPU finishing from 1.5 hours.

I agree when you have enough RAM to run multiple single core tasks is more efficient.
The advantage of running multicore VM's is for those with less RAM be able to do more jobs, but jobs with about the same run time would be the best solution to avoid idling CPU's.
btw: multi-core is not (yet) enabled in this production project, only in LHC's dev-project.
ID: 28608 · Report as offensive     Reply Quote
Jesse Viviano

Send message
Joined: 12 Feb 14
Posts: 71
Credit: 1,807,236
RAC: 1,215
Message 28634 - Posted: 23 Jan 2017, 16:16:24 UTC

I would think that actually running multicore jobs could be more efficient barring any limitations like poor schedulers that leave cores idle. Because more cores are executing the same process, there are fewer processes fighting over the last level cache(s) and the memory controller(s). Therefore, there is less likelihood that a competing process would cause a cache block eviction, leading to more cache hits so that more work gets done in less time and therefore less memory system overhead due to fewer processes fighting over it and less need to actually go out to DRAM.
ID: 28634 · Report as offensive     Reply Quote
Profile Laurence
Project administrator
Project developer

Send message
Joined: 20 Jun 14
Posts: 348
Credit: 237,918
RAC: 0
Message 28661 - Posted: 25 Jan 2017, 9:59:00 UTC - in response to Message 28634.  

Here is a blog post and presentation covering some of the fun with using multi-core VMs internally for our batch work nodes in OpenStack.

It seems that there is some scope for some tuning.
ID: 28661 · Report as offensive     Reply Quote
Toby Broom
Volunteer moderator

Send message
Joined: 27 Sep 08
Posts: 700
Credit: 444,765,736
RAC: 188,239
Message 28668 - Posted: 25 Jan 2017, 18:26:59 UTC

I imagine that this doesn't effect so many people I only have 1 (2 at the moment) computer with NUMA nodes in my dual machine. The rest are all single processor machines.

The testing that Wenjing Wu did looks like it could be similar to the other CERN testing.

I seem like I get better BOINC RAC with lots of low core count tasks, is this maybe due to some miss calculation with the credits?
ID: 28668 · Report as offensive     Reply Quote

Message boards : Number crunching : Max # CPUs vs projects?


©2022 CERN