Message boards : Sixtrack Application : Sixtrack BOINC application on track to be ported to GPUs?
Message board moderation

To post messages, you must log in.

1 · 2 · 3 · Next

AuthorMessage
bozz4science

Send message
Joined: 3 May 20
Posts: 8
Credit: 540,828
RAC: 737
Message 45005 - Posted: 25 May 2021, 16:40:44 UTC

Recently I have watched the recordings from the BOINC workshop this year and in a video from LHC@Home it was mentioned that the initial code base for the Sixtrack application goes back almost 30 yrs if I am not mistaken. Furthermore, Laurence Field mentioned in his talk that the application had already successfully been developed/deployed and tested server side at LHC and was found to be performing at a level roughly equivalent of 250 CPU cores.

As this was referred to as an ongoing investigation into the possibility of bringing this app to GPUs here on LHC@Home, I was wondering if so from the project lead / management could provide some information about the plans for the future in the sense of a roadmap for Q3/4 2021 for the Sixtrack application.

Thank you
ID: 45005 · Report as offensive     Reply Quote
tullio

Send message
Joined: 19 Feb 08
Posts: 704
Credit: 4,305,953
RAC: 871
Message 45006 - Posted: 25 May 2021, 17:14:32 UTC

GPU tasks on OpenPandemics at WorldCommunityGrid take about ten minutes on my GTX 1060, I believe a Pascal chip board. On GPUGRID I have a task running since two days on a GTX 1650 board, which uses a Turing chip. GPU tasks running times depend on many variables, they are not always a solution.
Tullio
ID: 45006 · Report as offensive     Reply Quote
bozz4science

Send message
Joined: 3 May 20
Posts: 8
Credit: 540,828
RAC: 737
Message 45008 - Posted: 25 May 2021, 18:18:46 UTC - in response to Message 45006.  

That much I am aware of. I see your point but don't get really get your reference in regard to my question. I am just curious in the potential energy efficiency boost and core allocation of my machine if this idea were to materialise into a BOINC application. And I wouldn't have brought it up, if it wasn't for the explicit note on the slides in the mentioned BOINC workshop talk.

Would highly appreciate any insight into LHC@H's plans on this!
ID: 45008 · Report as offensive     Reply Quote
Profile Viking69
Avatar

Send message
Joined: 24 Jul 05
Posts: 56
Credit: 5,542,209
RAC: 394
Message 45062 - Posted: 17 Jun 2021, 1:07:45 UTC

I cannot Hyper-V my Win 10 PC, so only sixtrack will work for me. I'd love to continue to crunch for little green men/women!
Let's crunch for our future.
ID: 45062 · Report as offensive     Reply Quote
Erich56

Send message
Joined: 18 Dec 15
Posts: 1532
Credit: 49,001,212
RAC: 58,235
Message 45063 - Posted: 17 Jun 2021, 3:17:30 UTC - in response to Message 45008.  

That much I am aware of. I see your point but don't get really get your reference in regard to my question. I am just curious in the potential energy efficiency boost and core allocation of my machine if this idea were to materialise into a BOINC application. And I wouldn't have brought it up, if it wasn't for the explicit note on the slides in the mentioned BOINC workshop talk.

Would highly appreciate any insight into LHC@H's plans on this!
+1
ID: 45063 · Report as offensive     Reply Quote
bozz4science

Send message
Joined: 3 May 20
Posts: 8
Credit: 540,828
RAC: 737
Message 45068 - Posted: 18 Jun 2021, 9:10:21 UTC
Last modified: 18 Jun 2021, 9:10:55 UTC

Here is the link of the YouTube video I referred to in my initial post. You can watch it here: 2021 BOINC Workshop - Day 02 - Laurence Field

By the way, here is a recent news post (27 May, 2021), that mentioned the video recordings of the 2 talks given by LHC representatives at the BOINC workshop. Both very worthwhile watching IMHO. Cheers
ID: 45068 · Report as offensive     Reply Quote
Toby Broom
Volunteer moderator

Send message
Joined: 27 Sep 08
Posts: 731
Credit: 504,817,506
RAC: 284,539
Message 45069 - Posted: 18 Jun 2021, 17:16:02 UTC
Last modified: 18 Jun 2021, 17:17:58 UTC

In 2016 when a it was investigated sixtrack uses FP64 compute which consumer GPU are very weak on and the memory bandwidth that GPU have are not needed.

https://indico.cern.ch/event/506317/contributions/2017945/attachments/1241758/1826458/SixTrackGPU.pdf

in 2018 it was revisited, you can see that a GTX 1080 is similar same speed as a 20 core low clock speed Xeon CPU, where as a Tesla V100 is 16x quicker.

https://bt.pa.msu.edu/CPO-10/talks/23Tue/AM1/S2F/23Tue_AM1_0915_S2F_De_Maria.pdf

Given that most people don't have a Tesla V100, even if there was a GPU version the real world increase performance would be low, maybe double assuming people have high end GPU's like 1080.

Given that sixtrack work is bursty, it does not seem like CERN even needs more calculations in sixtrack, so why develop a faster method if its not needed

Since sixtrack is open source then I would propose that you can build your on GPU version and test if it gives a performance boost yourself, maybe then you can pitch this to the CERN team for integration or could be deployed via the BOINC anonymous platform from your own hosted source.
ID: 45069 · Report as offensive     Reply Quote
Toby Broom
Volunteer moderator

Send message
Joined: 27 Sep 08
Posts: 731
Credit: 504,817,506
RAC: 284,539
Message 45070 - Posted: 18 Jun 2021, 17:30:27 UTC

ID: 45070 · Report as offensive     Reply Quote
bozz4science

Send message
Joined: 3 May 20
Posts: 8
Credit: 540,828
RAC: 737
Message 45071 - Posted: 18 Jun 2021, 18:10:46 UTC
Last modified: 18 Jun 2021, 18:12:57 UTC

Thanks Toby for referencing this helpful information. I will make time tonight to carefully read all about the Sixtrack app. I definitely see why this wasn't enforced/followed up on as nearly no volunteer would meet the requirements that would allow for this performance boost. Just was curious after it was explicitly mentioned in the talk and stated on the slides to know about the application and potential plans surrounding it. And I also see the point of infrequent Sixtrack work submission.

Unfortunately, I am not proficient in the programming languages needed to develop such a CUDA/opencl app. I'll be happy to continue contributing with my little machines to the CPU app version whenever there is work. But I'll take a closer look at the open source code as you sparked my interest to dig a little deeper.

Thx again for your sources! (The last paper/slide deck mentioned from 2020 definitely looks intimidating...)
ID: 45071 · Report as offensive     Reply Quote
[VENETO] boboviz
Avatar

Send message
Joined: 7 May 08
Posts: 141
Credit: 1,404,162
RAC: 109
Message 45083 - Posted: 26 Jun 2021, 13:54:53 UTC - in response to Message 45071.  

I definitely see why this wasn't enforced/followed up on as nearly no volunteer would meet the requirements that would allow for this performance boost. Just was curious after it was explicitly mentioned in the talk and stated on the slides to know about the application and potential plans surrounding it. And I also see the point of infrequent Sixtrack work submission.


Maybe will be more wus when LHC will restart (September/October).
I hope gpu will be published...
ID: 45083 · Report as offensive     Reply Quote
maeax

Send message
Joined: 2 May 07
Posts: 1620
Credit: 75,382,983
RAC: 218,874
Message 45102 - Posted: 4 Jul 2021, 8:19:21 UTC

ID: 45102 · Report as offensive     Reply Quote
[VENETO] boboviz
Avatar

Send message
Joined: 7 May 08
Posts: 141
Credit: 1,404,162
RAC: 109
Message 45124 - Posted: 14 Jul 2021, 10:36:48 UTC

Kronos Group released OpenCL 3.0.8, with some fixes, clarifications and a "useful extension for accelerated inferencing using quantized neural networks".
ID: 45124 · Report as offensive     Reply Quote
Filipe

Send message
Joined: 9 Aug 05
Posts: 36
Credit: 7,437,122
RAC: 137
Message 45222 - Posted: 18 Aug 2021, 11:11:25 UTC

Any update?
ID: 45222 · Report as offensive     Reply Quote
[VENETO] boboviz
Avatar

Send message
Joined: 7 May 08
Posts: 141
Credit: 1,404,162
RAC: 109
Message 45386 - Posted: 24 Sep 2021, 7:22:48 UTC - in response to Message 45222.  

Any update?


Yep, with 8 hrs cpu wus, a gpu app will be welcome :-)
ID: 45386 · Report as offensive     Reply Quote
maeax

Send message
Joined: 2 May 07
Posts: 1620
Credit: 75,382,983
RAC: 218,874
Message 45387 - Posted: 24 Sep 2021, 10:39:54 UTC - in response to Message 45386.  

HL-LHC is on the way (2022).
So, no time for new software, to reduce the runtime.
Only the success of proven.
ID: 45387 · Report as offensive     Reply Quote
[VENETO] boboviz
Avatar

Send message
Joined: 7 May 08
Posts: 141
Credit: 1,404,162
RAC: 109
Message 45388 - Posted: 24 Sep 2021, 13:44:44 UTC - in response to Message 45387.  

HL-LHC is on the way (2022).


?? HL-LHC will start in 2025. This is the Run3 of LHC.
ID: 45388 · Report as offensive     Reply Quote
Frederik

Send message
Joined: 22 Mar 16
Posts: 8
Credit: 112,715
RAC: 0
Message 45653 - Posted: 9 Nov 2021, 11:01:14 UTC

Hello,

Currently our teams are developing a new tracking code, named XTrack, as an addition/alternative to SixTrack.
As it is written in python with JIT to C, the possibility to run on GPUs was a logical step (also leveraging the experience done by writing sixtracklib, the code referenced above). The code hence runs natively on CPU and GPUs alike.
It is still in its early stages (less than a year old), but is showing remarkable stability and reliance, so much that it is already being used in production research by some people.

The aim is to make our BOINC environment compatible with it, implying that we would be able to start running on BOINC GPUs as well. However this is by far not an easy task, as the current implementation is completely tailored to the SixTrack executables. Hence getting XTrack on BOINC implies a total overhaul of our infrastructure. We will almost certainly do it, but we will need some time. I'd estimate this to be in the order of 1-1.5 years. Difficult to pin point, as it also depends on division of work - we have new people coming in, so the timeframe is rather flexible.

Cheers

Frederik
ID: 45653 · Report as offensive     Reply Quote
bozz4science

Send message
Joined: 3 May 20
Posts: 8
Credit: 540,828
RAC: 737
Message 45654 - Posted: 9 Nov 2021, 16:59:11 UTC
Last modified: 9 Nov 2021, 17:01:14 UTC

Exciting news! I'll read through the whole message again later to digest all of the information you relayed to us!

I am curious though as to what your affilition to LHC is as you mention "your teams" a couple times. Your structured and informed message lets me only speculate but I'd guess that you are part of a team of scienctiests working directly with LHC, am I right? Is there already a public Github or Gitlab repo for the XTrack code where we can take a look and track the progress? Can you give us a ballpark number of what performance improvement we could expect with a GPU client version?

Surely, your plan sounds ambitious to deploy the new application here on LHC@Home in the future but I am excited to see the sparked interest in improving the SixTrack code! I'll keep tabs on you and would appreciate if you could occasionally update us on the overall progress.

Have a great week
ID: 45654 · Report as offensive     Reply Quote
computezrmle
Volunteer moderator
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 15 Jun 08
Posts: 2048
Credit: 154,212,691
RAC: 148,045
Message 45658 - Posted: 10 Nov 2021, 9:25:11 UTC - in response to Message 45653.  

Back in Oct 2016 milkyway@home had a huge server side performance problem mainly caused by very short runtimes in connection with modern GPUs getting faster and faster.
Details can be seen here together with a suggestion that helps to attenuate the issue:
https://milkyway.cs.rpi.edu/milkyway/forum_thread.php?id=4042#65537

Since then GPUs have reached much higher performance levels, so I'm sure other measures might also be required to ensure the project servers and the DB behind them can handle the expected throughput.
I remember LHC@home already had DB performance issues although it doesn't yet have a GPU app.
The mentioned time frame of 1-1.5 years should be used to develop a solution.
ID: 45658 · Report as offensive     Reply Quote
Frederik

Send message
Joined: 22 Mar 16
Posts: 8
Credit: 112,715
RAC: 0
Message 45659 - Posted: 10 Nov 2021, 9:43:39 UTC - in response to Message 45654.  

Indeed, I work at CERN on the LHC. I'm Alessio's successor (in being the link between the beams and IT departments concerning BOINC).
Also, as until last year I was the main user of BOINC, you've probably been crunching my jobs :-)
ID: 45659 · Report as offensive     Reply Quote
1 · 2 · 3 · Next

Message boards : Sixtrack Application : Sixtrack BOINC application on track to be ported to GPUs?


©2022 CERN