Message boards : CMS Application : New version v47.80
Message board moderation

To post messages, you must log in.

AuthorMessage
Profile Laurence
Project administrator
Project developer

Send message
Joined: 20 Jun 14
Posts: 372
Credit: 238,712
RAC: 0
Message 33766 - Posted: 10 Jan 2018, 9:54:58 UTC

This new version contains a configuration improvement for CVMFS which will hopefully result in the VMs booting quicker.
ID: 33766 · Report as offensive     Reply Quote
Erich56

Send message
Joined: 18 Dec 15
Posts: 1686
Credit: 100,447,314
RAC: 103,171
Message 33767 - Posted: 10 Jan 2018, 11:27:11 UTC

hm, on December 6, 2017, you wrote exactly the same for the new version 47.70.
What's the difference now between 47.70 and 47.80?
ID: 33767 · Report as offensive     Reply Quote
computezrmle
Volunteer moderator
Volunteer developer
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 15 Jun 08
Posts: 2386
Credit: 223,002,758
RAC: 136,195
Message 33768 - Posted: 10 Jan 2018, 12:51:33 UTC

I got the new vdi file on 1 of my hosts and it runs fine so far.

What makes me wonder is that the filename codes a date before xmas (CMS_2017_12_20.vdi).
It looks like the distribution of this version was postponed until today, was it?

According to my network logs the this version introduces a couple of new CVMFS communication domains:
s1bnl-cvmfs.openhtc.io
s1cern-cvmfs.openhtc.io
s1fnal-cvmfs.openhtc.io
s1ral-cvmfs.openhtc.io

Is this the expected change?


Cheers
ID: 33768 · Report as offensive     Reply Quote
Profile Laurence
Project administrator
Project developer

Send message
Joined: 20 Jun 14
Posts: 372
Credit: 238,712
RAC: 0
Message 33769 - Posted: 10 Jan 2018, 12:58:54 UTC - in response to Message 33767.  

Version 47.70 didn't work as expected to we had to do a few more configuration changes.
ID: 33769 · Report as offensive     Reply Quote
Profile Laurence
Project administrator
Project developer

Send message
Joined: 20 Jun 14
Posts: 372
Credit: 238,712
RAC: 0
Message 33770 - Posted: 10 Jan 2018, 13:02:16 UTC - in response to Message 33768.  

Yes. This should now use a content delivery network so the proxies used for the file downloads should be closer and hence improve the performance of CVMFS.
ID: 33770 · Report as offensive     Reply Quote
Erich56

Send message
Joined: 18 Dec 15
Posts: 1686
Credit: 100,447,314
RAC: 103,171
Message 33771 - Posted: 10 Jan 2018, 13:13:30 UTC - in response to Message 33770.  

Thanks, Laurence, for the information :-)
ID: 33771 · Report as offensive     Reply Quote
computezrmle
Volunteer moderator
Volunteer developer
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 15 Jun 08
Posts: 2386
Credit: 223,002,758
RAC: 136,195
Message 33776 - Posted: 10 Jan 2018, 20:18:30 UTC

Meanwhile I got a couple of CMS WUs.
1st phase until cmsRun is completely set up needs less than 4:45 min.
No obvious errors.

Good job Laurence.
ID: 33776 · Report as offensive     Reply Quote
Profile Laurence
Project administrator
Project developer

Send message
Joined: 20 Jun 14
Posts: 372
Credit: 238,712
RAC: 0
Message 33782 - Posted: 11 Jan 2018, 13:30:24 UTC - in response to Message 33776.  

I will have to go through the results now to measure the performance improvement.
ID: 33782 · Report as offensive     Reply Quote
computezrmle
Volunteer moderator
Volunteer developer
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 15 Jun 08
Posts: 2386
Credit: 223,002,758
RAC: 136,195
Message 33784 - Posted: 11 Jan 2018, 14:13:47 UTC - in response to Message 33782.  

To avoid confusion you may find it useful to know what I mean when I write "cmsRun is completely set up".
It's the moment when the download of
http://cmsfrontier.cern.ch:8000/FrontierProd/Frontier/type=frontier_request:1:DEFAULT&encoding=BLOBzip5&p1=eNptiz0KgDAMRq8ScgBx6OogyWcVrCltQZ1y-1tYcXV9PxU7pBFrjgrPEZYUrTItxRKxpOpih3oIl78Srdw8sJXZ5cs72aQ-TP-0XFFAvClPND6pWB3l

appears in the (tailed and filtered) download log.
This download is like a lighthouse because
- it's size is much larger compared to the downloads before
- it's always at the end of a download series
- it's the last download for a while
- the CPU usage stays stable near 100% after the download

Cheers
ID: 33784 · Report as offensive     Reply Quote
ivan
Volunteer moderator
Project tester
Volunteer developer
Volunteer tester
Project scientist
Avatar

Send message
Joined: 29 Aug 05
Posts: 998
Credit: 6,264,307
RAC: 71
Message 33785 - Posted: 11 Jan 2018, 14:16:06 UTC - in response to Message 33784.  

Yes, that's when the conditions database for the detector is downloaded and the geometry complete, so that the simulation of collisions can begin.
ID: 33785 · Report as offensive     Reply Quote
Erich56

Send message
Joined: 18 Dec 15
Posts: 1686
Credit: 100,447,314
RAC: 103,171
Message 33842 - Posted: 14 Jan 2018, 10:44:14 UTC

this is not directly related to the recent version changes, but still I'd like to ask this question, out of curiosity:

since some time ago (I dont's know exactly since when, I guess several weeks ago or so), the network traffic when running CMS tasks got reduced to about one third compared to what is was before.
Not that I am complaining about this - although for me the higher traffic (about 20GB per day for 8 CMS tasks running concurrently) was no problem at all, having an unlimited flat rate with my ISP (150Mbits down, 15Mbits up).
Whereas I am sure that crunchers with limited internet ressources will, of course, be very pleased.

What exactly has been changed, technically, with the CMS tasks, causing this marked reduction in internet traffic?
ID: 33842 · Report as offensive     Reply Quote
computezrmle
Volunteer moderator
Volunteer developer
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 15 Jun 08
Posts: 2386
Credit: 223,002,758
RAC: 136,195
Message 33843 - Posted: 14 Jan 2018, 11:05:50 UTC - in response to Message 33842.  

Is it upload or download?
How do you measure CMS transfers compared to other transfers, e.g. LHCb or even non LHC projects?
ID: 33843 · Report as offensive     Reply Quote
Erich56

Send message
Joined: 18 Dec 15
Posts: 1686
Credit: 100,447,314
RAC: 103,171
Message 33844 - Posted: 14 Jan 2018, 11:38:17 UTC

I am measuring the total traffic of each of my PCs with DUMeter.
Of course, this tool does not differentiate between the various programs running and their share of the total traffic.
However, when in my previous posting I was talking about approx. 20GB per day, this was mainly CMS. The only other project that is running on the same PC is GPUGRID, the up- and download figure of which is max. 0,5 GB per day.

DUMeter shows up- and downloads seperately, and from what I remember, about 2/3 to 3/4 of the total volume was upload, the rest was download (about same situation as it is now).
ID: 33844 · Report as offensive     Reply Quote

Message boards : CMS Application : New version v47.80


©2024 CERN