Message boards : Number crunching : Recommended CVMFS Configuration for native Apps
Message board moderation

To post messages, you must log in.

AuthorMessage
computezrmle
Volunteer moderator
Volunteer developer
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 15 Jun 08
Posts: 2386
Credit: 222,993,861
RAC: 136,385
Message 41861 - Posted: 9 Mar 2020, 21:03:12 UTC

As of 2020-03-09 the recommended CVMFS settings to run ATLAS native and Theory native are as follows.
Volunteers are asked to check/update their local configuration.

/etc/cvmfs/default.local
CVMFS_REPOSITORIES="atlas,atlas-condb,grid,cernvm-prod,sft,alice"
CVMFS_SEND_INFO_HEADER=yes


/etc/cvmfs/domain.d/cern.ch.local
CVMFS_SERVER_URL="http://s1cern-cvmfs.openhtc.io/cvmfs/@fqrn@;http://s1ral-cvmfs.openhtc.io/cvmfs/@fqrn@;http://s1bnl-cvmfs.openhtc.io/cvmfs/@fqrn@;http://s1fnal-cvmfs.openhtc.io/cvmfs/@fqrn@;http://s1unl-cvmfs.openhtc.io/cvmfs/@fqrn@;http://s1asgc-cvmfs.openhtc.io:8080/cvmfs/@fqrn@;http://s1ihep-cvmfs.openhtc.io/cvmfs/@fqrn@"


/etc/cvmfs/config.d/atlas-nightlies.cern.ch.local
CVMFS_SERVER_URL="http://s1cern-cvmfs.openhtc.io/cvmfs/@fqrn@;http://s1bnl-cvmfs.openhtc.io/cvmfs/@fqrn@"



The mentioned config files may contain additional settings but those listed above are essential.





Experienced volunteers who are familiar with their internet neighborhood may resort the server list based on their home region (Asia, Europe, North America). As a rule of thumb the closest server should be listed first, followed by the 2nd closest and so on.
No server should be removed to ensure it can be used for failover.
In case of openhtc.io all servers mentioned in the list resolve to the closest Cloudflare proxy but the names include a code to tell Cloudflare where to get updates from:
s1cern (Switzerland)
s1ral (UK)
s1bnl (US, Brookhaven)
s1fnal (US, Fermilab)
s1unl (US, Nebraska)
s1asgc (Taiwan)
s1ihep (China)


Only if the server lists have been resorted, geolocation may be switched off.
Add to:
/etc/cvmfs/domain.d/cern.ch.local
/etc/cvmfs/config.d/atlas-nightlies.cern.ch.local
CVMFS_USE_GEOAPI=no



As always:
To activate the changes run "sudo cvmfs_config reload".
ID: 41861 · Report as offensive
Jim1348

Send message
Joined: 15 Nov 14
Posts: 602
Credit: 24,371,321
RAC: 0
Message 41862 - Posted: 9 Mar 2020, 21:15:06 UTC - in response to Message 41861.  

As of 2020-03-09 the recommended CVMFS settings to run ATLAS native and Theory native are as follows.
Volunteers are asked to check/update their local configuration.

/etc/cvmfs/default.local
CVMFS_REPOSITORIES="atlas,atlas-condb,grid,cernvm-prod,sft,alice"
CVMFS_SEND_INFO_HEADER=yes

Thanks a lot. I have recently been downloading this file:
sudo wget https://lhcathome.cern.ch/lhcathome/download/default.local -O /etc/cvmfs/default.local

Do you think that gives the same result?
ID: 41862 · Report as offensive
computezrmle
Volunteer moderator
Volunteer developer
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 15 Jun 08
Posts: 2386
Credit: 222,993,861
RAC: 136,385
Message 41863 - Posted: 9 Mar 2020, 21:35:07 UTC - in response to Message 41862.  

Do you think that gives the same result?

In this case: YES
Since CVMFS_DEFAULT_DOMAIN=cern.ch is usually set it will expand all repository names without a dot.
Example:
atlas --> atlas.cern.ch

Not used here but inside a CMS VM:
oasis.opensciencegrid.org --> oasis.opensciencegrid.org
ID: 41863 · Report as offensive
gyllic

Send message
Joined: 9 Dec 14
Posts: 202
Credit: 2,533,875
RAC: 0
Message 41886 - Posted: 11 Mar 2020, 17:30:04 UTC - in response to Message 41861.  
Last modified: 11 Mar 2020, 17:30:31 UTC

/etc/cvmfs/config.d/atlas-nightlies.cern.ch.local
CVMFS_SERVER_URL="http://s1cern-cvmfs.openhtc.io/cvmfs/@fqrn@;http://s1bnl-cvmfs.openhtc.io/cvmfs/@fqrn@"


What is this needed for? Sorry if this has been explained before; was absent for some time.
ID: 41886 · Report as offensive
computezrmle
Volunteer moderator
Volunteer developer
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 15 Jun 08
Posts: 2386
Credit: 222,993,861
RAC: 136,385
Message 41887 - Posted: 11 Mar 2020, 19:00:23 UTC - in response to Message 41886.  

/etc/cvmfs/config.d/atlas-nightlies.cern.ch.local
CVMFS_SERVER_URL="http://s1cern-cvmfs.openhtc.io/cvmfs/@fqrn@;http://s1bnl-cvmfs.openhtc.io/cvmfs/@fqrn@"


What is this needed for? Sorry if this has been explained before; was absent for some time.

Unlike other repositories atlas-nightlies.cern.ch is currently only available from CERN and BNL.
Hence ATLAS tasks mount atlas-nightlies.cern.ch at startup using settings from /cvmfs/cvmfs-config.cern.ch/etc/cvmfs/config.d/atlas-nightlies.cern.ch.conf.
Since this configuration file has a higher ranking than /etc/cvmfs/domain.d/cern.ch.local we have to place the openhtc.io servers in /etc/cvmfs/config.d/atlas-nightlies.cern.ch.local to overrule the automatic configuration.

The shorter list here ensures that requests to CVMFS servers not serving atlas-nightlies.cern.ch will be avoided.
ID: 41887 · Report as offensive
computezrmle
Volunteer moderator
Volunteer developer
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 15 Jun 08
Posts: 2386
Credit: 222,993,861
RAC: 136,385
Message 44234 - Posted: 30 Jan 2021, 16:41:14 UTC

A new version of the HowTo can be found here:
https://lhcathome.cern.ch/lhcathome/forum_thread.php?id=5594

New comments and questions should be posted here:
https://lhcathome.cern.ch/lhcathome/forum_thread.php?id=5595
ID: 44234 · Report as offensive

Message boards : Number crunching : Recommended CVMFS Configuration for native Apps


©2024 CERN