Message boards :
ATLAS application :
Atlas native: cvmfs2 - fuse - parrot
Message board moderation
Author | Message |
---|---|
Send message Joined: 2 May 07 Posts: 2267 Credit: 175,671,719 RAC: 2,060 ![]() ![]() |
cvmfs2 with fuse have for /cvmfs/alice.cern.ch, /cvmfs/atlas-cond.cern.ch, /cvmfs/atlas-nightlies.cern.ch, /cvmfs/atlas.cern.ch, /cvmfs/cernvm-prod.cern.ch, /cvmfs/cvmfs.config.cern.ch, /cvmfs/grid.cern.ch and /cvmfs/sft.cern.ch a reservation of 4 GByte as default. Means this every native Atlas is needing 32 GByte of data-space for this eight Data's? Is it possible to use this Data with parrot on a NAS or DAS-System from every VM in the local network of the Volunteer? |
![]() Send message Joined: 15 Jun 08 Posts: 2634 Credit: 272,025,947 RAC: 59,352 ![]() ![]() |
/cvmfs does not need any disk space except a single directory entry to provide the mountpoint. All subdirectories below /cvmfs are just mappings mounted by fuse. The real data is stored at the CERN datacenters (or others like OSG, ...). CVMFS retrieves it via HTTP on request and stores it in a local cache, usually in /var/lib/cvmfs (set by CVMFS_CACHE_BASE). 4GB is the default quota that tells CVMFS not to use more disk space below /var/lib/cvmfs than set via CVMFS_QUOTA_LIMIT. The cache size is shared among all configured repositories. Too low settings result in more HTTP requests, too high settings result in lots of outdated data in /var/lib/cvmfs. The efficiency can be checked with cvmfs_config stat Example: Running /usr/bin/cvmfs_config stat atlas.cern.ch: VERSION PID UPTIME(M) MEM(K) REVISION EXPIRES(M) NOCATALOGS CACHEUSE(K) CACHEMAX(K) NOFDUSE NOFDMAX NOIOERR NOOPEN HITRATE(%) RX(K) SPEED(K/S) HOST PROXY ONLINE 2.7.0.0 50842 5486 51440 60909 2 79 6558675 7077889 8400 65024 0 3471089 99.9488 2430880 7815 http://s1cern-cvmfs.openhtc.io/cvmfs/atlas.cern.ch http://<proxy_IP>:3128 1 Running /usr/bin/cvmfs_config stat atlas-nightlies.cern.ch: VERSION PID UPTIME(M) MEM(K) REVISION EXPIRES(M) NOCATALOGS CACHEUSE(K) CACHEMAX(K) NOFDUSE NOFDMAX NOIOERR NOOPEN HITRATE(%) RX(K) SPEED(K/S) HOST PROXY ONLINE 2.7.0.0 61859 5485 29960 33876 2 1 6558675 7077889 0 65024 0 1032 87.6938 354060 3254 http://s1cern-cvmfs.openhtc.io/cvmfs/atlas-nightlies.cern.ch http://<proxy_IP>:3128 1 In this example a CVMFS_QUOTA_LIMIT of 7GB results in a hitrate of >99% regarding a heavily used repository like atlas.cern.ch and a hitrate of 87% regarding atlas-nightlies.cern.ch which is used less often. 8 repositories use a total of 6.5 GB (>90% of 7GB). Fillgrade and hitrate together point out that 7GB might be a good quota for this CVMFS client. CVMFS together with the fuse automounter is the recommended combination to run on a single (linux) computer. The recommended way to support more CVMFS clients on different computers within the same LAN is to use an HTTP proxy (preferably Squid). This also applies if more than just a few VMs are running, each of them using it's own CVMFS client. |
Send message Joined: 2 May 07 Posts: 2267 Credit: 175,671,719 RAC: 2,060 ![]() ![]() |
In the default.local are atlas-nightlies.cern.ch and cvmfs-config.cern.ch not as entry for cvmfs_repositories. Do we need them in the default.local? Edit: Atlas-message is always: 9.5 Gbyte are needed. Sometime two or three tasks are downloaded, The last message is always, there is not enough disk space for 9.5 GByte! |
![]() Send message Joined: 15 Jun 08 Posts: 2634 Credit: 272,025,947 RAC: 59,352 ![]() ![]() |
In the default.local are atlas-nightlies.cern.ch and cvmfs-config.cern.ch not as entry for cvmfs_repositories. Right. Do we need them in the default.local? No. Your local CVMFS configuration lists only repositories that are required to start a task. Other repositories, e.g. atlas-nightlies.cern.ch, are added and configured on demand by the setup scripts. It is not recommended to list repositories in the local CVMFS configuration if they are not needed as that will cause periodical downloads of the repositories' catalog files which can easily be >20MB each. Atlas-message is always: 9.5 Gbyte are needed. I don't understand what is meant by this. If a quota is meant that is used by the BOINC client then it has nothing to do with CVMFS since the BOINC client doesn't control nor even read the size of the CVMFS directories. |
Send message Joined: 2 May 07 Posts: 2267 Credit: 175,671,719 RAC: 2,060 ![]() ![]() |
Atlas-message is always: 9.5 Gbyte are needed. This means, have for all native same configuration, but Atlas is downloading different number of tasks. The last have this message. Edit:Had now made a reboot. cvmfs2 is empty, only wipecache let downloading new tasks. Two ATM. |
Send message Joined: 2 May 07 Posts: 2267 Credit: 175,671,719 RAC: 2,060 ![]() ![]() |
Is the limit of 9.5 GByte for native-VM as message from Atlas a static parameter or a placeholder like 20 GByte in Virtualbox for Boinc-VM under Windows? |
Send message Joined: 2 May 07 Posts: 2267 Credit: 175,671,719 RAC: 2,060 ![]() ![]() |
Two native-VM are running without app_config.xml. plan_class was native_mt. They are running with WCG in combination, no problems so long. The other two will be proofed to run also without app_config.xml. One have also docker for -dev. |
©2025 CERN