LHC@home

Hi

before we ramp up our pool of boinc hosts for ATLAS we would like to set up the cvmfs squid.

I did have a look at the threads dealing with this, but they are not so helpful, since these cover setting up your own squid instance. We do already have a cvmfs squid for our local WLCG Tier2 cluster and the obvious plan is to make our boinc hosts connect to it.

Does anybody have a working configuration for this scenario and could share it?

Thanks a lot, Stefan

We do already have a cvmfs squid for our local WLCG Tier2 cluster and the obvious plan is to make our boinc hosts connect to it.
Does anybody have a working configuration for this scenario and could share it?

Years ago, for Atlas-native I simply used the same basic configuration of CVMFS on the boinc nodes as on the Grid cluster nodes.
As you're going with VirtualBox I have no idea how it will work out: defining a proxy in BOINC will route all web traffic through it, including the VM's Frontier traffic as well as BOINC job requests. Also, AIUI the Atlas VMs here have the CVMFS preconfigured to use the Cloudflare stratum one server instead of the CERN ones in which case I don't think you'll get the expected efficiency either, as the squid would end up having to hold two copies of each file. My inclination would be to just run a BOINC-specific squid on each machine to hold the local CVMFS cache across successive VM instances.

---

As you're going with VirtualBox ... defining a proxy in BOINC will route all web traffic through it, including the VM's Frontier traffic as well as BOINC job requests.

Right.

... the Atlas VMs here have the CVMFS preconfigured to use the Cloudflare stratum one server instead of the CERN ones ...

Right.

... in which case I don't think you'll get the expected efficiency either, as the squid would end up having to hold two copies of each file.

In fact it happens (fail-over, multiple server aliases...) regarding CVMFS.
Regarding Frontier only in case of fail-over.
Even then a local proxy will be more efficient than a direct connection.

My inclination would be to just run a BOINC-specific squid on each machine to hold the local CVMFS cache across successive VM instances.

One single squid instance is powerful and efficient enough to serve several 100 worker nodes as long as there's no network bottleneck between squid and the worker nodes.
A separate squid instance on each worker box will reduce efficiency and increase maintenance effort.

I run just one computer with squid and it seems fine as computezrmle said, I only see upto 50 MB/s of network traffic so even the net requirements aren't that bad.

A separate squid instance on each worker box will reduce efficiency and increase maintenance effort.

As the OP claims to be close to a WLCG Tier2 site I figured network bandwidth wouldn't be an issue, and if there's enough machines that maintenance issues are significant then you want to be using a configuration management system anyway in which case I found keeping machines identical made things simpler! And on a corporate/enterprise network, a separate squid is likely to result in questions from network admins about what it's doing there, who can connect through it, and to where...

For CVMFS the biggest single gain is in maintaining the cache across successive VM initialisations; the local squid route gets that without affecting any other activity on the machine, and without exposing new services on the network.
If I was in the OP's situation - and I've been close - I'd start there. Once the OP has some experience and a track record with their networking team they can always take another step, whether that's the standalone BOINC squid or fudgerating the local squids to use the Tier2 one or whatever else. (It'll also depend on what else the machine is used for - the OP mentioned a separate batch system in another post.)

---