LHC@home

This is the default.local ftm,
modified with CVMFS_IPFAMILY_PREFER for using IPv4 and/or IPv6:

CVMFS_REPOSITORIES="atlas,atlas-condb,grid,cernvm-prod,sft,alice"
CVMFS_USE_CDN=yes
CVMFS_HTTP_PROXY="http://xx.xxx.xxx.xx:3128;DIRECT"
CVMFS_KCACHE_TIMEOUT=2
CVMFS_MAX_RETRIES=3
CVMFS_IPFAMILY_PREFER=4|6
CVMFS_USE_GEOAPI=yes

Important is for the PROXY the second Parameter DIRECT - This is the fallover!!

btw: The CVMFS-Scratch is now working for me!

modified with CVMFS_IPFAMILY_PREFER for using IPv4 and/or IPv6:
.
.
.
CVMFS_IPFAMILY_PREFER=4|6

There's usually no reason to configure CVMFS_IPFAMILY_PREFER.
https://cvmfs.readthedocs.io/en/stable/cpt-configure.html#ip-protocol-version


In fact, if you do it, do it right!
The way it's listed above is wrong since the configuration lines are run by a shell which interprets an unquoted "|" as a pipe.
Either use
CVMFS_IPFAMILY_PREFER=4
or use
CVMFS_IPFAMILY_PREFER=6

CVMFS_HTTP_PROXY="http://xx.xxx.xxx.xx:3128;DIRECT"
Important is for the PROXY the second Parameter DIRECT - This is the fallover!!

If it's configured that way, "cvmfs_config stat" should be run to ensure the proxy is NOT bypassed under normal conditions.

Users running no local proxy or configure a proxy with WPAD should set this parameter to:

CVMFS_HTTP_PROXY="auto;DIRECT"

CVMFS_IPFAMILY_PREFER=4|6
CVMFS_USE_GEOAPI=yes
Important is for the PROXY the second Parameter DIRECT - This is the fallover!!
btw: The CVMFS-Scratch is now working for me!

1. Only 36 MByte access.log since 20 hours. Showing Application x-CVMFS.
2. Showing IPv6-Adresses in access.log.
3. Scratch-CVMFS works.

The cleaning of the access.log and cache.log does the RedHat-VM itself (daily at 3 hour localtime)

VM with CentOS8(Stream) including Squid have 50% more Creditpoints for LHC@Home
with the same running Tasks (Atlas, CMS and Theory) as before.
No LAN-Conflict as with Squid as a Standalone Program under Windows!
Hoping the Squid Data is not readable for other people, had made this experience under Windows.

For ipv6 using,
acl localnet source need ipv6 Adress together with ipv4.

Examples:

acl localnet src fc00::/7       	# RFC 4193 local private network range
acl localnet src fe80::/10      	# RFC 4291 link-local (directly plugged) machines

as described here:
http://www.squid-cache.org/Doc/config/acl/

Further information:
https://wiki.squid-cache.org/Features/IPv6

Just tested with Alpine Linux 3.16 bundled Squid version 5.5 and this upload issue is still present.

So the recommendation still stands true to this day:

The preferred version should be the most recent squid package from your Linux distribution repository.
Version >=3.5.27 and <5.x

Is this a problem of Squid, when most of the multiattach Atlas-Tasks running 1 hour (3.600 sec.)?
Local cache digest enabled; rebuild/rewrite every 3600/3600 sec
How is it possible to change?
This was a question for multiattach of Atlas,
but Moderator changed this question to this folder!

Add this line to squid.conf, then reload squid:

digest_generation off

The digest is not required as long as there is only 1 squid instance.
Even if there are just a few sibling squids using digests may just add more load.

Is this a problem of Squid, when most of the multiattach Atlas-Tasks running 1 hour (3.600 sec.)?
Local cache digest enabled; rebuild/rewrite every 3600/3600 sec
How is it possible to change?
This was a question for multiattach of Atlas,
but Moderator changed this question to this folder!

No.
This has nothing to do with ATLAS nor with Multiattach.
It's clearly a Squid configuration question, hence I moved it here and explained it in the comment you already got.
I also gave an answer how you can switch digests off.

Squid 5.x workaround to make large uploads work

In squid.conf set

client_request_buffer_max_size xxx yy

with "xxx yy" being larger than the expected maximum upload size.
So far "512 MB" should be fine since in the past there were ATLAS uploads with close to but less than 500 MB.


Even tested that workaround didn't succeed on my systems for Squid versions < v5.6.
Thanks to Evangelos Katikos who reminded me to repeat the test with v5.6 and v5.7.

Thank you Evangelos Katikos.
Threadripper is now running with Squid for Atlas, also -dev.

Tried it with Squid 5.2 from Ubuntu 22.04 repository and it worked, ATLAS uploads went through ok. I wonder what was different with prior Squids that they didn't need this flag?

... what was different with prior Squids that they didn't need this flag?

It might be related to this:
https://bugs.squid-cache.org/show_bug.cgi?id=5214

Setting "client_request_buffer_max_size" should be seen as a workaround.
Just think about what would happen if a project tries to upload a file that is larger than the configured buffer.

What about my solution?

Set for "lhcathome-upload.cern.ch" the no proxy option in Boinc-Manager.

On my side this works like a charm for weeks now.

It depends on the szenario the Squid is part of.

In your case you allow (BOINC-)clients to bypass the proxy.
This works fine as long as your firewall policy allows direct HTTP traffic between the clients and external servers.

Other szenarios may configure Squid as part of the firewall and force all HTTP traffic through Squid.
Even if clients are configured not to use a proxy, they may not even notice the redirection.
Here, Squid must reliably handle the traffic.


Another point is that the project's vdi files are distributed via 'lhcathome-upload.cern.ch' (although they are downloads).
The suggested squid.conf allows to store those large files in the cache for multiple reuse.
This does not work if you bypass Squid.

A recent example related to the dev project's vdi files.

This morning I downloaded CMS_2022_09_07.vdi.gz (1.6 GB) from the dev server to test a new app version.
Since the same *.gz file was also used for the previous app version a few days ago Squid still had a fresh copy in it's cache.
As a result the BOINC client completed the download within 16 seconds.

Mo 07 Nov 2022 09:10:35 CET | lhcathome-dev | Started download of CMS_2022_09_07.vdi
Mo 07 Nov 2022 09:10:51 CET | lhcathome-dev | Finished download of CMS_2022_09_07.vdi

[07/Nov/2022:09:10:50 +0100] "GET http://lhcathome-test.cern.ch/lhcathome-dev/download/CMS_2022_09_07.vdi.gz HTTP/1.1" 200 1607884608 "-" "BOINC client (x86_64-suse-linux-gnu 7.21.0)" TCP_REFRESH_UNMODIFIED:HIER_DIRECT

How is it possible to protect the Info of Squid-IP Clearname in the Tasks of Atlas or CMS?

Now I tried the workaround like suggested:
"client_request_buffer_max_size 512 MB"
Works so far, but while uploading squid 5.7 takes 100% of one logical core.
Is this common behavior?