LHC@home

I actually don't have that many processors. Most of those are from a single PC with multiple installations (Windows, Hyper-V Ubuntu, WSL2s). That PC is Ryzen 5900X with 32 GB RAM which is what I use for Atlas & Theory. Due to RAM constraints I should be able to run anywhere form 8 1-core to 2 12-core native ATLAS tasks simultaneously or up to 24 native Theory tasks simultaneously. My understanding is that Hyper-V is a type 1 hypervisor so would be considered bare metal. That would make both Windows and Ubuntu installations VMs, even though that sounds strange/wrong. In that case it seems like installing Squid on Hyper-V Ubuntu would be the way to go. Is my thinking correct here?

In this context "bare metal" would mean real hardware, so (simply spoken) a case with a CPU, mainboard, RAM, ...
=> your Ryzen 5900X box

I suspect the 1st OS that is active when you boot the system is Windows.
If yes I would suggest to run the Windows version of Squid.
This promises best stability, performance and availability as well as least overhead and least resource usage compared to a Squid installed on any of the hypervisors or inside a SW container.

At boot time you should ensure Squid is already running before you (your clients) send the first request to it.
At shutdown time you should ensure Squid is still running when you send the last request to it.


At least in the past using different hypervisors concurrently (not only but especially Hyper-V and VirtualBox) caused many problems and should be avoided.
Hence, if you run a Squid inside a VM and you need to disable that VM/hypervisor Squid's service would also be unavailable.

With Squid version 5.0.6 I'm having difficulties uploading results to LHC.
In the logs I seem to be getting a lot of messages like these:

1635870641.453    135 192.168.3.69 TCP_MISS/200 348 POST http://lhcathome-upload.cern.ch/lhcathome_cgi/file_upload_handler - HIER_DIRECT/188.185.101.107 text/plain
1635870643.445    120 192.168.3.69 TCP_MISS_ABORTED/100 0 POST http://lhcathome-upload.cern.ch/lhcathome_cgi/file_upload_handler - HIER_DIRECT/188.185.101.107 -
1635871151.414    332 192.168.3.69 TCP_MISS/200 329 POST http://lhcathome-upload.cern.ch/lhcathome_cgi/file_upload_handler - HIER_DIRECT/188.185.125.226 text/plain
1635871151.414    332 192.168.3.69 TCP_MISS/200 348 POST http://lhcathome-upload.cern.ch/lhcathome_cgi/file_upload_handler - HIER_DIRECT/188.185.125.226 text/plain
1635871153.135    129 192.168.3.69 TCP_MISS_ABORTED/100 0 POST http://lhcathome-upload.cern.ch/lhcathome_cgi/file_upload_handler - HIER_DIRECT/188.185.125.226 -
1635871153.137    132 192.168.3.69 TCP_MISS_ABORTED/100 0 POST http://lhcathome-upload.cern.ch/lhcathome_cgi/file_upload_handler - HIER_DIRECT/188.185.125.226 -
1635874194.290    155 192.168.3.69 TCP_MISS/200 348 POST http://lhcathome-upload.cern.ch/lhcathome_cgi/file_upload_handler - HIER_DIRECT/188.185.75.236 text/plain
1635874194.301    165 192.168.3.69 TCP_MISS/200 353 POST http://lhcathome-upload.cern.ch/lhcathome_cgi/file_upload_handler - HIER_DIRECT/188.185.75.236 text/plain
1635874196.274    134 192.168.3.69 TCP_MISS_ABORTED/100 0 POST http://lhcathome-upload.cern.ch/lhcathome_cgi/file_upload_handler - HIER_DIRECT/188.185.75.236 -
1635874196.284    144 192.168.3.69 TCP_MISS_ABORTED/100 0 POST http://lhcathome-upload.cern.ch/lhcathome_cgi/file_upload_handler - HIER_DIRECT/188.185.75.236 -
1635877114.137     90 192.168.3.69 TCP_MISS/200 348 POST http://lhcathome-upload.cern.ch/lhcathome_cgi/file_upload_handler - HIER_DIRECT/188.185.75.236 text/plain
1635877114.137     90 192.168.3.69 TCP_MISS/200 348 POST http://lhcathome-upload.cern.ch/lhcathome_cgi/file_upload_handler - HIER_DIRECT/188.185.75.236 text/plain
1635877115.554    101 192.168.3.69 TCP_MISS_ABORTED/100 0 POST http://lhcathome-upload.cern.ch/lhcathome_cgi/file_upload_handler - HIER_DIRECT/188.185.75.236 -
1635877115.561    108 192.168.3.69 TCP_MISS_ABORTED/100 0 POST http://lhcathome-upload.cern.ch/lhcathome_cgi/file_upload_handler - HIER_DIRECT/188.185.75.236 -
1635883410.768    157 192.168.3.69 TCP_MISS/200 329 POST http://lhcathome-upload.cern.ch/lhcathome_cgi/file_upload_handler - HIER_DIRECT/188.185.86.60 text/plain
1635883410.769    158 192.168.3.69 TCP_MISS/200 348 POST http://lhcathome-upload.cern.ch/lhcathome_cgi/file_upload_handler - HIER_DIRECT/188.185.86.60 text/plain
1635883412.859    104 192.168.3.69 TCP_MISS_ABORTED/100 0 POST http://lhcathome-upload.cern.ch/lhcathome_cgi/file_upload_handler - HIER_DIRECT/188.185.86.60 -
1635883412.884    129 192.168.3.69 TCP_MISS_ABORTED/100 0 POST http://lhcathome-upload.cern.ch/lhcathome_cgi/file_upload_handler - HIER_DIRECT/188.185.86.60 -

Downgrading to version 4.17 solved the blocked upload.

Any idea what has evolved between these versions to explain this ?

Any idea what has evolved...

Not yet.
Might be that the issues I mentioned in the CMS thread are caused by the same reason.

My suggestion would be not to upgrade to 5.x ATM.

I'm trying to install a newer version (4.17) of Squid from source but ./configure has many different options and I'm not sure which to use (except 2). Which should be used for a 1 to 2 PC home network to run LHC? I've previously, on a different system, successfully installed an older version that's available prepackaged and used the configuration file with appropriate modifications found on the forum.

The preferred version should be the most recent squid package from your Linux distribution repository.
Version >=3.5.27 and <5.x

The command "squid -v" prints the active options.
You may also check the keywords in squid.conf against the manual.
If a keyword requires a build option to be set, it is mentioned there.
Example:
http://www.squid-cache.org/Versions/v4/cfgman/follow_x_forwarded_for.html
To use "follow_x_forwarded_for" "--enable-follow-x-forwarded-for" must be configured.


Here is my list (v4.16), but it may slightly differ from what you require, e.g. CFLAGS.

--host=x86_64-suse-linux-gnu
--build=x86_64-suse-linux-gnu
--program-prefix=
--prefix=/usr
--exec-prefix=/usr
--bindir=/usr/bin
--sbindir=/usr/sbin
--sysconfdir=/etc
--datadir=/usr/share
--includedir=/usr/include
--libdir=/usr/lib64
--libexecdir=/usr/libexec
--localstatedir=/var
--sharedstatedir=/var/lib
--mandir=/usr/share/man
--infodir=/usr/share/info
--disable-dependency-tracking
--disable-strict-error-checking
--sysconfdir=/etc/squid
--libexecdir=/usr/libexec/squid
--datadir=/usr/share/squid
--sharedstatedir=/var/squid
--with-logdir=/var/log/squid
--with-pidfile=/run/squid.pid
--with-dl
--enable-disk-io
--enable-storeio
--enable-removal-policies=heap,lru
--enable-icmp
--enable-delay-pools
--enable-esi
--enable-icap-client
--enable-useragent-log
--enable-referer-log
--enable-kill-parent-hack
--enable-arp-acl
--enable-ssl-crtd
--with-openssl
--enable-forw-via-db
--enable-cache-digests
--enable-linux-netfilter
--with-large-files
--enable-underscores
--enable-auth
--enable-auth-basic=SMB_LM,DB,fake,getpwnam,LDAP,NCSA,NIS,PAM,POP3,RADIUS,SASL,SMB
--enable-auth-ntlm=SMB_LM,fake
--enable-auth-negotiate
--enable-auth-digest
--enable-external-acl-helpers=LDAP_group,eDirectory_userip,file_userip,kerberos_ldap_group,session,unix_group,wbinfo_group,time_quota
--enable-stacktraces
--enable-x-accelerator-vary
--with-default-user=squid
--disable-ident-lookups
--enable-follow-x-forwarded-for
--disable-arch-native
--enable-security-cert-generators
--enable-security-cert-validators
build_alias=x86_64-suse-linux-gnu
host_alias=x86_64-suse-linux-gnu
CFLAGS=-O2 -Wall -D_FORTIFY_SOURCE=2 -fstack-protector-strong -funwind-tables -fasynchronous-unwind-tables -fstack-clash-protection -Werror=return-type -g -fPIE -fPIC -DOPENSSL_LOAD_CONF
LDFLAGS=-Wl,--as-needed -Wl,--no-undefined -Wl,-z,relro,-z,now -pie
CXXFLAGS=-O2 -Wall -D_FORTIFY_SOURCE=2 -fstack-protector-strong -funwind-tables -fasynchronous-unwind-tables -fstack-clash-protection -Werror=return-type -g -fPIE -fPIC -DOPENSSL_LOAD_CONF
PKG_CONFIG_PATH=:/usr/lib64/pkgconfig:/usr/share/pkgconfig

Have installed a CentOS8 with a including Squid-Program.
This is running the Proxy inside in Cache.
Working since two days.

https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/networking_guide/configuring-the-squid-caching-proxy-server

#
# Recommended minimum configuration:
#

# Example rule allowing access from your local networks.
# Adapt to list your (internal) IP networks from where browsing
# should be allowed
#acl localnet src 0.0.0.1-0.255.255.255 # RFC 1122 "this" network (LAN)
#acl localnet src 10.0.0.0/8 # RFC 1918 local private network (LAN)
#acl localnet src 100.64.0.0/10 # RFC 6598 shared address space (CGN)
#acl localnet src 169.254.0.0/16 # RFC 3927 link-local (directly plugged) machines
#acl localnet src 192.168.0.0/16 # RFC 1918 local private network (LAN)
#acl localnet src fc00::/7 # RFC 4193 local private network range
#acl localnet src fe80::/10 # RFC 4291 link-local (directly plugged) machines

acl SSL_ports port 443
acl Safe_ports port 80 # http
acl Safe_ports port 21 # ftp
acl Safe_ports port 443 # https
acl Safe_ports port 70 # gopher
acl Safe_ports port 210 # wais
acl Safe_ports port 1025-65535 # unregistered ports
acl Safe_ports port 280 # http-mgmt
acl Safe_ports port 488 # gss-http
acl Safe_ports port 591 # filemaker
acl Safe_ports port 777 # multiling http
acl CONNECT method CONNECT

#
# Recommended minimum Access Permission configuration:
#
# Deny requests to certain unsafe ports
http_access deny !Safe_ports

# Deny CONNECT to other than secure SSL ports
http_access deny CONNECT !SSL_ports

# Only allow cachemgr access from localhost
http_access allow localhost manager
http_access deny manager

# We strongly recommend the following be uncommented to protect innocent
# web applications running on the proxy server who think the only
# one who can access services on "localhost" is a local user
http_access deny to_localhost

#
# INSERT YOUR OWN RULE(S) HERE TO ALLOW ACCESS FROM YOUR CLIENTS
#

# worldcommunitygrid
acl wcg_nocache dstdomain .worldcommunitygrid.org
always_direct allow wcg_nocache
cache deny wcg_nocache

# Example rule allowing access from your local networks.
# Adapt localnet in the ACL section to list your (internal) IP networks
# from where browsing should be allowed
http_access allow localnet
http_access allow localhost

# And finally deny all other access to this proxy
http_access deny all

# Squid normally listens to port 3128
http_port xx.xxx.xxx.xx:3128

# Uncomment and adjust the following to add a disk cache directory.
#cache_dir ufs /var/spool/squid 100 16 256

# Leave coredumps in the first cache dir
coredump_dir none
#coredump_dir /var/spool/squid

#
# Add any of your own refresh_pattern entries above these.
#
refresh_pattern ^ftp: 1440 20% 10080
refresh_pattern ^gopher: 1440 0% 1440
refresh_pattern -i (/cgi-bin/|\?) 0 0% 0
refresh_pattern . 0 20% 4320

It's not helpful to simply post an incomplete list of configuration options without comments/questions.
There are important differences between your squid.conf snippet and the squid.conf posted here:
https://lhcathome.cern.ch/lhcathome/forum_thread.php?id=5473#42987

The given link to RedHat also does not include LHC@home specific settings.

It would have been better to ask why distinct options are used this way or that way in the HowTo.

@computezrmle
9. Acknowledgements
Thanks to maeax and Harri Liljeroos for running the Windows test configuration.

CentOS8-VM: 12 MByte in access.log in two hours. Squid IP-Adress is shown in Clearname!

This is a thread for Comments and discussion. We have a lot of User with a good IT-Background to find solutions here for using squid.

The squid.conf from https://lhcathome.cern.ch/lhcathome/forum_thread.php?id=5474&postid=46134 is incomplete, hence wrong!

http_access allow localnet

This line allows "localnet" to use the proxy but "localnet" is not defined before since all your acl lines start with a "#".
The HowTo also shows examples but explains what must be done, your snippet explains nothing.

http_port xx.xxx.xxx.xx:3128

You hide your local IP (from a local network range!) and claim it's due to "security" ("... Squid IP-Adress is shown in Clearname!"), although that IP range is blocked on every network border router.
The routers MUST block those IPs otherwise the internet wouldn't work due to permanent address conflicts.
See:
https://datatracker.ietf.org/doc/html/rfc1918
We had that discussion a couple of times!


On the other hand you open the proxy for protocols like FTP, gopher, ...
Not a single BOINC project transfers data via those protocols, hence they should be closed - for security reasons!

#cache_dir ufs /var/spool/squid 100 16 256

This line is inactive, hence the proxy has no disk cache to store larger files, e.g. CVMFS catalog files.
Even if you activate it - the disk cache would be only 100 MB.
=> far too small since each of those catalog files could be 10-60 MB.
That setup wouldn't be able to cache vdi files - in case you want to download them once and use them for a couple of clients.

refresh_pattern ...

Those lines represent squid's default if no other refresh_pattern is set.
Again: Why ftp, gopher, ...?
We will not see any of them.


The squid.conf from the HowTo defines it this way:

# 1 line is required to avoid the ancient default settings
# be conservative
# don't violate the HTTP standards
refresh_pattern .	0	0%	0

See:
http://www.squid-cache.org/Doc/config/refresh_pattern/


Lots of other settings are missing.
Why didn't you configure them?


What you still didn't answer:
Why did you post a link and an incomplete (at least partly) wrong config without any further comment?

My Router have no OPEN PORTS!
You can Build this CentOS8-VM by your own!
I'm trusting Cern, IBM (including RedHat) and Microsoft (Win10pro,Win11pro and Win-Workstation AND AVM.

computezrmle,
fyi the squid is running in the RAM.(11GByte)

Running a Squid without optimized setup is like driving a Porsche only in the 1st gear.
It works, of course, makes loud noise and you can easily overtake all bobby cars in the neighbourhood.
However, on a highway you would be an obstacle, even for slower cars and trucks.

Who is driving the Bobbycar and who is driving the Porsche? ;-)
Ok, maybe there is something wrong with squid.conf,
have not many experiences with squid definitions.
First CMS is running ftm. Theory and Atlas no problem.

Running a Squid without optimized setup is like driving a Porsche only in the 1st gear.
It works, of course, makes loud noise and you can easily overtake all bobby cars in the neighbourhood.
However, on a highway you would be an obstacle, even for slower cars and trucks.

What's about a script for this RedHat CentOS8 squid with the correct squid.conf?
We define our local IP-Adress for using.
It's only a idea!

Who is driving the Bobbycar and who is driving the Porsche?

The bobby-car-drivers are the volunteers running lots of worker nodes without a local HTTP proxy.
You are a Porsche-1st-gear-highway-driver (as long as you use the squid.conf you posted).

What's about a script ...

What should that script do?
Mainly copy and paste.
No script can guess or decide which of your local IPs should be allowed to send requests to the internet.
No script can guess or decide how much disk space you really want to use for your cache.
No script ...
It's faster and more reliable to directly enter that in (the optimized) squid.conf.

In addition, to edit squid.conf or restart the proxy root (admin) rights are required.
You wouldn't want anybody to send you a script that you don't understand but you run it as root.

If you feel better with a script, write it, test it and present it.

Ok, searching for the old Squid.conf from Windows Test and analyzing it.

refresh_pattern set to 0....... and
dns_nameservers line added with DNS-Adress of the router.
Testing tomorrow.

● squid.service - Squid caching proxy
Loaded: loaded (/usr/lib/systemd/system/squid.service; enabled; vendor prese>
Active: active (running) since Fri 2022-02-04 03:59:40 CET; 11min ago
Docs: man:squid(8)
Process: 1767 ExecReload=/usr/bin/kill -HUP $MAINPID (code=exited, status=0/S>
Process: 74237 ExecStartPre=/usr/libexec/squid/cache_swap.sh (code=exited, st>
Main PID: 74242 (squid)
Tasks: 3 (limit: 67188)
Memory: 53.8M
CGroup: /system.slice/squid.service
├─74242 /usr/sbin/squid --foreground -f /etc/squid/squid.conf
├─74245 (squid-1) --kid squid-1 --foreground -f /etc/squid/squid.conf
└─74246 (logfile-daemon) /var/log/squid/access.log

Feb 04 03:59:40 RYZEN9COS8SQ systemd[1]: Starting Squid caching proxy...
Feb 04 03:59:40 RYZEN9COS8SQ squid[74242]: Squid Parent: will start 1 kids
Feb 04 03:59:40 RYZEN9COS8SQ squid[74242]: Squid Parent: (squid-1) process 7424>
Feb 04 03:59:40 RYZEN9COS8SQ systemd[1]: Started Squid caching proxy.