LHC@home

It keeps most objects used by a series of tasks close to the computer(s) where they are used.
Hence, it helps both, the server side as well as the client side.

See:
https://lhcathome.cern.ch/lhcathome/forum_thread.php?id=5473

To get an impression how many requests are served by just 1 (out of hundreds) of Squids at CERN/WLCG within 1 day, see:

http://wlcg-squid-monitor.cern.ch/awstats/bin/awstats.pl?config=atlasfrontier.cern.ch&databasebreak=day&day=01

And many of the clients getting objects from that Squid are also Squids sending requests to their parent only if an object is not (or not fresh) in their local cache.

Okay thanks but is it a cache for everyone or is it only local for my computers?

The configuration suggested in this forum is intended to be used only for your local network.
Be aware that the example squid.conf does not allow any client to access the internet.
You will have to add your computer(s)/network first to grant permission.

Hi,

After installing squid with apt (ubuntu server 22.04), running 'squid -z' and starting squid,

'sudo systemctl status squid.service' gives:
...
Active: active (running) since Wed 2023-05-24 17:24:54 UTC; 9s ago
...
ERROR: listen(..., 1024) system call failed: (98) Address already in use
listening port: 3128

I guess this is wrong. What should I do?

Kind regards,
Nejc


my /etc/squid/squid.conf is:
acl crunchers src 192.168.0.145
acl localnet src 0.0.0.1-0.255.255.255 # RFC 1122 "this" network (LAN)
acl localnet src 10.0.0.0/8 # RFC 1918 local private network (LAN)
acl localnet src 100.64.0.0/10 # RFC 6598 shared address space (CGN)
acl localnet src 169.254.0.0/16 # RFC 3927 link-local (directly plugged) machines
acl localnet src 172.16.0.0/12 # RFC 1918 local private network (LAN)
acl localnet src 192.168.0.0/16 # RFC 1918 local private network (LAN)
acl localnet src fc00::/7 # RFC 4193 local private network range
acl localnet src fe80::/10 # RFC 4291 link-local (directly plugged) machines
acl SSL_ports port 443
acl Safe_ports port 80 # http
acl Safe_ports port 21 # ftp
acl Safe_ports port 443 # https
acl Safe_ports port 70 # gopher
acl Safe_ports port 210 # wais
acl Safe_ports port 1025-65535 # unregistered ports
acl Safe_ports port 280 # http-mgmt
acl Safe_ports port 488 # gss-http
acl Safe_ports port 591 # filemaker
acl Safe_ports port 777 # multiling http
acl CONNECT method CONNECT

follow_x_forwarded_for allow localhost
follow_x_forwarded_for deny all

acl wcg_nocache dstdomain .worldcommunitygrid.org
cache deny wcg_nocache
acl cvmfs_geoapi urlpath_regex -i ^/+cvmfs/+[0-9a-z._~-]+/+api/+[0-9a-z._~-]+/+geo/+[0-9a-z._~-]+/+[0-9a-z.,_~-]+
cache deny cvmfs_geoapi
acl boinc_nocache urlpath_regex -i /download[0-9a-z._~-]*/+[0-9a-z._~-]+/+.+
cache deny boinc_nocache
acl PragmaNoCache req_header Pragma no-cache
cache deny PragmaNoCache
acl Purge method PURGE

http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access allow localhost manager
http_access deny manager
http_access deny to_localhost
http_access allow crunchers
http_access allow localhost
http_access deny all
http_port 3128

dns_nameservers 192.168.0.1
max_filedescriptors 4096
client_dst_passthru off
cache_mem 256 MB
maximum_object_size_in_memory 24 KB
memory_replacement_policy heap GDSF
cache_replacement_policy heap LFUDA
maximum_object_size 6144 MB
cache_dir aufs /var/cache/squid 20000 16 64 min-size=7937
logfile_rotate 10
logformat my_awstats %>A %lp %ui %un [%tl] "%rm %>ru HTTP/%rv" %>Hs %st "%{Referer}>h" "%{User-Agent}>h" %Ss:%Sh
access_log stdio:/var/log/squid/access.log logformat=my_awstats
strip_query_terms off
coredump_dir none
ftp_user anonymous@
max_stale 37 days
refresh_pattern . 0 0% 0
store_avg_object_size 1800 KB
shutdown_lifetime 0 seconds
collapsed_forwarding on
client_persistent_connections on
server_persistent_connections on
log_icp_queries off
dns_defnames on
forwarded_for transparent

I forgot to mention that squid -k reconfigure gave no errors.

According to this lines your IPs are taken from the private ipv4 range:
acl crunchers src 192.168.0.145
dns_nameservers 192.168.0.1


To check whether the networks in squid.conf match your local settings post the output of this command:
ip -f inet addr show


Simplify your squid.conf.
If you use 192.168.0.x there's no need to enable other networks in squid.conf.
Disable them (red #):
acl crunchers src 192.168.0.145
#acl localnet src 0.0.0.1-0.255.255.255 # RFC 1122 "this" network (LAN)
#acl localnet src 10.0.0.0/8 # RFC 1918 local private network (LAN)
#acl localnet src 100.64.0.0/10 # RFC 6598 shared address space (CGN)
#acl localnet src 169.254.0.0/16 # RFC 3927 link-local (directly plugged) machines
#acl localnet src 172.16.0.0/12 # RFC 1918 local private network (LAN)
#acl localnet src 192.168.0.0/16 # RFC 1918 local private network (LAN)
#acl localnet src fc00::/7 # RFC 4193 local private network range
#acl localnet src fe80::/10 # RFC 4291 link-local (directly plugged) machines





Check for errors here

After each modification of squid.conf run:
squid -k parse


After a Squid reload/restart check /var/log/squid/cache.log


As for this error post the output of:
ps -AF |grep " $(lsof -ti:3128) "

ERROR: listen(..., 1024) system call failed: (98) Address already in use

I commented the mentioned lines.
I didn't mention it before - I installed squid on the same machine that is crunching - is this a problem? Would it be better if I make a VM on one of my other servers and use that one for squid?

Since I installed and configured squid none of my project can connect (Can't create HTTP response output file notices/ wcg, rosetta, sidock, gpugrid) and the other two projects (lhc and einstein) are not even visible in the boinc-client/boinctui.
Before that everything was running normal - cvmfs was installed per guide that makes it possible to run cms and other tasks natively.

So, after commenting the lines you mentioned:
squid -k parse:
2023/05/25 12:18:40| Startup: Initializing Authentication Schemes ...
2023/05/25 12:18:40| Startup: Initialized Authentication Scheme 'basic'
2023/05/25 12:18:40| Startup: Initialized Authentication Scheme 'digest'
2023/05/25 12:18:40| Startup: Initialized Authentication Scheme 'negotiate'
2023/05/25 12:18:40| Startup: Initialized Authentication Scheme 'ntlm'
2023/05/25 12:18:40| Startup: Initialized Authentication.
2023/05/25 12:18:40| Processing Configuration File: /etc/squid/squid.conf (depth 0)
2023/05/25 12:18:40| Processing: acl crunchers src 192.168.0.145
2023/05/25 12:18:40| Processing: acl SSL_ports port 443
2023/05/25 12:18:40| Processing: acl Safe_ports port 80 # http
2023/05/25 12:18:40| Processing: acl Safe_ports port 21 # ftp
2023/05/25 12:18:40| Processing: acl Safe_ports port 443 # https
2023/05/25 12:18:40| Processing: acl Safe_ports port 70 # gopher
2023/05/25 12:18:40| Processing: acl Safe_ports port 210 # wais
2023/05/25 12:18:40| Processing: acl Safe_ports port 1025-65535 # unregistered ports
2023/05/25 12:18:40| Processing: acl Safe_ports port 280 # http-mgmt
2023/05/25 12:18:40| Processing: acl Safe_ports port 488 # gss-http
2023/05/25 12:18:40| Processing: acl Safe_ports port 591 # filemaker
2023/05/25 12:18:40| Processing: acl Safe_ports port 777 # multiling http
2023/05/25 12:18:40| Processing: acl CONNECT method CONNECT
2023/05/25 12:18:40| Processing: follow_x_forwarded_for allow localhost
2023/05/25 12:18:40| Processing: follow_x_forwarded_for deny all
2023/05/25 12:18:40| Processing: acl wcg_nocache dstdomain .worldcommunitygrid.org
2023/05/25 12:18:40| Processing: cache deny wcg_nocache
2023/05/25 12:18:40| Processing: acl cvmfs_geoapi urlpath_regex -i ^/+cvmfs/+[0-9a-z._~-]+/+api/+[0-9a-z._~-]+/+geo/+[0-9a-z._~-]+/+[0-9a-z.,_~-]+
2023/05/25 12:18:40| Processing: cache deny cvmfs_geoapi
2023/05/25 12:18:40| Processing: acl boinc_nocache urlpath_regex -i /download[0-9a-z._~-]*/+[0-9a-z._~-]+/+.+
2023/05/25 12:18:40| Processing: cache deny boinc_nocache
2023/05/25 12:18:40| Processing: acl PragmaNoCache req_header Pragma no-cache
2023/05/25 12:18:40| Processing: cache deny PragmaNoCache
2023/05/25 12:18:40| Processing: acl Purge method PURGE
2023/05/25 12:18:40| Processing: http_access deny !Safe_ports
2023/05/25 12:18:40| Processing: http_access deny CONNECT !SSL_ports
2023/05/25 12:18:40| Processing: http_access allow localhost manager
2023/05/25 12:18:40| Processing: http_access deny manager
2023/05/25 12:18:40| Processing: http_access deny to_localhost
2023/05/25 12:18:40| Processing: include /etc/squid/conf.d/*.conf
2023/05/25 12:18:40| Processing Configuration File: /etc/squid/conf.d/debian.conf (depth 1)
2023/05/25 12:18:40| Processing: logfile_rotate 0
2023/05/25 12:18:40| Processing: http_access allow crunchers
2023/05/25 12:18:40| Processing: http_access allow localhost
2023/05/25 12:18:40| Processing: http_access deny all
2023/05/25 12:18:40| Processing: http_port localhost:3128
2023/05/25 12:18:40| Processing: http_port 3128
2023/05/25 12:18:40| Processing: dns_nameservers 192.168.0.1
2023/05/25 12:18:40| Processing: max_filedescriptors 4096
2023/05/25 12:18:40| Processing: client_dst_passthru off
2023/05/25 12:18:40| Processing: cache_mem 256 MB
2023/05/25 12:18:40| Processing: maximum_object_size_in_memory 24 KB
2023/05/25 12:18:40| Processing: memory_replacement_policy heap GDSF
2023/05/25 12:18:40| Processing: cache_replacement_policy heap LFUDA
2023/05/25 12:18:40| Processing: maximum_object_size 6144 MB
2023/05/25 12:18:40| Processing: cache_dir aufs /var/cache/squid 20000 16 64 min-size=7937
2023/05/25 12:18:40| Processing: logfile_rotate 10
2023/05/25 12:18:40| Processing: logformat my_awstats %>A %lp %ui %un [%tl] "%rm %>ru HTTP/%rv" %>Hs %st "%{Referer}>h" "%{User-Agent}>h" %Ss:%Sh
2023/05/25 12:18:40| Processing: access_log stdio:/var/log/squid/access.log logformat=my_awstats
2023/05/25 12:18:40| Processing: strip_query_terms off
2023/05/25 12:18:40| Processing: coredump_dir none
2023/05/25 12:18:40| Processing: ftp_user anonymous@
2023/05/25 12:18:40| Processing: max_stale 37 days
2023/05/25 12:18:40| Processing: refresh_pattern . 0 0% 0
2023/05/25 12:18:40| Processing: store_avg_object_size 1800 KB
2023/05/25 12:18:40| Processing: shutdown_lifetime 0 seconds
2023/05/25 12:18:40| Processing: collapsed_forwarding on
2023/05/25 12:18:40| Processing: client_persistent_connections on
2023/05/25 12:18:40| Processing: server_persistent_connections on
2023/05/25 12:18:40| Processing: log_icp_queries off
2023/05/25 12:18:40| Processing: dns_defnames on
2023/05/25 12:18:40| Processing: forwarded_for transparent
2023/05/25 12:18:40| Initializing https:// proxy context

ip -f inet addr show:
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
5: bond0: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
inet 192.168.0.145/24 metric 100 brd 192.168.0.255 scope global dynamic bond0
valid_lft 496sec preferred_lft 496sec

ps -AF |grep " $(lsof -ti:3128) " - is REALLy long - should I post the whole of it here?

Thank you for helping.

Since you run a couple of computers, is it this one?
https://lhcathome.cern.ch/lhcathome/show_host_detail.php?hostid=10830136
or this one?
https://lhcathome.cern.ch/lhcathome/show_host_detail.php?hostid=10821458

I installed squid on the same machine that is crunching - is this a problem?

No

Would it be better if I make a VM on one of my other servers and use that one for squid?

No.
You may run Squid on a VM for testing (until you are familiar with the process) but at the end it should run on a real box for better performance.

Since I installed and configured squid none of my project can connect (Can't create HTTP response output file notices/ wcg, rosetta, sidock, gpugrid) and the other two projects (lhc and einstein) are not even visible in the boinc-client/boinctui.

This doesn't make sense.
If you just install and start Squid it binds to port 3128.
Up to this point Squid does not modify anything in connection with BOINC.
The HowTo clearly states to test Squid first, e.g. using a browser.
If this succeeds BOINC can be told to run that Squid but even this setting does not modify BOINC's project list or the way it generates requests.
BOINC just sends them to Squid instead of the original destination.

cvmfs was installed per guide that makes it possible to run cms...

CMS doesn't have a native app that would use a local CVMFS.

5: bond0 ...

Looks like you have more than the 2 network interfaces that those you posted.
May be one of them is not configured correctly.

To ensure Squid binds only to 127.0.0.1 and 192.168.0.145 replace
http_port localhost:3128
http_port 3128

with
http_port localhost:3128
http_port 192.168.0.145:3128
tcp_outgoing_address 192.168.0.145

ps -AF |grep " $(lsof -ti:3128) " - is REALLy long - should I post the whole of it here?

No.
A long list points out "(lsof ...)" was empty.
This happens if Squid is not running.
Instead, start your Squid service and check /var/log/squid/cache.log for recent errors.

So on the Can't create HTTP response output file notices... as expected - mea culpa. I ran boinc as service not daemon (got mixed up on which machine I am working :) ).

Will start with squid from the begining slowly and taking into account all that you wrote.

Hopefully I willl post back with success.

Great thanks computezrmle

Have running Squid in a CentOS9-VM.
You don't need a separate Hardware for use it.

After correcting squid.conf as mentioned systemctl status gives no errors.
Does it mean that squid is running correctly? How can I check (no GUI)?

If i now do: boinccmd --set_proxy_settings squid_hostname_or_IP 3128 '' '' '' '' '' '' '' (as metnioned at top of the thread) will all my projects be connecting through proxy correctly or do I need to add some more stuff to squid.conf?

After correcting squid.conf as mentioned systemctl status gives no errors.
Does it mean that squid is running correctly? How can I check (no GUI)?

If i now do: boinccmd --set_proxy_settings squid_hostname_or_IP 3128 '' '' '' '' '' '' '' (as metnioned at top of the thread) will all my projects be connecting through proxy correctly or do I need to add some more stuff to squid.conf?

If "systemctl status ..." doesn't print an error this does not mean your clients are permitted to use Squid.
Either test this with a browser on a computer you want to connect via Squid or configure BOINC to use Squid.
Worst case would be that the computer is not (yet) permitted and BOINC fails to connect to the internet.
Then, add a line like this to squid.conf for each IPv4 you want to allow to use Squid.
Example:
acl crunchers src 192.168.0.146
acl crunchers src 192.168.0.147
acl crunchers src 192.168.0.148

Then run (as root) "squid -k reconfigure"


If a browser on a computer in your LAN can contact internet sites, BOINC on the same computer will also be permitted.
In addition you will get log entries from that computer in /var/log/squid/access.log
To follow that file you may try this command:
tail -F /var/log/squid/access.log


Be aware that your access.log may grow very quickly since you have a large number of worker nodes.
Hence, the logfile should be rotated regularly (suggestion: once a day) either by a cronjob or via systemd running "squid -k rotate".
Make yourself familiar with those basic methods since they are often required - not only by Squid.



WCG is (better: was) known to be problematic in connection with the local cache, hence this is switched off in squid.conf and only Squid's proxy function is used.
Other projects should work fine.
If Squid related problems happen this needs to be investigated.

Thank you for all the help. I believe it is working now.

So how can I actually see if the traffic of boinc or IP address in squid?
And how can I check what squid is doing "for my clients;from it's RAM-Cache" as Yeti mentioned?

So how can I actually see if the traffic of boinc or IP address in squid?
And how can I check what squid is doing "for my clients;from it's RAM-Cache" as Yeti mentioned?

As already mentioned, run this (as root) on the squid box:
tail -F /var/log/squid/access.log


Or try this commands (also on the squid box):
squidclient mgr:info
squidclient mgr:refresh
squidclient mgr:client_list

Is there an easier way to do this? I'm surprised I can't find a simple windows program to cache internet access from other computers on the local network.

I've tried squid twice. The first time was successfull for a time. I followed the instructions in here and it worked for a couple of months, then decided not to. I tried repairing it which failed, so I tried uninstalling it, and it refused. I deleted the squid folder and tried to reinstall and it still didn't work.

So I tried a second time today on a different computer which has never seen squid, I got stuck at the first instruction: "Open "Squid Terminal" as Administrator and run: squid -k shutdown"
I did this and it said "0 [main] squid 228 cygwin_exception::open_stackdumpfile: Dumping stack trace to squid.exe.stackdump"

Without following any further instructions, squid is not functioning out of the box for a browser on another computer directed to it as I would expect.

Is there an alternative cache which isn't a badly ported linux program?

I'm not entirely sure why we have to do this anyway, at least within one computer. On Windows anyway, LHC runs every single task in it's own virtualbox. And those VBs don't seem to have access outside themselves, to the local disk in Windows. If they did, anything they download could be kept for another task running on the same computer. As it is, one computer is downloading the same files multiple times. Is this some kind of Boinc limitation?

I just tried putting LHC onto a Linux machine without VB installed, and it's not working, I assumed LHC didn't use VB if in Linux already? Am I wrong? They're coming up as "native theory".

Have only one PC (64-Core) running with a CentOS9-VM for Squid.
The Transfer of Data is 2 TByte inside the PC for Theory-Tasks in a few days.

Have only one PC (64-Core) running with a CentOS9-VM for Squid.
The Transfer of Data is 2 TByte inside the PC for Theory-Tasks in a few days.

There's something terribly wrong with the design of the Boinc/LHC programs if it's downloading the same file more than once on the same PC. It should be stored in the Boinc folders for later use.

The design is fine.
It's you who refuse to understand it.

The design is fine.
It's you who refuse to understand it.

How is it fine to download huge data file x 24 times for 24 tasks running on the same PC?