Message boards : Number crunching : Setting up a local Squid to work with LHC@home - Comments and Questions
Message board moderation

To post messages, you must log in.

Previous · 1 . . . 5 · 6 · 7 · 8

AuthorMessage
kotenok2000
Avatar

Send message
Joined: 21 Feb 11
Posts: 86
Credit: 586,107
RAC: 1
Message 48453 - Posted: 16 Aug 2023, 16:52:29 UTC

Append CVMFS_QUOTA_LIMIT=10000 to /etc/cvmfs/default.local to make local cache 10000 mb. You can even set it to 100000.
ID: 48453 · Report as offensive     Reply Quote
Profilezepingouin
Avatar

Send message
Joined: 7 Jan 07
Posts: 41
Credit: 16,123,537
RAC: 22
Message 48454 - Posted: 17 Aug 2023, 5:28:09 UTC - in response to Message 48365.  

Is there an easier way to do this? I'm surprised I can't find a simple windows program to cache internet access from other computers on the local network.

I've tried squid twice. The first time was successfull for a time. I followed the instructions in here and it worked for a couple of months, then decided not to. I tried repairing it which failed, so I tried uninstalling it, and it refused. I deleted the squid folder and tried to reinstall and it still didn't work.

So I tried a second time today on a different computer which has never seen squid, I got stuck at the first instruction: "Open "Squid Terminal" as Administrator and run: squid -k shutdown"
I did this and it said "0 [main] squid 228 cygwin_exception::open_stackdumpfile: Dumping stack trace to squid.exe.stackdump"

Without following any further instructions, squid is not functioning out of the box for a browser on another computer directed to it as I would expect.

Is there an alternative cache which isn't a badly ported linux program?


I compiled Squid for use with Windows 10 and Cygwin, you may find binaries and instruction on this thread.
Latest is Squid-6.2 and works fine.
ID: 48454 · Report as offensive     Reply Quote
computezrmle
Volunteer moderator
Volunteer developer
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 15 Jun 08
Posts: 2755
Credit: 304,271,457
RAC: 116,232
Message 48461 - Posted: 19 Aug 2023, 7:28:44 UTC - in response to Message 48453.  

CVMFS_QUOTA_LIMIT is a config option for CVMFS.
It has nothing to do with Squid.


You can even set it to 100000

100 GB - far too much for the LHC@home use case.
Of course you can, but this would just waste disk space.
A quota between 4-10 GB results in a hitrate between 95-99 %.

Let the CVMFS client run a couple of days (may be a week).
Then check the hitrate of heavily used repos like cernvm-prod.cern.ch, sft.cern.ch (both Theory) or atlas.cern.ch (ATLAS - once it is back)
Run "cvmfs_config stat [repo_name]" to get some stats including the hitrate.
ID: 48461 · Report as offensive     Reply Quote
Toby Broom
Volunteer moderator

Send message
Joined: 27 Sep 08
Posts: 946
Credit: 784,373,554
RAC: 158,973
Message 48629 - Posted: 23 Sep 2023, 1:18:39 UTC

ATLAS seem to have stopped uplaoding, is there something that I can check in the config files?
ID: 48629 · Report as offensive     Reply Quote
Mr P Hucker
Avatar

Send message
Joined: 12 Aug 06
Posts: 470
Credit: 15,295,644
RAC: 9,181
Message 48630 - Posted: 23 Sep 2023, 1:55:04 UTC - in response to Message 48629.  

ATLAS seem to have stopped uplaoding, is there something that I can check in the config files?
I uploaded one 10 minutes after your post.
ID: 48630 · Report as offensive     Reply Quote
Toby Broom
Volunteer moderator

Send message
Joined: 27 Sep 08
Posts: 946
Credit: 784,373,554
RAC: 158,973
Message 48631 - Posted: 23 Sep 2023, 1:57:12 UTC

I found it the config was needed to be updated

https://lhcathome.cern.ch/lhcathome/forum_thread.php?id=5249
ID: 48631 · Report as offensive     Reply Quote
kotenok2000
Avatar

Send message
Joined: 21 Feb 11
Posts: 86
Credit: 586,107
RAC: 1
Message 48632 - Posted: 23 Sep 2023, 1:59:12 UTC
Last modified: 23 Sep 2023, 2:22:37 UTC

Someone needs to create a step by step guide for installing and configuring cvmfs, boinc and apptainer, without building from source.
ID: 48632 · Report as offensive     Reply Quote
Mr P Hucker
Avatar

Send message
Joined: 12 Aug 06
Posts: 470
Credit: 15,295,644
RAC: 9,181
Message 48635 - Posted: 23 Sep 2023, 3:35:25 UTC - in response to Message 48632.  

Someone needs to create a step by step guide for installing and configuring cvmfs, boinc and apptainer, without building from source.
I keep saying this, but they don't like it. Apparently it's already simple?
ID: 48635 · Report as offensive     Reply Quote
computezrmle
Volunteer moderator
Volunteer developer
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 15 Jun 08
Posts: 2755
Credit: 304,271,457
RAC: 116,232
Message 48639 - Posted: 23 Sep 2023, 7:40:06 UTC - in response to Message 48632.  

In most cases there's no need to compile any of them from the sources.
Recent Linux distributions provide BOINC, VirtualBox, Runc, Apptainer, Squid.

CERN provides precompiled CVMFS packets as described here:
https://lhcathome.cern.ch/lhcathome/forum_thread.php?id=5594&postid=48539

Most packets just need to be installed, except CVMFS and Squid which need to be configured as described here:
https://lhcathome.cern.ch/lhcathome/forum_thread.php?id=5594&postid=48539
https://lhcathome.cern.ch/lhcathome/forum_thread.php?id=5473


If this is too complex or the OS can't run native apps (Windows) install VirtualBox and let BOINC run vbox apps.
ID: 48639 · Report as offensive     Reply Quote
kotenok2000
Avatar

Send message
Joined: 21 Feb 11
Posts: 86
Credit: 586,107
RAC: 1
Message 48641 - Posted: 23 Sep 2023, 11:37:59 UTC

Then someone needs to unpin this thread
https://lhcathome.cern.ch/lhcathome/forum_thread.php?id=4840
ID: 48641 · Report as offensive     Reply Quote
kotenok2000
Avatar

Send message
Joined: 21 Feb 11
Posts: 86
Credit: 586,107
RAC: 1
Message 48642 - Posted: 23 Sep 2023, 12:17:37 UTC - in response to Message 48639.  

You should also add instructions for disabling cgroups v2 so Theory tasks don't crash.
ID: 48642 · Report as offensive     Reply Quote
Raythean Buggy Cam
Avatar

Send message
Joined: 30 Jan 22
Posts: 1
Credit: 190,659
RAC: 4,254
Message 53382 - Posted: 8 Apr 2026, 20:18:29 UTC

Where’s the best place to install the Squid application when running Windows 11 with the BOINC Docker app? Right now, it’s installed on my Windows system, but I’m getting a lot of TCP_MISS messages.

1775632134.196  63742 127.0.0.1 TCP_MISS/200 327 POST http://lhcathome-upload.cern.ch/lhcathome_cgi/file_upload_handler - HIER_DIRECT/188.184.89.143 text/plain
1775632199.383  63929 127.0.0.1 TCP_MISS/200 298 POST http://lhcathome-upload.cern.ch/lhcathome_cgi/file_upload_handler - HIER_DIRECT/188.185.81.88 text/plain
1775632271.721  71270 127.0.0.1 TCP_TUNNEL/200 14010 CONNECT lhcathome.cern.ch:443 - HIER_DIRECT/188.184.93.91 -

ID: 53382 · Report as offensive     Reply Quote
Mr P Hucker
Avatar

Send message
Joined: 12 Aug 06
Posts: 470
Credit: 15,295,644
RAC: 9,181
Message 53383 - Posted: 8 Apr 2026, 20:24:54 UTC

I spotted they miss on upload, that is to be expected. It can't remember the result you created, because it's new and unique.

Gemini explains the rest better than me!

Ideally, you should run Squid as a **Docker container** on the same network as your BOINC Docker app, rather than as a standalone Windows service.
While having it on the Windows host works, running it in Docker allows for better networking performance via the **WSL 2 backend** and makes it easier to manage the internal routing between the BOINC client and the proxy.
### Why you are seeing TCP_MISS
The TCP_MISS/200 entries in your log are actually expected for the specific traffic you're showing.
1. **POST Requests:** Your logs show POST requests to a file_upload_handler. By default, Squid (and most proxies) **does not cache POST requests** because they are intended to send data to the server, not retrieve it.
2. **HTTPS Tunneling:** The TCP_TUNNEL/200 for port 443 indicates an encrypted connection. Squid cannot "see" or cache the content of encrypted traffic unless you set up **SSL Bump** (SSL MITM), which is complex and usually unnecessary for BOINC.
### Recommended Setup: Squid in Docker
To get the most out of your setup on Windows 11, follow these steps to move Squid into the Docker environment:
#### 1. Create a Docker Network
This allows your BOINC container and Squid container to communicate using internal container names.
```bash
docker network create boinc-net

```
#### 2. Run Squid in a Container
You can use a lightweight image like ubuntu/squid. Ensure you mount a volume for the cache so it persists when the container restarts.
```bash
docker run -d --name squid-proxy \
--network boinc-net \
-v squid-cache:/var/spool/squid \
-p 3128:3128 \
ubuntu/squid

```
#### 3. Optimise squid.conf for BOINC
If you want to cache the large **download** files (the actual work units/VM images) from CERN, you need to adjust the maximum object size in your squid.conf, as BOINC files are often larger than the default 4MB limit:
```bash
# Allow larger files to be cached (e.g., 500MB)
maximum_object_size 500 MB
cache_dir ufs /var/spool/squid 5000 16 256

```
### Tips for LHC@home
* **Ignore Upload "Misses":** Don't worry about TCP_MISS on POST requests to file_upload_handler. These are your finished results leaving your system; there is no benefit to caching them.
* **Focus on Downloads:** The real benefit of Squid for BOINC is caching the .vdi or .iso files when you have multiple tasks or machines. You will see TCP_HIT when the client requests a file it has downloaded previously.
* **Client Configuration:** In the BOINC Manager, ensure your proxy settings point to the Docker Desktop gateway (usually 172.17.0.1 or the specific container name if using a custom network).
Since you are using a **Google Pixel 6 Pro**, you can actually monitor your BOINC status and these logs on the go using the **BOINC Monitor** app or similar third-party tools, provided you've opened the remote desktop ports in your Windows firewall.
ID: 53383 · Report as offensive     Reply Quote
computezrmle
Volunteer moderator
Volunteer developer
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 15 Jun 08
Posts: 2755
Credit: 304,271,457
RAC: 116,232
Message 53384 - Posted: 8 Apr 2026, 20:46:31 UTC - in response to Message 53382.  

In reply to Raythean Buggy Cam's message of 8 Apr 2026:
Where’s the best place to install the Squid application when running Windows 11 with the BOINC Docker app? Right now, it’s installed on my Windows system, but I’m getting a lot of TCP_MISS messages.

1775632134.196  63742 127.0.0.1 TCP_MISS/200 327 POST http://lhcathome-upload.cern.ch/lhcathome_cgi/file_upload_handler - HIER_DIRECT/188.184.89.143 text/plain
1775632199.383  63929 127.0.0.1 TCP_MISS/200 298 POST http://lhcathome-upload.cern.ch/lhcathome_cgi/file_upload_handler - HIER_DIRECT/188.185.81.88 text/plain
1775632271.721  71270 127.0.0.1 TCP_TUNNEL/200 14010 CONNECT lhcathome.cern.ch:443 - HIER_DIRECT/188.184.93.91 -

TCP_MISS means that the data from the given URL is not in your local cache.
In case of a POST (=upload) you will always get a TCP_MISS since it makes no sense to cache uploads.
ID: 53384 · Report as offensive     Reply Quote
Previous · 1 . . . 5 · 6 · 7 · 8

Message boards : Number crunching : Setting up a local Squid to work with LHC@home - Comments and Questions


©2026 CERN