I see that the squid proxy does not work to localhost

stderr from container:
Environment proxy 'http://192.168.1.179:3128' set but can't be connected
Could not find a local HTTP proxy

CMS and other apps can use this path.

To me it's not clear what exactly does not work.

The local CVMFS doesn't use the Squid?
Then check your CVMFS configuration.

The Theory docker app doesn't use the Squid?
Then ensure the proxy is set in the [engine] section of containers.conf, like:

[engine]
env = [
     "http_proxy=http://192.168.1.179:3128",
   ]

Anything else?

It seems like the docker is not using Squid.

I have in my /etc/cvmfs/default.local

CVMFS_HTTP_PROXY="http://192.168.1.179:3128;DIRECT"

I put the proxy info in [containers] and its working fine on all other computers, I try moving to [engine] amd see if the error goes.

It only only the computer that has squid that I see an issue.

Do you run a CVMFS and a Squid on the same computer?
Is this the computer with IP 192.168.1.179?
Can you confirm that CVMFS uses the Squid?

If so, Squid is correctly configured.
=> You need to look into the container config.

In Copilot Troubleshooting checklist:
double {{ and NOT one { string.
Is this the correct Reason for podman defining ?

Yes, I have CVMFS and squid on the same computer 192.168.1.179.

I don't think CVMFS is using the proxy either:

Environment proxy 'http://192.168.1.179:3128' set but can't be connected
Could not find a local HTTP proxy
CVMFS and Frontier will have to use DIRECT connections

In reply to Toby Broom's message of 24 Apr 2026:

Yes, I have CVMFS and squid on the same computer 192.168.1.179.

I don't think CVMFS is using the proxy either:

Environment proxy 'http://192.168.1.179:3128' set but can't be connected
Could not find a local HTTP proxy
CVMFS and Frontier will have to use DIRECT connections

This message tells you that the proxy setting is passed to the container.
Please post a link to a task log from that computer.

You have to ensure your Squid allows requests from it's own host.
Run some tests and in parallel monitor if Squid's access.log reports corresponding lines with TCP_DENIED.

Check if 192.168.1.179 is included in the output or in a range from the output:
sudo squid -f /your/squid.conf -k parse 2>&1 |grep '192\.168'


Check if CVMFS on the host gets data via Squid.
cvmfs_config stat sft.cern.ch

The output should look like this:
ERSION PID UPTIME(M) MEM(K) REVISION EXPIRES(M) NOCATALOGS CACHEUSE(K) CACHEMAX(K) NOFDUSE NOFDMAX NOIOERR NOOPEN HITRATE(%) RX(K) SPEED(K/S) HOST PROXY ONLINE
2.12.3.3 21333 6544 373436 34975 0 3778 6626347 8388609 287 130560 0 910455 99.915 175515 785 http://s1cern-cvmfs.openhtc.io/cvmfs/sft.cern.ch http://192.168.1.179:3128 1


Ensure the netcat package (optional ncat) is installed to run some basic network test.
Post the output of all tests:
nc -zvw 3 192.168.1.179 3128
your result:
nc -zvw 3 localhost 3128
your result:
nc -zvw 3 -s 127.0.0.1 192.168.1.179 3128
your result:
nc -zvw 3 -s 127.0.0.1 localhost 3128
your result:


If all previous tests didn't report an error, continue with some container tests.
Temporarily set '<dont_check_file_sizes>1</dont_check_file_sizes>' in BOINC's cc_config.xml.
Modify projects/lhcathome.cern.ch_lhcathome/job_2025_04_15.toml like this:
create_args = "--cap-add=SYS_ADMIN --env=http_proxy=http://192.168.1.179:3128 --device /dev/fuse -v /cvmfs:/cvmfs:shared"
create_args = "--cap-add=SYS_ADMIN --network pasta:-4 --device /dev/fuse -v /cvmfs:/cvmfs:shared"
create_args = "--cap-add=SYS_ADMIN --network pasta:-4 --env=http_proxy=http://192.168.1.179:3128 --device /dev/fuse -v /cvmfs:/cvmfs:shared"

For each setting run a Theory task.
Which (if any) container gets data via Squid?

This is the host:

https://lhcathome.cern.ch/lhcathome/results.php?hostid=10840239

acl crunchers src 192.168.1.179 is in the squid config.

no TCP_DENIED in the squid logs

I grepped for the hostname and I have plenty of items like:

Oxygen 3128 - - [24/Apr/2026:21:05:53 +0200] "GET http://s1cern-cvmfs.openhtc.io/cvmfs/cms-ib.cern.ch/.cvmfspublished HTTP/1.1" 200 1963 "-" "cvmfs Fuse 2.7.2 cbeefb35-e2f9-46d3-92c2-980508de09da" TCP_MEM_HIT:HIER_NONE

cvmfs_config stat sft.cern.ch
VERSION PID UPTIME(M) MEM(K) REVISION EXPIRES(M) NOCATALOGS CACHEUSE(K) CACHEMAX(K) NOFDUSE NOFDMAX NOIOERR NOOPEN HITRATE(%) RX(K) SPEED(K/S) HOST PROXY ONLINE
2.13.3.0 20433 1517 190508 34978 3 3259 2543840 4096001 285 130560 0 47870 99.973 27174 1840 http://s1cern-cvmfs.openhtc.io/cvmfs/sft.cern.ch http://192.168.1.179:3128 1

nc -zvw 3 192.168.1.179 3128
Connection to 192.168.1.179 3128 port [tcp/*] succeeded!
nc -zvw 3 localhost 3128
Connection to localhost (127.0.0.1) 3128 port [tcp/*] succeeded!
nc -zvw 3 -s 127.0.0.1 192.168.1.179 3128
Connection to 192.168.1.179 3128 port [tcp/*] succeeded!
nc -zvw 3 -s 127.0.0.1 localhost 3128
Connection to localhost (127.0.0.1) 3128 port [tcp/*] succeeded!

I updated the toml, I didn't comment out the orginal
create_args = "--cap-add=SYS_ADMIN --device /dev/fuse -v /cvmfs:/cvmfs:shared"

So it crashed out with TOML error: Error: line 5: Multiple same key: create_args

Let see how it goes with the 3 lines you gave

Doing some reading it seems like pasta networking does funkyness on localhost.

I tried to do something like:

podman exec -it d8561a1c654e curl --head http://s1cern-cvmfs.openhtc.io/cvmfs/cms-ib.cern.ch/.cvmfspublished

But there was connection refused.

So from the container, the loopback address is host.containers.internal

Setting the containers.conf to host.containers.internal now allows networking between the container and the proxy on localhost.

HTTP/1.1 200 OK
Via: 1.1 front-lcg-pdc-01.cern.ch (squid/frontier-squid-6.14-1.3), 1.1 Oxygen (squid/6.14)

I'm not sure how the CVMFS proxy config is passed to the container?

Seems like it works:

stderr from container:
Got a proxy from the local environment
Will use it for CVMFS and Frontier
VERSION HOST PROXY
2.13.3.0 http://s1cern-cvmfs.openhtc.io http://192.168.1.179:3128
Environment HTTP proxy: http://host.containers.internal:3128

sudo systemctl status squid

squid.service - Squid caching proxy.

Squid is working now inside of the VM!
Thank you Toby.

In reply to Toby Broom's message of 24 Apr 2026:

...
Seems like it works:

stderr from container:
Got a proxy from the local environment
Will use it for CVMFS and Frontier
VERSION HOST PROXY
2.13.3.0 http://s1cern-cvmfs.openhtc.io http://192.168.1.179:3128
Environment HTTP proxy: http://host.containers.internal:3128

That's good news.
We should keep it in mind.


Another good news is that even your older Theory (docker) tasks were using cached data since they reported:
Using CVMFS on the host.
.
.
.
VERSION HOST PROXY
2.13.3.0 http://s1cern-cvmfs.openhtc.io http://192.168.1.179:3128

Your host CVMFS in turn uses the local Squid.
Your tasks for Theory (vbox) and CMS are also using it.

Theory (vbox) looks al little confusing in the logs:

2026-04-21 14:56:35 (583501): Guest Log: Got a proxy from the local BOINC client
2026-04-21 14:56:35 (583501): Guest Log: Will use it for CVMFS and Frontier
.
.
.
2026-04-21 14:56:38 (583501): Guest Log: VERSION HOST PROXY
2026-04-21 14:56:38 (583501): Guest Log: 2.13.1.0 http://s1cern-cvmfs.openhtc.io http://192.168.1.179:3128
2026-04-21 14:56:38 (583501): Guest Log: Environment HTTP proxy: not set

But its the same on all my hosts so I assume its working

It's working fine.

The environment proxy is nice to have for CMS (only) since CMS requests many objects from a distinct CERN webserver (but by far the most from CVMFS!). Theory does not use any data not provided via CVMFS.

Since the scripts were developed for CMS but are now also used for Theory you get that message about the missing environment proxy.
The reason is not the script but a missing tool at Theory (VM as well as podman image).
So far it makes no sense to install that tool on Theory.
Just focus on CVMFS and ignore the message.