Questions and Answers : Unix/Linux : How to setup parrot connector on a LHC@Home client behind firewall
Message board moderation

To post messages, you must log in.

AuthorMessage
Cyanr & Cinny
Avatar

Send message
Joined: 1 Jul 20
Posts: 6
Credit: 18,348,912
RAC: 9,721
Message 49120 - Posted: 2 Jan 2024, 6:41:37 UTC

Hi all experienced Linux admins here and there

I have a Linux PC behind a firewall, which means, it does not allow fuse or autofs to access cvmfs shared dataset. The only possible connecting method outside for Internet is via http(s) proxy.

Is there any know method to configure cvmfs to use parrot connector and then mount these /cvmfs stuffs, and then BOINC client of LHC@home can access them transparently?

I searched the Internet by Google, and found there are some topic talks about parrot connector for certain distributed application (e.g., singularity) of cvmfs usage, but nothing specifically mentioned about the configuration (on setup) of LHC@home (as far as I know, LHC@home has several apps for each research topics and they run natively or on a virtualbox).

I also tried some experiments based on doc at
https://cvmfs.readthedocs.io/en/stable/cpt-configure.html
but nothing is luck enough for me to connect /cvmfs for lhc-researches at cern dot ch.

I even do not know the correct way to configure a typical cvmfs's default.local to feed the parrot connector (the debug messages output by parrot connector is very few clued)

Thanks for any resource or knowledge sharing, then I could study them by myself.
ID: 49120 · Report as offensive     Reply Quote
maeax

Send message
Joined: 2 May 07
Posts: 2114
Credit: 159,867,104
RAC: 92,239
Message 49121 - Posted: 2 Jan 2024, 7:04:16 UTC

ID: 49121 · Report as offensive     Reply Quote
computezrmle
Volunteer moderator
Volunteer developer
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 15 Jun 08
Posts: 2420
Credit: 226,976,219
RAC: 131,807
Message 49122 - Posted: 2 Jan 2024, 7:27:25 UTC - in response to Message 49120.  

Would you mind giving a deeper explanation?

I have a Linux PC behind a firewall, which means, it does not allow fuse or autofs to access cvmfs shared dataset.

Fuse/autofs (in connection with CVMFS) are responsible for local mount points.
A firewall doesn't deny local mounting.



The only possible connecting method outside for Internet is via http(s) proxy.

If a firewall blocks HTTP requests from your CVMFS client it would also block HTTP requests from any process including Parrot.
If your pc is allowed to use the proxy just configure CVMFS to use that proxy.
ID: 49122 · Report as offensive     Reply Quote
maeax

Send message
Joined: 2 May 07
Posts: 2114
Credit: 159,867,104
RAC: 92,239
Message 49123 - Posted: 2 Jan 2024, 7:55:24 UTC

Cyarn and Cinny,
no Country, fake?
ID: 49123 · Report as offensive     Reply Quote
Cyanr & Cinny
Avatar

Send message
Joined: 1 Jul 20
Posts: 6
Credit: 18,348,912
RAC: 9,721
Message 49124 - Posted: 2 Jan 2024, 8:24:39 UTC - in response to Message 49123.  

hmmm... prefer not to say my home country. :) I am a real user...

ID: 49124 · Report as offensive     Reply Quote
Cyanr & Cinny
Avatar

Send message
Joined: 1 Jul 20
Posts: 6
Credit: 18,348,912
RAC: 9,721
Message 49125 - Posted: 2 Jan 2024, 8:35:40 UTC - in response to Message 49122.  

The explanation of CVMFS_HTTP_PROXY in default.local mentioned in CVMFS Online doc is confused me.... it delivered many technologies beyond http(s) stuffs.

I once setup a local squid proxy and connect to a parent upstream of another proxy (the firewall will not block http(s) communication by that specific proxy). But cvmfs setup chkconfig failed to retrieve those /cvmfs/some.cern.h/XXXXX information from my own squid.

Is there a way to make sure that those autofs/fuse activities are compatible to a "standard" squid proxy protocol/practice? Or may I wrongly configure my own squid server?

Anyway, I then will try to connect the upstream via CVMFS_HTTP_PROXY and bypass my local squid, which is not supported the cvmfs setup chkconfig.
ID: 49125 · Report as offensive     Reply Quote
maeax

Send message
Joined: 2 May 07
Posts: 2114
Credit: 159,867,104
RAC: 92,239
Message 49126 - Posted: 2 Jan 2024, 8:41:15 UTC - in response to Message 49124.  

In this moment you connect to Cern, or other projects of boinc,
you know, there is no useful time for be unidentified, otherwhise,
you don't connect to boinc.
Noone have an interest to find you.
ID: 49126 · Report as offensive     Reply Quote
computezrmle
Volunteer moderator
Volunteer developer
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 15 Jun 08
Posts: 2420
Credit: 226,976,219
RAC: 131,807
Message 49127 - Posted: 2 Jan 2024, 9:32:58 UTC - in response to Message 49125.  

Well, again:
Fuse/automount refers to local mountpoints on your pc.
It makes objects from the CVMFS cache visible below /cvmfs via standard filesystem methods.
This has nothing to do with networking.



Networking happens when an object is not in the local CVMFS cache.
Then the CVMFS client creates an HTTP request like
http://s1cern-cvmfs.openhtc.io/cvmfs/grid.cern.ch/.cvmfspublished
where
s1cern-cvmfs.openhtc.io is the closest server from the list configured via CVMFS_SERVER_URL
grid.cern.ch is the repository name (looks like a domain although it isn't!)
.cvmfspublished is the object name (=filename)

Those HTTP requests can either be send directly to the server or via an HTTP proxy.
It can be configured using CVMFS_HTTP_PROXY:

# recommended setting for direct connections
CVMFS_HTTP_PROXY="auto;DIRECT"

# example setting(s) to be used if a local proxy is available
# use either the proxy name/FQDN plus it's port
CVMFS_HTTP_PROXY="http://my_local_proxy.example.com:3128"
# or use the proxy IP (example!)
CVMFS_HTTP_PROXY="http://198.51.100.10:3128"



I once setup a local squid proxy and connect to a parent upstream
.
.
.
will try to connect the upstream via CVMFS_HTTP_PROXY and bypass my local squid

Either should work.
If you are not the admin of the upstream proxy the recommended way would be to set up your own proxy and use the existing one as parent.
Details can be found in the Squid manual.
If unsure ask the admin of the upstream proxy for help/permission.
ID: 49127 · Report as offensive     Reply Quote
computezrmle
Volunteer moderator
Volunteer developer
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 15 Jun 08
Posts: 2420
Credit: 226,976,219
RAC: 131,807
Message 49128 - Posted: 2 Jan 2024, 9:45:35 UTC

Cyanr & Cinny wrote:
hmmm... prefer not to say my home country. :) I am a real user...

+1


maeax wrote:
In this moment you connect to Cern, or other projects of boinc,
you know, there is no useful time for be unidentified, otherwhise,
you don't connect to boinc.
Noone have an interest to find you.

@maeax
Although it can be helpful in certain cases to know where a volunteer comes from (less helpful) or to have direct access to the logs via the project server (more helpful) no volunteer is forced to make that information public.
Hence, respect the volunteer's decision.
If you don't agree with it feel free to ignore them.
ID: 49128 · Report as offensive     Reply Quote
maeax

Send message
Joined: 2 May 07
Posts: 2114
Credit: 159,867,104
RAC: 92,239
Message 49129 - Posted: 2 Jan 2024, 10:52:24 UTC - in response to Message 49128.  

If you don't agree with it feel free to ignore them.

No comment
ID: 49129 · Report as offensive     Reply Quote

Questions and Answers : Unix/Linux : How to setup parrot connector on a LHC@Home client behind firewall


©2024 CERN