Message boards : Number crunching : CentOS9 squid Win11pro
Message board moderation

To post messages, you must log in.

AuthorMessage
maeax

Send message
Joined: 2 May 07
Posts: 1752
Credit: 136,505,063
RAC: 28,175
Message 46851 - Posted: 7 Jun 2022, 8:34:35 UTC
Last modified: 7 Jun 2022, 8:59:27 UTC

Using normaly CentOS8 Squid as VM.

When installing a CentOS9-VM Squid on a Win11pro PC, Boinc discovers this VM and use it for this Win11pro, but the Squid is connected on a CentOS8-VM in this Boinc!

Have this now on two PC's.
No idea why.

Atlas stuck for upload, when using this CentOS9 Squid in Boinc.
CMS using and uploading is successful.
This is a example for Atlas:
07.06.2022 10:56:29 | LHC@home | Backing off 00:03:15 on upload of yjiNDmywgI1nfZGDcpSWOuwoABFKDmABFKDmTzMbDmY1DLDmjpebSm_0_r934943942_ATLAS_result
07.06.2022 10:56:30 | | Internet access OK - project servers may be temporarily down.
ID: 46851 · Report as offensive     Reply Quote
computezrmle
Volunteer moderator
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 15 Jun 08
Posts: 2182
Credit: 185,907,747
RAC: 186,985
Message 46852 - Posted: 8 Jun 2022, 7:34:30 UTC - in response to Message 46851.  

It's very hard to understand the post.


Using normaly CentOS8 Squid as VM.

Does it mean you run a CentOS 8 VM and within this VM you run a Squid from the CentOS 8 repository?
Does it mean you configured most of your BOINC clients to use that Squid?
Which Squid version did you install on CentOS 8?
squid --version




When installing a CentOS9-VM Squid on a Win11pro PC

So, you installed CentOS 9 on another VM (on a Win 11 host, but the host OS doesn't really matter), including the Squid package from the CentOS 9 repository?
Which Squid version did you install on CentOS 9?
Just in case CentOS 9 installs Squid version >4:
AFAIK Squid version >4 is not yet certified to be used in the CERN environment.




Boinc discovers this VM and use it for this Win11pro ...

What does "discovers" mean? It's totally unclear.
... but the Squid is connected on a CentOS8-VM in this Boinc

Which Squid do you mean with "the Squid"?
Do you mean the proxy entry in BOINC (which BOINC client)?



Atlas stuck for upload, when using this CentOS9 Squid in Boinc.
CMS using and uploading is successful.

So far you explained all your BOINC clients connect via the Squid on CentOS 8 (which worked fine even for ATLAS).

CMS job uploads are done from inside the CMS VM and always bypass the Squid you configured in BOINC.
ATLAS result uploads are first copied back from the VM to BOINC and BOINC handles the upload either directly or via the configured proxy.
ID: 46852 · Report as offensive     Reply Quote
maeax

Send message
Joined: 2 May 07
Posts: 1752
Credit: 136,505,063
RAC: 28,175
Message 46853 - Posted: 8 Jun 2022, 8:01:00 UTC - in response to Message 46851.  
Last modified: 8 Jun 2022, 8:02:39 UTC

Using normaly CentOS8 Squid as VM.

CentOS8-VM Squid 4.15
CentOS9-VM Squid 5.2 (only CMS atm)

Both with using HTCP.
ID: 46853 · Report as offensive     Reply Quote
maeax

Send message
Joined: 2 May 07
Posts: 1752
Credit: 136,505,063
RAC: 28,175
Message 46854 - Posted: 8 Jun 2022, 9:54:59 UTC - in response to Message 46852.  

Atlas stuck for upload, when using this CentOS9 Squid in Boinc.
CMS using and uploading is successful.

So far you explained all your BOINC clients connect via the Squid on CentOS 8 (which worked fine even for ATLAS).

CMS job uploads are done from inside the CMS VM and always bypass the Squid you configured in BOINC.
ATLAS result uploads are first copied back from the VM to BOINC and BOINC handles the upload either directly or via the configured proxy.


CMS is only running for testing CentOS9-VM in this Win10pro:https://lhcathome.cern.ch/lhcathome/show_host_detail.php?hostid=10408749

Boinc 7.16.20 in both: Win10pro and Win11pro.

Atlas works with Squid CentOS9-VM, BUT upload stuck, see first message. Squid CentOS8-VM no problems.
Same squid.conf in CentOS8-VM and CentOS9-VM, but different LAN-Adresses, of course.
ID: 46854 · Report as offensive     Reply Quote
computezrmle
Volunteer moderator
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 15 Jun 08
Posts: 2182
Credit: 185,907,747
RAC: 186,985
Message 46855 - Posted: 8 Jun 2022, 10:35:46 UTC - in response to Message 46854.  

Atlas works with Squid CentOS9-VM, BUT upload stuck, see first message. Squid CentOS8-VM no problems.
Same squid.conf in CentOS8-VM and CentOS9-VM, but different LAN-Adresses, of course.

As you wrote: you use a Squid 5.2 on your CentOS 9 VM and that may cause the upload problem.

If you want a stable system it is suggested to stay on Squid 4.x.
In your case (many BOINC clients and lots of hardware) it may be a good idea to run one of the smaller computers with a recent Linux, e.g. CentOS 8 if you like it, and use it exclusively for Squid.
ID: 46855 · Report as offensive     Reply Quote
maeax

Send message
Joined: 2 May 07
Posts: 1752
Credit: 136,505,063
RAC: 28,175
Message 46856 - Posted: 8 Jun 2022, 10:44:21 UTC - in response to Message 46855.  

CentOS8 have no Mirror, out of support since 2022/01.
40 Year Windows, my age allowed me no new OS (Linux!)
ID: 46856 · Report as offensive     Reply Quote
Henry Nebrensky

Send message
Joined: 13 Jul 05
Posts: 163
Credit: 14,819,914
RAC: 1,296
Message 46857 - Posted: 8 Jun 2022, 20:41:12 UTC - in response to Message 46856.  

CentOS8 have no Mirror, out of support since 2022/01.
As announced a couple of years ago, CentOS became "CentOS Stream". Conversion instructions for CentOS8 to CentOS Stream8 are at https://www.centos.org/centos-stream/.
(I've not tried them; I've been reverting to C7 instead).
ID: 46857 · Report as offensive     Reply Quote
maeax

Send message
Joined: 2 May 07
Posts: 1752
Credit: 136,505,063
RAC: 28,175
Message 46858 - Posted: 9 Jun 2022, 4:59:19 UTC - in response to Message 46855.  

In your case (many BOINC clients and lots of hardware) it may be a good idea to run one of the smaller computers with a recent Linux, e.g. CentOS 8 if you like it, and use it exclusively for Squid.


This Computer have a VM for using Squid:
PCRYZEN9 with RYZEN9COS8SQ (Squid-CentOS8)
RYZENMP with RYZENMPSQ (Squid-CentOS9)
P620-3995 with P620-CentOS9 (Squid-CentOS9)

Where PCRYZEN9 is a Win10-Workstation OS.
The other two are Win11pro OS.
ID: 46858 · Report as offensive     Reply Quote
computezrmle
Volunteer moderator
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 15 Jun 08
Posts: 2182
Credit: 185,907,747
RAC: 186,985
Message 46859 - Posted: 9 Jun 2022, 11:19:53 UTC - in response to Message 46856.  

CentOS8 have no Mirror

Archived Centos packages can be downloaded here:
https://vault.centos.org

Depending on the libs a package is linked to it may or may not work on the recent Centos version.


This is a direct link to Squid 4.15 (for Centos 8.5) on the archive server:
https://vault.centos.org/8.5.2111/AppStream/x86_64/os/Packages/squid-4.15-1.module_el8.5.0+860+303ccf92.x86_64.rpm
Just install it on Centos 9 and see if it works.



As an option there's a CERN/Fermilab customized Squid version included in the Frontier package.
See:
https://twiki.cern.ch/twiki/bin/view/Frontier/InstallSquidTarball
The whole Frontier package is primarily thought for data centers, hence may be a bit too heavy for private use.
ID: 46859 · Report as offensive     Reply Quote
maeax

Send message
Joined: 2 May 07
Posts: 1752
Credit: 136,505,063
RAC: 28,175
Message 46860 - Posted: 9 Jun 2022, 12:22:14 UTC - in response to Message 46859.  

Have 2 GByte Squid-Data in access.log in a few days for this CentOS8-VM.
CentOS9-VM is for testing with CMS.
Have no problem to wait and see what happens.
Secure-Patches for this CentOS8-VM coming from Cern.

Thank you for this alternates. All is well.
ID: 46860 · Report as offensive     Reply Quote
maeax

Send message
Joined: 2 May 07
Posts: 1752
Credit: 136,505,063
RAC: 28,175
Message 46861 - Posted: 9 Jun 2022, 13:36:05 UTC - in response to Message 46860.  

Have one time forgotten to enable the CentOS9-VM in Boinc for CMS.
wpad.dat show this:
// no squid found matching the remote ip address
function FindProxyForURL(url, host) {
if (shExpMatch(url, "*.openhtc.io*")) {
return "DIRECT";
}
return "PROXY http://lhchomeproxy.cern.ch:3125; PROXY http://lhchomeproxy.fnal.gov:3125";
}
ID: 46861 · Report as offensive     Reply Quote
maeax

Send message
Joined: 2 May 07
Posts: 1752
Credit: 136,505,063
RAC: 28,175
Message 46939 - Posted: 27 Jun 2022, 15:11:22 UTC - in response to Message 46861.  

Theory is also working with Squid 5.2 in CentOS9-VM.
ID: 46939 · Report as offensive     Reply Quote
maeax

Send message
Joined: 2 May 07
Posts: 1752
Credit: 136,505,063
RAC: 28,175
Message 46972 - Posted: 4 Jul 2022, 9:35:33 UTC - in response to Message 46939.  

Squid 5.5 on CentOS9-VM:
2022/07/04 09:48:01 kid1| Accepting HTTP Socket connections at conn3 local=10.241.124.15:3128 remote=[::] FD 11 flags=9
2022/07/04 09:48:01 kid1| Accepting HTCP messages on [::]:4827
2022/07/04 09:48:01 kid1| Sending HTCP messages from [::]:4827
2022/07/04 09:48:02 kid1| storeLateRelease: released 0 objects

Using only Squid on CentOS8-VM atm, but testing using of CentOS9-VM!
ID: 46972 · Report as offensive     Reply Quote
maeax

Send message
Joined: 2 May 07
Posts: 1752
Credit: 136,505,063
RAC: 28,175
Message 46978 - Posted: 6 Jul 2022, 5:20:11 UTC - in response to Message 46972.  

Using only Squid on CentOS8-VM atm, but testing using of CentOS9-VM!

access.log in CentOS8-VM reaching 2.3 GByte in three days now, before more than one week!
ID: 46978 · Report as offensive     Reply Quote

Message boards : Number crunching : CentOS9 squid Win11pro


©2023 CERN