Best/Preferred Proxy

Author	Message
TribbleRED Send message Joined: 9 May 10 Posts: 14 Credit: 3,104,500 RAC: 0	Message 44784 - Posted: 22 Apr 2021, 7:31:33 UTC Hello All, This is for both Win and Linux but I opted to post here instead of there... I am looking for a proxy solution to handle several nodes running 48wu at a time. Presently my list has come down to Squid, Varnish, and NGINX; with NGINX being more of a web server that can proxy. I would also very much like to set up with fail-over capability as well. Presently all my other projects are able to run on the nodes(48-56 threads per) in my racks but I can't do the same for either LHC or CPDN. Hence the proxy... Any help would be appreciated. Cheers ID: 44784 · Reply Quote

Crystal Pellet Volunteer moderator Volunteer tester Send message Joined: 14 Jan 10 Posts: 1422 Credit: 9,484,585 RAC: 1,038	Message 44785 - Posted: 22 Apr 2021, 8:10:47 UTC Last modified: 22 Apr 2021, 8:11:18 UTC Did you found this thread: Setting up a local Squid to work with LHC@home - HowTo ? ID: 44785 · Reply Quote

TribbleRED Send message Joined: 9 May 10 Posts: 14 Credit: 3,104,500 RAC: 0	Message 44786 - Posted: 22 Apr 2021, 8:53:51 UTC - in response to Message 44785. Last modified: 22 Apr 2021, 8:57:08 UTC Doh. I had searched for threads to answer these questions but I didn't seem to find this one. Thank you for posting the link. So it appears the one that is suggested is Squid, but why? Is it for it's reverse proxy capabilities or because its free...? There could be better solutions than Squid and like I said I am also in need of some fail-over capability. Thoughts? ID: 44786 · Reply Quote

computezrmle Volunteer moderator Volunteer developer Volunteer tester Help desk expert Send message Joined: 15 Jun 08 Posts: 2541 Credit: 254,608,838 RAC: 28,391	Message 44787 - Posted: 22 Apr 2021, 9:28:01 UTC - in response to Message 44786. Feel free to use any HTTP proxy you are familiar with, especially if you claim there is a better one than Squid. CERN and other large science organizations from all over the world use hundreds of Squids for many years to serve billions of requests per day. Cloudflare distributes LHC@home data via "openhtc.io" using hundreds of Squids in their datacenters. Fail-over can easily be implemented if your local apps support it - and that's the pitfall. It's BOINC that does NOT support an automatic fail-over configuration. 48 concurrently running tasks would just be a warmup for a single Squid instance. ID: 44787 · Reply Quote

TribbleRED Send message Joined: 9 May 10 Posts: 14 Credit: 3,104,500 RAC: 0	Message 44790 - Posted: 22 Apr 2021, 17:26:07 UTC - in response to Message 44787. Last modified: 22 Apr 2021, 17:27:17 UTC But why is it used? Specifically Squid. Do you know? ID: 44790 · Reply Quote

computezrmle Volunteer moderator Volunteer developer Volunteer tester Help desk expert Send message Joined: 15 Jun 08 Posts: 2541 Credit: 254,608,838 RAC: 28,391	Message 44803 - Posted: 24 Apr 2021, 11:24:24 UTC - in response to Message 44790. This is a 24-h-statistics page for atlasfrontier-ai.cern.ch: http://wlcg-squid-monitor.cern.ch/awstats/bin/awstats.pl?config=atlasfrontier.cern.ch&databasebreak=day&day=16 Total requests served: 188015011 Served from the cache: 175401418 (93%; HITS and MEM_HITS) Top 4 clients are other proxies inside the CERN network: ca-mey-frontier-*.cern.ch Each of them made 40-46 million requests (176.33 million in total) and distributed them to their clients. Guessed they run at the same efficiency of 93% their clients made 2.52 billion requests within 1 day. You wouldn't want to treat your original servers with that many requests, hence a proxy (cache) cascade. As already mentioned atlasfrontier-ai.cern.ch is just 1 out of many proxies used by CERN and others. They use Squid for many years as it is fast, reliable and fulfills all other major and minor requirements. ID: 44803 · Reply Quote

LHC@home