Message boards : Number crunching : Scheduler request to https://lhcathome.cern.ch/lhcathome_cgi/cgi failed: HTTP gateway timeout
Message board moderation
| Author | Message |
|---|---|
|
Send message Joined: 27 Sep 08 Posts: 931 Credit: 781,051,044 RAC: 84,707    |
Is anyone else seeing this in there logs? |
 Send message Joined: 28 Sep 04 Posts: 802 Credit: 65,421,179 RAC: 24,418 
|
Yes, I'm seeing this also. But not on every request. My oldest message logs start from 5th of March and this message was there already then but I think situation has been getting worse since then. 
|
|
Send message Joined: 27 Sep 08 Posts: 931 Credit: 781,051,044 RAC: 84,707 |
I agree it does seem like it got worse. It a little irritating as it seems to stop the scheduler request so completed WU pile up it seem until I can get a sucessful connection |
|
Send message Joined: 17 Oct 06 Posts: 99 Credit: 65,246,259 RAC: 13,283 
|
This has been happening to me to. For the latest outage it started around 7:19 PM Est yesterday (Mar 12th) and just cleared up at 8:51 am EST (Mar 13th). Is the scheduling server getting turned off over night or something lol? |
|
Send message Joined: 27 Sep 08 Posts: 931 Credit: 781,051,044 RAC: 84,707 |
Any update to this? I'm still seeing: HTTP gateway timeout / (HTTP gateway timeout + Scheduler request completed ) Failure rate: 21% Failure rate: 21% Failure rate: 16% Failure rate: 21% Failure rate: 27% |
 Send message Joined: 15 Jun 08 Posts: 2739 Credit: 301,811,487 RAC: 83,089 |
In reply to Toby Broom's message of 7 Apr 2026: Any update to this? I'm still seeing: @Laurence @Nils This is around for a couple of months! It happens every day although we currently don't have any load from ATLAS/CMS/XTrack. Please investigate and fix it. |
|
Send message Joined: 27 Sep 08 Posts: 931 Credit: 781,051,044 RAC: 84,707 |
The timeout is after 1 min, which is the default in squid, should we increase the setting? |
|
Send message Joined: 15 Jun 08 Posts: 2739 Credit: 301,811,487 RAC: 83,089 |
In reply to Toby Broom's message of 8 Apr 2026: The timeout is after 1 min, which is the default in squid, should we increase the setting? Which timeout do you mean? Squid has many of them. |
|
Send message Joined: 27 Sep 08 Posts: 931 Credit: 781,051,044 RAC: 84,707 |
this one https://www.squid-cache.org/Doc/config/connect_timeout/ the other don't seem to be 1 min by default |
|
Send message Joined: 15 Jun 08 Posts: 2739 Credit: 301,811,487 RAC: 83,089 |
This means Squid times out if a connection can't be established within 1 min. Usually connections should be established within a few ms. If connections to other destinations via Squid work fine a timeout in this phase means this destination has some issues. Transfers happen once the connection is fully established. They have other (longer) timeouts. |
|
Send message Joined: 14 Sep 08 Posts: 53 Credit: 84,677,282 RAC: 46,857
|
I honestly don't think the scheduler request timeout has anything to do with squid (or our local setup) as I don't run squid but still getting it. I noticed that my own tasks page also no longer opens, with 504 Gateway Timeout. This feels like some kind of overload on the server side. |
©2026 CERN