Message boards : CMS Application : singularity.conf not found
Message board moderation

To post messages, you must log in.

AuthorMessage
computezrmle
Volunteer moderator
Volunteer developer
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 15 Jun 08
Posts: 2401
Credit: 225,562,121
RAC: 121,264
Message 38245 - Posted: 14 Mar 2019, 22:44:42 UTC

From time to time I notice a CMS VM that prints the error "... singularity.conf not found".
Those VMs switch to idle a while later.

I suspect that at least one of the mirrors where the software packets are downloaded from is not up to date.
ID: 38245 · Report as offensive     Reply Quote
computezrmle
Volunteer moderator
Volunteer developer
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 15 Jun 08
Posts: 2401
Credit: 225,562,121
RAC: 121,264
Message 38246 - Posted: 15 Mar 2019, 7:10:51 UTC

Got a bit more of this error message.
It looks like:
"sed: can't read .... singularity.conf"
It appears short before the condor ping.

Sorry, the setup continues too fast to get the rest (yet).
ID: 38246 · Report as offensive     Reply Quote
ivan
Volunteer moderator
Project tester
Volunteer developer
Volunteer tester
Project scientist
Avatar

Send message
Joined: 29 Aug 05
Posts: 1005
Credit: 6,269,877
RAC: 404
Message 38247 - Posted: 15 Mar 2019, 8:40:04 UTC

There's a big spike in job failures for the last hour or two; haven't caught the problem yet.
ID: 38247 · Report as offensive     Reply Quote
Crystal Pellet
Volunteer moderator
Volunteer tester

Send message
Joined: 14 Jan 10
Posts: 1274
Credit: 8,480,870
RAC: 2,011
Message 38248 - Posted: 15 Mar 2019, 9:08:40 UTC - in response to Message 38247.  
Last modified: 15 Mar 2019, 9:12:43 UTC

There's a big spike in job failures for the last hour or two; haven't caught the problem yet.
Exit code 99109, that means Uncaught exception in WMAgent step executor
ID: 38248 · Report as offensive     Reply Quote
computezrmle
Volunteer moderator
Volunteer developer
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 15 Jun 08
Posts: 2401
Credit: 225,562,121
RAC: 121,264
Message 38249 - Posted: 15 Mar 2019, 9:23:05 UTC

Somebody at CERN may check the DNS nameservers that resolve gitlab.cern.ch.
I already asked Laurence.
ID: 38249 · Report as offensive     Reply Quote
Erich56

Send message
Joined: 18 Dec 15
Posts: 1688
Credit: 103,128,234
RAC: 121,135
Message 38250 - Posted: 15 Mar 2019, 11:37:39 UTC - in response to Message 38248.  
Last modified: 15 Mar 2019, 11:38:07 UTC

Exit code 99109, that means Uncaught exception in WMAgent step executor
didn't the WMAgent play a big role in some major disturbtions about a year ago (or longer back)?
ID: 38250 · Report as offensive     Reply Quote
ivan
Volunteer moderator
Project tester
Volunteer developer
Volunteer tester
Project scientist
Avatar

Send message
Joined: 29 Aug 05
Posts: 1005
Credit: 6,269,877
RAC: 404
Message 38251 - Posted: 15 Mar 2019, 14:39:39 UTC - in response to Message 38250.  

Exit code 99109, that means Uncaught exception in WMAgent step executor
didn't the WMAgent play a big role in some major disturbtions about a year ago (or longer back)?

There were some instabilities at one point, but I think we've caught most of the bugs by now. Since it's the thing that creates and manages all the jobs and steps of the work-flow (WM stands for work-flow management), it's not surprising that its name comes up from time to time.
ID: 38251 · Report as offensive     Reply Quote
Erich56

Send message
Joined: 18 Dec 15
Posts: 1688
Credit: 103,128,234
RAC: 121,135
Message 38252 - Posted: 15 Mar 2019, 17:07:24 UTC - in response to Message 38251.  

WM stands for work-flow management
thanks, Ivan, for this information; I was wondering anyway all time long what WM(Agent) exactly means :-)
ID: 38252 · Report as offensive     Reply Quote

Message boards : CMS Application : singularity.conf not found


©2024 CERN