Message boards :
CMS Application :
EXIT_NO_SUB_TASKS
Message board moderation
Previous · 1 . . . 8 · 9 · 10 · 11 · 12 · 13 · 14 . . . 16 · Next
| Author | Message |
|---|---|
 Send message Joined: 29 Aug 05 Posts: 1065 Credit: 7,921,500 RAC: 13,646  |
We want to make another update to the WMAgent codes on MondayIvan, is there a problem with the update? There are still no tasks available. Yes, it was delayed. We started up again about 2130 UTC.  |
|
Send message Joined: 29 Aug 05 Posts: 1065 Credit: 7,921,500 RAC: 13,646 |
We want to make another update to the WMAgent codes on Monday, ...a question just out of curiosity: why do they fiddle around with this WMAgent that frequently? Software evolves. At the moment they are updating everything to run in Kubernetes containers so there is extra development going on. You don't see every release of WMCore, there are sometimes 2 or 3 per week. Alan only updates our Agent with a stable release when it's needed to keep in step with other systems. |
 Send message Joined: 15 Jun 08 Posts: 2549 Credit: 255,377,016 RAC: 63,023   |
... We started up again about 2130 UTC. Subtasks may be there but ATM there are no new tasks available at the project server. https://lhcathome.cern.ch/lhcathome/server_status.php |
|
Send message Joined: 29 Aug 05 Posts: 1065 Credit: 7,921,500 RAC: 13,646 |
... We started up again about 2130 UTC. Unfortunately the job queue limits were not adjusted to our normal values last night, and WMAgent/Condor was limiting us to 100 jobs (as you might have noticed in the job graphs). I guess this screwed up Laurence's scripts for creating BOINC tasks depending on the number of queued jobs. I alerted Alan to this and we are back at our normal numbers now. |
|
Send message Joined: 15 Jun 08 Posts: 2549 Credit: 255,377,016 RAC: 63,023 |
... we are back at our normal numbers now. Yes, we are. Thanks. |
|
Send message Joined: 15 Jun 08 Posts: 2549 Credit: 255,377,016 RAC: 63,023 |
Since 8:00 UTC this morning fresh CMS tasks fail with "EXIT_NO_SUB_TASKS". |
|
Send message Joined: 29 Aug 05 Posts: 1065 Credit: 7,921,500 RAC: 13,646 |
Looks like a problem somewhere for a while. It seems to be recovering now but we're low on jobs; I'll submit some more. |
|
Send message Joined: 29 Aug 05 Posts: 1065 Credit: 7,921,500 RAC: 13,646 |
Advance Warning: We want to update our WMCore/WMAgent software next week. If I've done my sums right, we'll start running out of jobs sometime Monday night or early Tuesday. Please set your CMS project to No New Tasks on Monday night, or whenever you start to see the "Running Jobs" graph start to dip. Thanks, ivan |
|
Send message Joined: 18 Dec 15 Posts: 1831 Credit: 119,603,782 RAC: 46,039
|
Ivan, thanks for the advance information :-) |
|
Send message Joined: 18 Nov 17 Posts: 131 Credit: 55,876,215 RAC: 6,470
|
I see number of jobs to send on LHC status page. Is it time to turn NNT off? |
|
Send message Joined: 15 Jun 08 Posts: 2549 Credit: 255,377,016 RAC: 63,023 |
They obviously did the planned update this morning. ATM the number of running subtasks is increasing. Hence, it may be save to continue CMS. |
|
Send message Joined: 15 Jun 08 Posts: 2549 Credit: 255,377,016 RAC: 63,023 |
Looks like the new CMS subtasks process 5000 records instead of 10000. By intention? |
|
Send message Joined: 14 Jan 10 Posts: 1429 Credit: 9,524,756 RAC: 3,821  |
Looks like the new CMS subtasks process 5000 records instead of 10000.Can't recognize that: 'FirstEvent' : 5910001, 'LastEvent' : 5920000 |
|
Send message Joined: 15 Jun 08 Posts: 2549 Credit: 255,377,016 RAC: 63,023 |
Only the first 25 subtasks I got after the restart were short ones with 5000 records. Since then they run the usual 10000 records. |
|
Send message Joined: 18 Dec 15 Posts: 1831 Credit: 119,603,782 RAC: 46,039
|
having another problem since last night, see here: https://lhcathome.cern.ch/lhcathome/results.php?hostid=10679599 207 (0x000000CF) EXIT_NO_SUB_TASKS ... 2021-02-23 08:10:07 (11516): Guest Log: [ERROR] No jobs were available to run. |
|
Send message Joined: 18 Dec 15 Posts: 1831 Credit: 119,603,782 RAC: 46,039
|
at some point of time, there was an automatic stop of new CMS tasks to be downloaded once the system runs out of jobs. Is this no longer working? |
|
Send message Joined: 18 Dec 15 Posts: 1831 Credit: 119,603,782 RAC: 46,039
|
Ivan, can you estimate when jobs will be available again ? |
|
Send message Joined: 29 Aug 05 Posts: 1065 Credit: 7,921,500 RAC: 13,646 |
Looks like the new CMS subtasks process 5000 records instead of 10000. Sorry, didn't see this earlier. My colleague posted a small workflow after the intervention with different parameters to the ones I usually use. |
|
Send message Joined: 29 Aug 05 Posts: 1065 Credit: 7,921,500 RAC: 13,646 |
Ivan, can you estimate when jobs will be available again ? Haven't had a response from anyone at CERN yet, so I don't know if this is a repeat of Sunday night's problem. |
|
Send message Joined: 29 Aug 05 Posts: 1065 Credit: 7,921,500 RAC: 13,646 |
at some point of time, there was an automatic stop of new CMS tasks to be downloaded once the system runs out of jobs. A different problem the last few days. We haven't run out of jobs; there are plenty in the queue but the condor server isn't sending them out, for reasons I haven't found out yet. |
©2025 CERN