Message boards : CMS Application : no new WUs available
Message board moderation

To post messages, you must log in.

Previous · 1 . . . 10 · 11 · 12 · 13 · 14 · 15 · 16 . . . 20 · Next

AuthorMessage
Toby Broom
Volunteer moderator

Send message
Joined: 27 Sep 08
Posts: 798
Credit: 644,732,581
RAC: 233,772
Message 45974 - Posted: 2 Jan 2022, 7:57:28 UTC - in response to Message 45973.  

Me too +1
ID: 45974 · Report as offensive     Reply Quote
ivan
Volunteer moderator
Project tester
Volunteer developer
Volunteer tester
Project scientist
Avatar

Send message
Joined: 29 Aug 05
Posts: 997
Credit: 6,264,307
RAC: 71
Message 45975 - Posted: 2 Jan 2022, 12:40:43 UTC - in response to Message 45973.  

Yes, sorry, I thought we had more than enough tasks to be going on with when I checked yesterday, but we started running out early this morning. A new batch of jobs is working its way through the system
ID: 45975 · Report as offensive     Reply Quote
Erich56

Send message
Joined: 18 Dec 15
Posts: 1686
Credit: 100,369,175
RAC: 102,021
Message 45976 - Posted: 2 Jan 2022, 13:05:34 UTC - in response to Message 45975.  

...but we started running out early this morning...
too many people crunching CMS :-)
ID: 45976 · Report as offensive     Reply Quote
ivan
Volunteer moderator
Project tester
Volunteer developer
Volunteer tester
Project scientist
Avatar

Send message
Joined: 29 Aug 05
Posts: 997
Credit: 6,264,307
RAC: 71
Message 45977 - Posted: 2 Jan 2022, 13:17:17 UTC - in response to Message 45976.  
Last modified: 2 Jan 2022, 13:18:02 UTC

...but we started running out early this morning...
too many people crunching CMS :-)

Yes, last I looked there were no SixTrack jobs available, and we had briefly topped 1,000 jobs running, but my 10,000-job workflows usually last two days. That last one (submitted 21-12-31@1741 UTC) had only just started to run jobs yesterday, yet it ran out today at 0300 UTC. Guess I need to spend less time in bed and more at the keyboard for the rest of the holidays.
And yes, jobs/tasks are available again.
ID: 45977 · Report as offensive     Reply Quote
Erich56

Send message
Joined: 18 Dec 15
Posts: 1686
Credit: 100,369,175
RAC: 102,021
Message 45978 - Posted: 2 Jan 2022, 14:03:36 UTC - in response to Message 45977.  

Guess I need to spend less time in bed and more at the keyboard for the rest of the holidays.
:-) :-) :-)
ID: 45978 · Report as offensive     Reply Quote
Erich56

Send message
Joined: 18 Dec 15
Posts: 1686
Credit: 100,369,175
RAC: 102,021
Message 45981 - Posted: 3 Jan 2022, 12:40:37 UTC

what I now notice is a substantial drop in credit points with the tasks that were downloaded after the recent outage:

for example:

before: total time: 46,039.18 - CPU time: 45,421.00 - 454.66 points
after: total time: 48,611.40 - CPU time: 47,801.02 - 171.40 points

can anyone explain?
ID: 45981 · Report as offensive     Reply Quote
Toby Broom
Volunteer moderator

Send message
Joined: 27 Sep 08
Posts: 798
Credit: 644,732,581
RAC: 233,772
Message 45983 - Posted: 3 Jan 2022, 20:38:42 UTC - in response to Message 45981.  

https://boinc.berkeley.edu/trac/wiki/CreditNew

;)

It normally recovers after some time
ID: 45983 · Report as offensive     Reply Quote
Erich56

Send message
Joined: 18 Dec 15
Posts: 1686
Credit: 100,369,175
RAC: 102,021
Message 47846 - Posted: 13 Mar 2023, 13:56:20 UTC

queue empty :-(
ID: 47846 · Report as offensive     Reply Quote
Profile Magic Quantum Mechanic
Avatar

Send message
Joined: 24 Oct 04
Posts: 1114
Credit: 49,501,728
RAC: 4,157
Message 47851 - Posted: 13 Mar 2023, 20:37:48 UTC - in response to Message 47846.  

Same over at -dev so they must be cooking up a new batch on a monday.
ID: 47851 · Report as offensive     Reply Quote
ivan
Volunteer moderator
Project tester
Volunteer developer
Volunteer tester
Project scientist
Avatar

Send message
Joined: 29 Aug 05
Posts: 997
Credit: 6,264,307
RAC: 71
Message 47852 - Posted: 13 Mar 2023, 21:02:54 UTC - in response to Message 47846.  
Last modified: 13 Mar 2023, 21:05:42 UTC

Sorry, I thought the workflow was going to last longer, but we had a surge in usage -- perhaps another project ran out of tasks. (Ah, yes, ATLAS has run low lately.)
-dev uses the same job pool, so it runs out too.
ID: 47852 · Report as offensive     Reply Quote
Profile Magic Quantum Mechanic
Avatar

Send message
Joined: 24 Oct 04
Posts: 1114
Credit: 49,501,728
RAC: 4,157
Message 47853 - Posted: 14 Mar 2023, 0:50:23 UTC - in response to Message 47852.  
Last modified: 14 Mar 2023, 1:21:09 UTC


( I decided to delete my comment since it is just a waste of my time)
ID: 47853 · Report as offensive     Reply Quote
Erich56

Send message
Joined: 18 Dec 15
Posts: 1686
Credit: 100,369,175
RAC: 102,021
Message 47878 - Posted: 21 Mar 2023, 5:32:51 UTC

no new WUs available :-(
ID: 47878 · Report as offensive     Reply Quote
ivan
Volunteer moderator
Project tester
Volunteer developer
Volunteer tester
Project scientist
Avatar

Send message
Joined: 29 Aug 05
Posts: 997
Credit: 6,264,307
RAC: 71
Message 47879 - Posted: 21 Mar 2023, 15:32:01 UTC - in response to Message 47878.  

no new WUs available :-(

Grr... You take a day off and this happens!
Not sure what to make of it yet, my broadband access is limited to a 3G cell signal in a not-spot, struggling to make 4 Mbps on a good day. :-(
I'll poke around as best I can.
ID: 47879 · Report as offensive     Reply Quote
ivan
Volunteer moderator
Project tester
Volunteer developer
Volunteer tester
Project scientist
Avatar

Send message
Joined: 29 Aug 05
Posts: 997
Credit: 6,264,307
RAC: 71
Message 47882 - Posted: 22 Mar 2023, 14:56:25 UTC - in response to Message 47879.  

There seems to have been something changed at CERN, but no response from the team responsible yet.
ID: 47882 · Report as offensive     Reply Quote
Erich56

Send message
Joined: 18 Dec 15
Posts: 1686
Credit: 100,369,175
RAC: 102,021
Message 47883 - Posted: 22 Mar 2023, 15:50:35 UTC - in response to Message 47882.  

thanks, Ivan, for the interim information
ID: 47883 · Report as offensive     Reply Quote
ivan
Volunteer moderator
Project tester
Volunteer developer
Volunteer tester
Project scientist
Avatar

Send message
Joined: 29 Aug 05
Posts: 997
Credit: 6,264,307
RAC: 71
Message 47884 - Posted: 22 Mar 2023, 18:59:26 UTC - in response to Message 47882.  

The WMCore team cannot see why our jobs aren't running, so we need to take this up with other crews.
ID: 47884 · Report as offensive     Reply Quote
ivan
Volunteer moderator
Project tester
Volunteer developer
Volunteer tester
Project scientist
Avatar

Send message
Joined: 29 Aug 05
Posts: 997
Credit: 6,264,307
RAC: 71
Message 47890 - Posted: 23 Mar 2023, 8:08:08 UTC - in response to Message 47884.  

We are running again, but no news from anywhere as to what the problem was or who (if anyone) fixed it.
ID: 47890 · Report as offensive     Reply Quote
ivan
Volunteer moderator
Project tester
Volunteer developer
Volunteer tester
Project scientist
Avatar

Send message
Joined: 29 Aug 05
Posts: 997
Credit: 6,264,307
RAC: 71
Message 47892 - Posted: 23 Mar 2023, 13:13:33 UTC - in response to Message 47890.  

We are running again, but no news from anywhere as to what the problem was or who (if anyone) fixed it.

OK, we had a message from Submission Infrastructure team: "Hi it got broken as part of the move from GSI to IDTOKENS, the scheduler had a broken token and it was preventing it from connecting."
ID: 47892 · Report as offensive     Reply Quote
Erich56

Send message
Joined: 18 Dec 15
Posts: 1686
Credit: 100,369,175
RAC: 102,021
Message 47966 - Posted: 3 Apr 2023, 17:04:36 UTC

the queue has run dry :-(
ID: 47966 · Report as offensive     Reply Quote
ivan
Volunteer moderator
Project tester
Volunteer developer
Volunteer tester
Project scientist
Avatar

Send message
Joined: 29 Aug 05
Posts: 997
Credit: 6,264,307
RAC: 71
Message 47968 - Posted: 4 Apr 2023, 12:52:37 UTC - in response to Message 47966.  
Last modified: 4 Apr 2023, 12:53:36 UTC

Law of unintended consequences, I'm afraid. While trying to fix one problem, another one was created. It might be a bumpy ride over the next few days as we sort out a number of interdependencies. Hopefully we'll get it sorted but a) I'm not officially employed at the moment, due to bureaucratic bungling, and b) my Uni has a seven-day Easter break from tonight.
As of a few moments ago, jobs were flowing again.
ID: 47968 · Report as offensive     Reply Quote
Previous · 1 . . . 10 · 11 · 12 · 13 · 14 · 15 · 16 . . . 20 · Next

Message boards : CMS Application : no new WUs available


©2024 CERN