Message boards : CMS Application : All tasks failing
Message board moderation

To post messages, you must log in.

Previous · 1 · 2

AuthorMessage
Pascal

Send message
Joined: 13 May 20
Posts: 33
Credit: 1,191,508
RAC: 1,914
Message 48738 - Posted: 3 Oct 2023, 14:05:54 UTC - in response to Message 48737.  

ok merci
ID: 48738 · Report as offensive     Reply Quote
computezrmle
Volunteer moderator
Volunteer developer
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 15 Jun 08
Posts: 2443
Credit: 230,987,457
RAC: 123,261
Message 48739 - Posted: 3 Oct 2023, 14:14:33 UTC - in response to Message 48734.  

Your 1st job failed after a "[ERROR] glidein exited with return value 1.".
Those errors can't be investigated at the BOINC level since they happen deeper in the scripts.
ID: 48739 · Report as offensive     Reply Quote
computezrmle
Volunteer moderator
Volunteer developer
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 15 Jun 08
Posts: 2443
Credit: 230,987,457
RAC: 123,261
Message 48740 - Posted: 3 Oct 2023, 14:20:46 UTC - in response to Message 48738.  

All of your CMS tasks fail because the VMs don't have the permission to write the heartbeat file:
2023-10-03 01:50:09 (7176): VM Heartbeat file specified, but missing.

Together with other log entries it looks like your user account is not a member of the boinc group.
ID: 48740 · Report as offensive     Reply Quote
computezrmle
Volunteer moderator
Volunteer developer
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 15 Jun 08
Posts: 2443
Credit: 230,987,457
RAC: 123,261
Message 48741 - Posted: 3 Oct 2023, 14:29:52 UTC - in response to Message 48733.  

12:27:09 +0200 2023-10-03 [INFO] Requesting an idtoken from LHC@Home
X509_USER_PROXY = /tmp/x509up_u1000

Waiting, waiting.. since a quarter of hour.
Half an hour now, canceling after one hour of waiting...

2023-10-03 12:26:53 (28104): Guest Log: [INFO] Reading volunteer information
2023-10-03 12:27:08 (28104): Guest Log: [INFO] Requesting an X509 credential from LHC@home
2023-10-03 12:27:09 (28104): Guest Log: [INFO] CMS application starting. Check log files.
2023-10-03 12:27:10 (28104): Guest Log: [INFO] Requesting an idtoken from LHC@home
2023-10-03 13:14:39 (28104): Powering off VM.
2023-10-03 13:14:40 (28104): Successfully stopped VM.
2023-10-03 13:14:40 (28104): Deregistering VM. (boinc_a31803bb699ff414, slot#33)
2023-10-03 13:14:40 (28104): Removing network bandwidth throttle group from VM.
2023-10-03 13:14:40 (28104): Removing VM from VirtualBox.

Hypervisor System Log:

313:12:21.054578 Saving settings file "S:\ProgramData\BOINC\slots\17\boinc_9ebd457c4ec9cc22\boinc_9ebd457c4ec9cc22.vbox" with version "1.19-windows"

That task was running fine until an impatient user cancelled it due to a misinterpretation of the console 1 output:
Run time 	48 min 57 sec
CPU time 	44 min 59 sec

Should have been better to also check ALT-F2, ALT-F3 and ALT-F4.
ID: 48741 · Report as offensive     Reply Quote
Pascal

Send message
Joined: 13 May 20
Posts: 33
Credit: 1,191,508
RAC: 1,914
Message 48743 - Posted: 3 Oct 2023, 16:51:15 UTC - in response to Message 48741.  

pour moi ça refonctionne.j'ai désinstaller virtualbox 7.010 et j'ai installé la version 6.146.


for me it works again. I uninstall virtualbox 7.010 and I installed version 6.146.[/img]
ID: 48743 · Report as offensive     Reply Quote
Erich56

Send message
Joined: 18 Dec 15
Posts: 1713
Credit: 106,373,063
RAC: 72,334
Message 48744 - Posted: 3 Oct 2023, 19:53:21 UTC

something seems indeed to be wrong with all the CMS tasks I downloaded this early afternoon on several machines, i.e. about 8 hours ago.
CPU activity ended after about three hours, and since then the CPU is running idle. However, none of these tasks comes to an end :-(
Anyone making the same experience?
ID: 48744 · Report as offensive     Reply Quote
maeax

Send message
Joined: 2 May 07
Posts: 2151
Credit: 160,991,805
RAC: 53,647
Message 48745 - Posted: 4 Oct 2023, 1:41:59 UTC - in response to Message 48744.  

one job inside of a Task need two and a half hour.
The Task run about twelve hour.
ID: 48745 · Report as offensive     Reply Quote
Erich56

Send message
Joined: 18 Dec 15
Posts: 1713
Credit: 106,373,063
RAC: 72,334
Message 48746 - Posted: 4 Oct 2023, 2:22:22 UTC - in response to Message 48745.  

one job inside of a Task need two and a half hour.
The Task run about twelve hour.
yes, thats true. All tasks which I started yesterday early afternoon got finished during last night, after about 10-12 hours. So there is a big difference in the runtime compared to what it was before. But it's okay, as long as one is aware of it.
ID: 48746 · Report as offensive     Reply Quote
Harri Liljeroos
Avatar

Send message
Joined: 28 Sep 04
Posts: 684
Credit: 44,203,192
RAC: 13,451
Message 48748 - Posted: 4 Oct 2023, 7:15:31 UTC

From the server status page you can see what was the average runtime of a task (Runtime of recent tasks in hours: average, min, max).
ID: 48748 · Report as offensive     Reply Quote
Erich56

Send message
Joined: 18 Dec 15
Posts: 1713
Credit: 106,373,063
RAC: 72,334
Message 48749 - Posted: 4 Oct 2023, 9:06:51 UTC - in response to Message 48748.  

From the server status page you can see what was the average runtime of a task (Runtime of recent tasks in hours: average, min, max).
I know that there is this kind of information shown on the server status page.
However, if one sees something like
0.81 (0.03 - 7.89)
it's clear that this does not tell much.

Fact is that here on my machines, it now takes the tasks about 3 times as long as before. But that's the way it is at the moment, no problem for me.
ID: 48749 · Report as offensive     Reply Quote
Harri Liljeroos
Avatar

Send message
Joined: 28 Sep 04
Posts: 684
Credit: 44,203,192
RAC: 13,451
Message 48750 - Posted: 4 Oct 2023, 9:25:44 UTC

Yes, it is sometimes difficult to put it to perspective. Here is the same info as graphical format so you can view the history: https://grafana.kiska.pw/d/boinc/boinc?orgId=1&var-project=lhc@home&from=now-2d&to=now&refresh=30m

There are other projects there too.
ID: 48750 · Report as offensive     Reply Quote
Erich56

Send message
Joined: 18 Dec 15
Posts: 1713
Credit: 106,373,063
RAC: 72,334
Message 48751 - Posted: 4 Oct 2023, 9:36:12 UTC - in response to Message 48750.  

Yes, it is sometimes difficult to put it to perspective. Here is the same info as graphical format so you can view the history: https://grafana.kiska.pw/d/boinc/boinc?orgId=1&var-project=lhc@home&from=now-2d&to=now&refresh=30m

There are other projects there too.
Thanks, Harri, for the link.
The per app runtime graph is exactly in accordance with my observation, what concerns CMS; in fact, this is true also for ATLAS and Theory :-)
ID: 48751 · Report as offensive     Reply Quote
Previous · 1 · 2

Message boards : CMS Application : All tasks failing


©2024 CERN