Message boards : Theory Application : Made a small script to keep an eye on Theory jobs with correct % done
Message board moderation

To post messages, you must log in.

Previous · 1 · 2

AuthorMessage
seanr22a

Send message
Joined: 29 Nov 18
Posts: 40
Credit: 2,580,683
RAC: 732
Message 51730 - Posted: 19 Mar 2025, 16:35:35 UTC - in response to Message 51728.  
Last modified: 19 Mar 2025, 16:52:05 UTC


I'm neither admin nor do I force you to do so.
It's just the case that this forum is not made to handle things like code management.
Github is made exactly for this.


Volunteer moderator, I listen to those moderating the forum. It is as it should be.

I found my old github account, added the theory.sh + a README file:

https://github.com/seanr22a/LHC-Theory/blob/main/theory.sh
https://github.com/seanr22a/LHC-Theory/blob/main/README.md

There is an updated theory.sh there now with the latest updates. I don't think it ends there because I have no idea how many kinds of theory jobs there is so it will probably be many more updates.

If someone find anything wrong in the script, please PM me. Shell scripting is not my expertise so take it for what it is. I would prefer write it in C which is my home turf :)
ID: 51730 · Report as offensive     Reply Quote
seanr22a

Send message
Joined: 29 Nov 18
Posts: 40
Credit: 2,580,683
RAC: 732
Message 51732 - Posted: 20 Mar 2025, 7:28:40 UTC - in response to Message 51730.  
Last modified: 20 Mar 2025, 7:29:51 UTC

For those interested, check github for updates regulary. I will not post any more about updates here. Latest update was today 2025-03-20 14:01 Asia/Bangkok.

https://github.com/seanr22a/LHC-Theory/blob/main/theory.sh
https://github.com/seanr22a/LHC-Theory/blob/main/README.md

This is how it looks now with some small adjustments to get the terminal window width smaller:

ID: 51732 · Report as offensive     Reply Quote
M0CZY

Send message
Joined: 27 Apr 24
Posts: 17
Credit: 1,129,616
RAC: 268
Message 51837 - Posted: 15 Apr 2025, 14:16:54 UTC

When you are running Theory work units under VirtualBox, is there any easy way to check the actual progress?
Is there a file you can check to see the progress?
ID: 51837 · Report as offensive     Reply Quote
Crystal Pellet
Volunteer moderator
Volunteer tester

Send message
Joined: 14 Jan 10
Posts: 1450
Credit: 9,747,300
RAC: 363
Message 51839 - Posted: 15 Apr 2025, 15:10:29 UTC - in response to Message 51837.  

When you are running Theory work units under VirtualBox, is there any easy way to check the actual progress?
Is there a file you can check to see the progress?

Since you are using Linux: watch -n 1 tail -n 10 boincdatapath/slots/slotnumber/shared/runRivet.log
ID: 51839 · Report as offensive     Reply Quote
seanr22a

Send message
Joined: 29 Nov 18
Posts: 40
Credit: 2,580,683
RAC: 732
Message 51919 - Posted: 2 Jun 2025, 14:37:19 UTC - in response to Message 51837.  

The script I made for the native version of Theory doesn't work with the vbox version. There are similarities but after looking at one runRivet.log file I found no info about total events for the job so it's not possible to know how much is left. There is an event counter but nothing telling how far it will go.

I will look at more log files and see if there is any way to find how many events the job has. If I can find total events when it's easy to update the script.
ID: 51919 · Report as offensive     Reply Quote
computezrmle
Volunteer moderator
Volunteer developer
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 15 Jun 08
Posts: 2636
Credit: 277,682,002
RAC: 152,236
Message 51920 - Posted: 2 Jun 2025, 14:57:07 UTC - in response to Message 51919.  

Unlike older Theory vbox versions the one currently in use tails it's runRivet.log to '...\slots\x\shared\'.
The #events can be found around line 32 like:
nevts=100000


or in line 1 as the 2nd last parameter like:
===> [runRivet] Mon Jun  2 01:27:13 PM UTC 2025 [boinc pp w1j 7000 100 - pythia8 8.313 default 100000 1173]


Use either.
ID: 51920 · Report as offensive     Reply Quote
seanr22a

Send message
Joined: 29 Nov 18
Posts: 40
Credit: 2,580,683
RAC: 732
Message 51921 - Posted: 2 Jun 2025, 15:26:52 UTC - in response to Message 51920.  

the #events can be found around line 32


Thanks for the info! It confirms what I just found tracing one logfile from beginning to end.

I will do a vbox version of the script but it will take a while as I have to receive all the different kinds of jobs as before. I assume it will be the same as with native with many kinds of jobs the script must handle.

I'm late to the party I have not been running Boinc for a couple of months and found out today that Theory moved to vbox in April. I also found that I had to uncheck the box "Run native if available" in the settings to receive any Theory jobs :)
ID: 51921 · Report as offensive     Reply Quote
seanr22a

Send message
Joined: 29 Nov 18
Posts: 40
Credit: 2,580,683
RAC: 732
Message 51923 - Posted: 3 Jun 2025, 8:49:41 UTC - in response to Message 51921.  

First version is now available for theory - vbox: https://github.com/seanr22a/LHC-Theory/blob/main/theory-vbox.sh

Please PM me if you find any issues.

It was less work to make it running for vbox than I thought. I will load one of my servers with as many theory jobs as possible and let it run to catch any issues.
The jobs available now seems to be only Pythia whatever that is - my degree is not in physic :-) Looking at the old native version there was several different types of jobs so I assume it will be the same in the vbox version so I will have to wait and see what happens when other type of jobs get available.
ID: 51923 · Report as offensive     Reply Quote
seanr22a

Send message
Joined: 29 Nov 18
Posts: 40
Credit: 2,580,683
RAC: 732
Message 51937 - Posted: 7 Jun 2025, 7:21:38 UTC - in response to Message 51923.  

The vbox version is working fine. I received a lot of different jobs including those with a long Preparation time and everything is working. Download the theory-vbox.sh script from https://github.com/seanr22a/LHC-Theory/

Do not use the old theory.sh script as it works only with native jobs. It's not likely the native jobs comes back as everything is moving to use vbox so everything runs in a controlled environment.
ID: 51937 · Report as offensive     Reply Quote
Profile Guy
Avatar

Send message
Joined: 9 Feb 08
Posts: 58
Credit: 1,919,248
RAC: 477
Message 51941 - Posted: 8 Jun 2025, 10:57:29 UTC - in response to Message 51937.  
Last modified: 8 Jun 2025, 11:42:29 UTC

Works on my box.
Very handy!

--- LHC Theory - localhost.localdomain ---- 2025-06-08 11:42:29 ------------------------------------------------------
 
                                     |          Events          |
  Slot                     Job id    Total  Processed   Remaining     Elapsed time  Completed %   Err 
------------------------------------------------------------------------------------------------------
     2   Theory_2914-4642936-34_0   100000      91600        8400   0 day(s) 09:29         91.6       

--- Number of Theory jobs for host localhost.localdomain: 1 ----------------------------------------------------------

The 'Completed %' looks right.
But this job has been running (with a pause of computation of ~5 hours) for 1 day 12 hours according to BOINC Manager.

The last line in runRivet.log shows:
  Event 91600 ( 1h 7m 57s elapsed / 6m 13s left ) -> ETA: Fri Jun 06 18:44  
91600 events processed

Hmmm.
ID: 51941 · Report as offensive     Reply Quote
Profile Guy
Avatar

Send message
Joined: 9 Feb 08
Posts: 58
Credit: 1,919,248
RAC: 477
Message 51942 - Posted: 8 Jun 2025, 12:32:14 UTC
Last modified: 8 Jun 2025, 13:30:52 UTC

Well, looking at the figures, 132,860 s (elapsed time) as a percentage of 864,000 s (job duration) is 15.37%.

This might be helpful -
/var/lib/boinc/slots/2/stderr.txt
### trailing progress lines of the first run - ###
2025-06-07 19:02:25 (39258): Status Report: Job Duration: '864000.000000'
2025-06-07 19:02:25 (39258): Status Report: Elapsed Time: '90000.000000'
2025-06-07 19:02:25 (39258): Status Report: CPU Time: '88924.030000'
2025-06-07 20:41:27 (39258): Status Report: Job Duration: '864000.000000'
2025-06-07 20:41:27 (39258): Status Report: Elapsed Time: '96000.000000'
2025-06-07 20:41:27 (39258): Status Report: CPU Time: '94870.180000'
2025-06-07 20:55:39 (39258): Stopping VM.
2025-06-07 20:55:46 (39258): Successfully stopped VM.
    ### 5 hour pause ###
    ### second run start - ###
2025-06-08 02:12:31 (2771): vboxwrapper version 26210
2025-06-08 02:12:31 (2771): BOINC client version: 8.0.4
2025-06-08 02:12:34 (2771): Detected: VirtualBox VboxManage Interface (Version: 7.1.8)
    ### vbox/start-up posts omitted ###
    ### first 6 progress line of second run - ###
2025-06-08 02:12:46 (2771): Status Report: Job Duration: '864000.000000'
2025-06-08 02:12:46 (2771): Status Report: Elapsed Time: '96860.000000'
2025-06-08 02:12:46 (2771): Status Report: CPU Time: '95722.470000'
2025-06-08 03:51:48 (2771): Status Report: Job Duration: '864000.000000'
2025-06-08 03:51:48 (2771): Status Report: Elapsed Time: '102860.000000'
2025-06-08 03:51:48 (2771): Status Report: CPU Time: '101672.980000'
    ### last 3 progress lines - ###
2025-06-08 12:07:02 (2771): Status Report: Job Duration: '864000.000000'
2025-06-08 12:07:02 (2771): Status Report: Elapsed Time: '132860.000000'
2025-06-08 12:07:02 (2771): Status Report: CPU Time: '131240.800000'
ID: 51942 · Report as offensive     Reply Quote
seanr22a

Send message
Joined: 29 Nov 18
Posts: 40
Credit: 2,580,683
RAC: 732
Message 51943 - Posted: 9 Jun 2025, 11:23:27 UTC - in response to Message 51941.  

Please read the readme file at github. It explains that the time shown in the script is absolute time from when the job was created in Boinc. If you would pause the job for 2 days it would show those days too. I leave all the CPU time calculations to Boinc.
ID: 51943 · Report as offensive     Reply Quote
Profile microchip
Avatar

Send message
Joined: 27 Jun 06
Posts: 10
Credit: 3,221,582
RAC: 3,486
Message 51944 - Posted: 9 Jun 2025, 19:03:56 UTC
Last modified: 9 Jun 2025, 19:04:47 UTC

You can also use: boinccmd --get_task_summary pwscerd

I have an old script which does a similar thing, but is not restricted to LHC@Home project

Download link + pics: https://sourceforge.net/projects/boincctrl/
ID: 51944 · Report as offensive     Reply Quote
Anne Havinga

Send message
Joined: 4 Mar 20
Posts: 14
Credit: 6,280,700
RAC: 5,772
Message 51945 - Posted: 10 Jun 2025, 14:20:27 UTC - in response to Message 51937.  

@seanr22a Thank you again for this vbox script.
ID: 51945 · Report as offensive     Reply Quote
Previous · 1 · 2

Message boards : Theory Application : Made a small script to keep an eye on Theory jobs with correct % done


©2025 CERN