Message boards : Number crunching : web preferences - queue tasks but run only 1
Message board moderation

To post messages, you must log in.

AuthorMessage
greg_be

Send message
Joined: 28 Dec 08
Posts: 318
Credit: 4,148,677
RAC: 2,010
Message 45196 - Posted: 12 Aug 2021, 21:35:24 UTC

I would like to queue like 2 extra tasks on top of what I am running.
BUT...I only can run 1 task on 4 cores without ATLAS choking.

IF I set my preferences to 3 tasks and 4 cores (the cores is already set, tasks = 1) will BOINC/ATLAS run only one task and queue 2 others?

In the past I tried to increase tasks, but then BOINC would try to run 2 and ATLAS would choke and cough up errors.

IF this can not be done via web preferences, then will a app_config script take care of that?
ID: 45196 · Report as offensive     Reply Quote
Toby Broom
Volunteer moderator

Send message
Joined: 27 Sep 08
Posts: 798
Credit: 644,721,856
RAC: 234,355
Message 45197 - Posted: 12 Aug 2021, 22:03:10 UTC - in response to Message 45196.  

How about using 4 and 4 which choked before but with

<name>ATLAS</name>
<max_concurrent>1</max_concurrent>
</app>

in your app config?
ID: 45197 · Report as offensive     Reply Quote
greg_be

Send message
Joined: 28 Dec 08
Posts: 318
Credit: 4,148,677
RAC: 2,010
Message 45202 - Posted: 13 Aug 2021, 23:23:51 UTC - in response to Message 45197.  

How about using 4 and 4 which choked before but with

<name>ATLAS</name>
<max_concurrent>1</max_concurrent>
</app>

in your app config?


I'll have to search for my old app_config file and then try that.
It's 1am saturday here in Europe. I'll see if I have time tonight to dig the file up.
ID: 45202 · Report as offensive     Reply Quote
greg_be

Send message
Joined: 28 Dec 08
Posts: 318
Credit: 4,148,677
RAC: 2,010
Message 45211 - Posted: 16 Aug 2021, 17:45:13 UTC - in response to Message 45202.  
Last modified: 16 Aug 2021, 17:52:22 UTC

How about using 4 and 4 which choked before but with

<name>ATLAS</name>
<max_concurrent>1</max_concurrent>
</app>

in your app config?


I'll have to search for my old app_config file and then try that.
It's 1am saturday here in Europe. I'll see if I have time tonight to dig the file up.



Ahh..just got a chance to read this again.
This is not what I am looking for from what I am reading in BOINC.

What I am looking for is to STORE say 2 tasks in addition to the running task and runnning only 1 task on 4 cores. I think my memory is to low to run 2 x 4 cores and my other projects all at the same time. It's on my list to upgrade my memory cards.[/b]
ID: 45211 · Report as offensive     Reply Quote
greg_be

Send message
Joined: 28 Dec 08
Posts: 318
Credit: 4,148,677
RAC: 2,010
Message 45212 - Posted: 16 Aug 2021, 18:04:15 UTC - in response to Message 45211.  

I think more like this, in a file called config_aux.xml
[url]https://boinc.berkeley.edu/trac/wiki/ProjectOptions#Joblimitsadvanced[/url}
<max_jobs_in_progress>
<project>
<total_limit>
<jobs>N</jobs>
</total_limit>
[ <gpu_limit> ]
<jobs>N</jobs>
[ <per_proc/> ]
if set, limit is per processor
[ <cpu_limit> ]
...
</project>
<app>
<app_name>name</app_name>
[ <total_limit> ... ]
[ <cpu_limit> ... ]
[ <gpu_limit> ... ]
</app>
...
</max_jobs_in_progress>
</config>[/url]
ID: 45212 · Report as offensive     Reply Quote
computezrmle
Volunteer moderator
Volunteer developer
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 15 Jun 08
Posts: 2386
Credit: 222,922,417
RAC: 138,013
Message 45213 - Posted: 16 Aug 2021, 18:21:07 UTC - in response to Message 45212.  

I think more like this, in a file called config_aux.xml...

That's a file used by the BOINC server.
You are running a BOINC client.
ID: 45213 · Report as offensive     Reply Quote
greg_be

Send message
Joined: 28 Dec 08
Posts: 318
Credit: 4,148,677
RAC: 2,010
Message 45214 - Posted: 16 Aug 2021, 19:16:14 UTC - in response to Message 45213.  
Last modified: 16 Aug 2021, 19:30:13 UTC

I think more like this, in a file called config_aux.xml...

That's a file used by the BOINC server.
You are running a BOINC client.


ok...I am just quick reading the material..obviously I missed that.

But where in the world or is there even a set of commands that does what I am looking for?
That's what I am trying to figure out and failing at. I'm searching and searching and I can't find any example of what I want to do.

Cores 4, tasks 1, store 2 tasks in reserve.
That is what I want to do, period.
I also want do just spread out the projects to run consistent across the cores.
4 for ATLAS, 1 or 2 for GPU and the rest split with Rosetta (out of work now), WCG (which is hogging all the non GPU cores at the moment) and Milkyway.

I don't see any option for this kind of storage in the web preferences. If I change the jobs, then it wants to run more jobs at the same time. That is not what I want.

Note: I found a post that says this is not possible on my level. Shame.
Then maybe ATLAS project could add the code I found as a way to store extra tasks? Not that they would...but it would be a nice thing.
ID: 45214 · Report as offensive     Reply Quote
computezrmle
Volunteer moderator
Volunteer developer
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 15 Jun 08
Posts: 2386
Credit: 222,922,417
RAC: 138,013
Message 45216 - Posted: 16 Aug 2021, 19:48:28 UTC - in response to Message 45214.  

BOINC never worked that way.
The client always tries to use all available resources except if a limit is set by the user.

Just a simple example with project A and project B, both CPU projects.
The project settings on the servers A an B allow you to set a relative weight (default 100).

Your client ensures that long term the runtime usage 1) is given 100/100 to each project.
In case project A doesn't send out work for a while your computer will run only project B.
Once project A starts sending work again your client will automatically prefer project A for a while.

The local work buffer is filled based on expected runtime, not based on number of tasks.


If you plan to spend a fix number of cores, e.g. 4, to only 1 (sub)project (ATLAS) the best solution would be to set up an additional client that is configured to run nothing but ATLAS.


1) simply spoken.
In reality it's much more complex.
ID: 45216 · Report as offensive     Reply Quote
greg_be

Send message
Joined: 28 Dec 08
Posts: 318
Credit: 4,148,677
RAC: 2,010
Message 45217 - Posted: 16 Aug 2021, 20:08:45 UTC - in response to Message 45216.  
Last modified: 16 Aug 2021, 20:11:45 UTC

BOINC never worked that way.
The client always tries to use all available resources except if a limit is set by the user.

Just a simple example with project A and project B, both CPU projects.
The project settings on the servers A an B allow you to set a relative weight (default 100).

Your client ensures that long term the runtime usage 1) is given 100/100 to each project.
In case project A doesn't send out work for a while your computer will run only project B.
Once project A starts sending work again your client will automatically prefer project A for a while.

The local work buffer is filled based on expected runtime, not based on number of tasks.


If you plan to spend a fix number of cores, e.g. 4, to only 1 (sub)project (ATLAS) the best solution would be to set up an additional client that is configured to run nothing but ATLAS.


1) simply spoken.
In reality it's much more complex.



UGGGH!!!! (Frustration) (Beating head on desk)
I give up! If its more complex then I just face the reality of the inability of BOINC to work in a way that fits what I want to do.
So then its 4 cores on a single task and roll the dice on when ATLAS will be able to run again. Crazy.
I moved LHC to 200 and WCG to 80 to try and push more towards ATLAS.
ID: 45217 · Report as offensive     Reply Quote
wolfman1360

Send message
Joined: 17 Feb 17
Posts: 42
Credit: 2,589,736
RAC: 0
Message 45223 - Posted: 18 Aug 2021, 14:45:28 UTC - in response to Message 45217.  

BOINC never worked that way.
The client always tries to use all available resources except if a limit is set by the user.

Just a simple example with project A and project B, both CPU projects.
The project settings on the servers A an B allow you to set a relative weight (default 100).

Your client ensures that long term the runtime usage 1) is given 100/100 to each project.
In case project A doesn't send out work for a while your computer will run only project B.
Once project A starts sending work again your client will automatically prefer project A for a while.

The local work buffer is filled based on expected runtime, not based on number of tasks.


If you plan to spend a fix number of cores, e.g. 4, to only 1 (sub)project (ATLAS) the best solution would be to set up an additional client that is configured to run nothing but ATLAS.


1) simply spoken.
In reality it's much more complex.



UGGGH!!!! (Frustration) (Beating head on desk)
I give up! If its more complex then I just face the reality of the inability of BOINC to work in a way that fits what I want to do.
So then its 4 cores on a single task and roll the dice on when ATLAS will be able to run again. Crazy.
I moved LHC to 200 and WCG to 80 to try and push more towards ATLAS.

Best of luck.
That resource setting, by the way, will take about 3 weeks to learn what you want.
I'm still figuring out the best configuration for this project - it by far requires the most messing about with various settings to optimize runtimes vs. available memory and available cores. I think I have it now.
ID: 45223 · Report as offensive     Reply Quote
Profile Ray Murray
Volunteer moderator
Avatar

Send message
Joined: 29 Sep 04
Posts: 281
Credit: 11,859,285
RAC: 1
Message 45224 - Posted: 18 Aug 2021, 20:56:43 UTC

I've not done Atlas jobs for a long time as my ageing hosts don't have the physical resources to run them successfully so I can't remember their runtime.
You can speed up the Boinc resource share synching with a tweak to the cc_config.xml.
The default rec_half_life_days is 10 days so Boinc uses that long to do its averaging.
If you change that to, say, 4, Boinc will synch your resources shares quicker but setting that too low will confuse it if there are long-running tasks

Toby's max_concurrent will only let it run 1 at a time but it should still queue another 1 or more when it thinks it needs to.
ID: 45224 · Report as offensive     Reply Quote

Message boards : Number crunching : web preferences - queue tasks but run only 1


©2024 CERN