Message boards : Number crunching : shorter deadline please!
Message board moderation

To post messages, you must log in.

AuthorMessage
Profile littleBouncer
Avatar

Send message
Joined: 23 Oct 04
Posts: 358
Credit: 1,439,205
RAC: 0
Message 7920 - Posted: 2 Jun 2005, 19:17:13 UTC
Last modified: 2 Jun 2005, 19:31:35 UTC

Please set the deadline to 7 day as PAH or EAH does..
If not the first LHC-WU will be crunched in 7 days with CC 4.43 and higher ; because of the deadline-handling of the client the faster deadlineWU is crunched first, or you have to suspend all other project.

Why the rules are not:
First: obey the resource shares
and second (!) : is there a deadline-conflict

only a suggestion.

Or how I can change the following restriction of the client:?
New CPU scheduler policy: earliest deadline first

greetz littleBouncer

ID: 7920 · Report as offensive     Reply Quote
Grenadier
Avatar

Send message
Joined: 2 Sep 04
Posts: 39
Credit: 441,128
RAC: 0
Message 7922 - Posted: 2 Jun 2005, 20:15:40 UTC - in response to Message 7920.  
Last modified: 2 Jun 2005, 20:18:45 UTC

> Please set the deadline to 7 day as PAH or EAH does..

This will make your problems worse.

> If not the first LHC-WU will be crunched in 7 days with CC 4.43 and higher ;
> because of the deadline-handling of the client the faster deadlineWU is
> crunched first, or you have to suspend all other project.

No, it will be crunched when your long-term debt on other projects is lower than the debt for LHC. If you're stuck in deadline mode, it's for a reason. You likely have too much work queued up for your system to handle, and/or a work unit that is going to miss it's deadline. Stop downloading so much work, and you'll have fewer problems.

> Why the rules are not:
> First: obey the resource shares
> and second (!) : is there a deadline-conflict

Those are the rules. But the resource shares are being obeyed long-term, not on an hourly basis. It's keeping track of what time each project used, and then playing catch-up later for the ones that didn't get their fair share. But this goes out the window in deadline mode.

> Or how I can change the following restriction of the client:?
> New CPU scheduler policy: earliest deadline first

This happens when you have too much work in your queue, or when one WU is in danger of missing its deadline. Reduce your queue size to avoid this in the future. If you're running more than one project, change it to 1 day or less. (0.5 is often suggested, especially if running Einstein.)

For now, just let it catch up and clear the queue of these late WU's.

-Gren
ID: 7922 · Report as offensive     Reply Quote
Profile littleBouncer
Avatar

Send message
Joined: 23 Oct 04
Posts: 358
Credit: 1,439,205
RAC: 0
Message 7923 - Posted: 2 Jun 2005, 20:49:09 UTC - in response to Message 7922.  
Last modified: 2 Jun 2005, 21:23:16 UTC

@ Grenadier
Thanks for your informativ reply!!!

There was never a deadline-problem with my setting with other clients on this box, even when I started with the CC 4.44, 11 days ago, it worked smoothly, that mean : when I reported a WU , I got immediately a new from that project. Since 4 days it has changed to this 'dealine-restriction-behaviour'. Two times I crunched to the point where nomore WU's were in the cache.
After downloading it begins at new with the earliest deadline.

What I want is that the client crunches 1 hour this project than change to the next and so on , as I have set in the prefs.
But it works now: first all WU's from PAH (5-6), then all from EAH(2-3), then SAH (3-4) and eventually LHC in alternation with SAH ...

greetz littleBouncer
I will try to change less than 1 day. But that's not the solution to get enough work from LHC (when there are some...)
[EDIT] If the cache is lower then 2 days the scheduler policy change to: highest debt first (what I prefere!)
[/EDIT]
ID: 7923 · Report as offensive     Reply Quote
Jayargh

Send message
Joined: 24 Oct 04
Posts: 79
Credit: 257,762
RAC: 0
Message 7924 - Posted: 2 Jun 2005, 21:22:35 UTC

Grenadier & littlebouncer you are both right. But that is no reason to change a deadline littlebouncer because YOU are having a problem-Grenadier is right too much work but you are right lb not a good way to get LHC work... so what do we do? I say you need to tweak your resource settings lb when LHC has work:)Thats what I do and if anyone has problems with thier scheduler not asking for work when you have none is to suspend all other projects/workunits and that project will give you work.
ID: 7924 · Report as offensive     Reply Quote
Profile Juerschi

Send message
Joined: 26 Oct 04
Posts: 12
Credit: 8,909
RAC: 0
Message 7925 - Posted: 2 Jun 2005, 21:40:47 UTC
Last modified: 2 Jun 2005, 21:41:06 UTC

This is from the Seti messageboard an posted from JM7:

There are three causes for entering earliest deadline mode:

1) Deadline within 24 hours.
2) Deadline withing 2 * duration between connections (connect every X).
3) If you order the WUs by deadline, if the sum of the remainig processing time exceeds 80% of the time to the deadline at any point in the chain.

I hope this will make things a little bit clearer


ID: 7925 · Report as offensive     Reply Quote
Profile The Gas Giant

Send message
Joined: 2 Sep 04
Posts: 309
Credit: 715,258
RAC: 0
Message 7927 - Posted: 2 Jun 2005, 22:05:50 UTC
Last modified: 2 Jun 2005, 22:10:59 UTC

Point #2 causes all the problems. I have my connect every preference set at 5 days. This used to ensure sufficient work was kept in the cache to last most outages on all the projects. Because of the 2 * connect every = 10 days and deadlines are 7 days with Pred and Einstein (but didn't they use to be 14?) when ever I download work they work gets crunched in deadline mode. Then the LT debt builds up for the other projects so when BOINC runs out of work the just completed project it doesn't down load new work.

This basically means that if you want BOINC to not operate in deadline mode then you need to set this preference to less than a quarter of the earliest deadline of all your projects. Which in the Pred and Einstein cases means 1.75 days and since you can't do decimals then 1 day. This is a ridiculous situation.

I have suggested that the 2 * duration is too tough and it should really only be 1*, but this has fallen on deaf ears and blind eyes. I believe that if a deadline is within the connect every preference then only then should BOINC worry about hitting deadline mode (and even that is a pretty strong reaction). In reality BOINC should only hit deadline mode when the sum of estimated completion times say 1.25 plus todays date/time is greater than the deadline time of the earliest deadline.

Live long and crunch!

Paul
(S@H1 8888)
BOINC/SAH BETA
ID: 7927 · Report as offensive     Reply Quote
Profile The Gas Giant

Send message
Joined: 2 Sep 04
Posts: 309
Credit: 715,258
RAC: 0
Message 7930 - Posted: 2 Jun 2005, 23:31:03 UTC

The new expletive "foinc" (freakin boinc).

Ah, it's foinc'd!
Oh don't foinc with me!
Go and foinc yourself!
Oh your computer is foinc'd (this one happens to me all thetime).
Foinc is as foinc does!
Are you foinc'n with me?
Don't foinc with it, you'll foinc it!

Live long and crunch!

Paul.
ID: 7930 · Report as offensive     Reply Quote
Jayargh

Send message
Joined: 24 Oct 04
Posts: 79
Credit: 257,762
RAC: 0
Message 7931 - Posted: 3 Jun 2005, 1:15:58 UTC
Last modified: 3 Jun 2005, 1:28:12 UTC

@Ziran Boinc has now made it personal and must I say challeging! My best option now is to Suspend! Suspend! Suspend! with a little bit of resource tweaking.So much for those who give thier calculations and formulas dt I have observed my 3.6gigHT down to 1 CPDN work unit with 2 other projects AVAILABLE! and still would dl no work til I suspended CPDN.... me thinks many quirks abound and they have gone from 1 extreme to another!!! Would just be nice to be able to do what I want w/o constantly tending to the farm dt Boinc issues :( It seems there are Demons in the Daemon!
ID: 7931 · Report as offensive     Reply Quote
genes
Avatar

Send message
Joined: 29 Sep 04
Posts: 25
Credit: 77,748
RAC: 0
Message 7932 - Posted: 3 Jun 2005, 2:46:20 UTC

@jrenkar - this is a known bug with duals and HT's. Hopefully they will fix it in the next version. What I've done is tweaked my resource share and edited the xml to get two CPDN's on each dual or HT box. That way, you never run out of work. (Well, maybe some day...) ;)

@GG - yes, you can do decimals without foincing it up! I ran with 0.5 for a long time.

Happy crunching!

ID: 7932 · Report as offensive     Reply Quote
Grenadier
Avatar

Send message
Joined: 2 Sep 04
Posts: 39
Credit: 441,128
RAC: 0
Message 7933 - Posted: 3 Jun 2005, 4:10:58 UTC

Second the vote for using decimal portions of days. The Einstein folks were strongly suggesting 0.5 for quite a while, and probably still are. As long as you are not on dialup, having a really full queue becomes counter productive. It screws up the scheduler, and the chances of all projects being down simultaneously is pretty low.

I was running 0.5 for quite a while, but just took some of my slower machines out of the farm, and got it back to 1.0. At 1.0, there are still plenty of WU's available, and I never enter deadline mode, since the smaller backlog insures that that situation does not arise.

I'd suggest starting at 0.5, and nudging it up if you see yourself running low and avoiding deadline mode.

Of course, if you're on dialup, this advice may go out the window. I'd still suggest a smaller queue, with more frequent connections, if that is an option for you.
ID: 7933 · Report as offensive     Reply Quote
Profile The Gas Giant

Send message
Joined: 2 Sep 04
Posts: 309
Credit: 715,258
RAC: 0
Message 7934 - Posted: 3 Jun 2005, 4:51:29 UTC - in response to Message 7932.  

> @GG - yes, you can do decimals without foincing it up! I ran with 0.5 for a
> long time.
>
Doh!....sometimes I wish I would think about things a little before I post them. I used to run foinc with my cache setting (as it was back then) set at 0.1 days.

ID: 7934 · Report as offensive     Reply Quote
Profile littleBouncer
Avatar

Send message
Joined: 23 Oct 04
Posts: 358
Credit: 1,439,205
RAC: 0
Message 7935 - Posted: 3 Jun 2005, 6:59:19 UTC

Thank you very much for all replies. They helped me a lot to understand the new CC's.

greetz from sunny Switzerland
littleBouncer
ID: 7935 · Report as offensive     Reply Quote
Heffed

Send message
Joined: 2 Sep 04
Posts: 71
Credit: 8,657
RAC: 0
Message 7937 - Posted: 3 Jun 2005, 16:23:22 UTC - in response to Message 7928.  

> Yes point #2 causes problems, but is necessary if your on dial up. So my
> suggestion is that cause #2 only would apply if you have specified that you
> are on dial up in your general preferences. I think that we will have to make
> the client behave in different ways if you are on dial up or not.

The current settings on the preferences page are not that accurate in interpreting a dial up connection.

The problem is the BOINC has never been great at respecting these settings, so I (and I'm sure I'm not the only one) actually have my settings set the same way an always on connection would be. I control my connection by disabling network access from the GUI.

What is needed is an additional setting. "I connect with a modem" or something like that. This way the CC can easily tell whether or not the user is on dial up.
ID: 7937 · Report as offensive     Reply Quote
Profile Paul D. Buck

Send message
Joined: 2 Sep 04
Posts: 545
Credit: 148,912
RAC: 0
Message 7947 - Posted: 3 Jun 2005, 21:36:00 UTC - in response to Message 7937.  

> > Yes point #2 causes problems, but is necessary if your on dial up. So my
> > suggestion is that cause #2 only would apply if you have specified that
> you
> > are on dial up in your general preferences. I think that we will have to
> make
> > the client behave in different ways if you are on dial up or not.
>
> The current settings on the preferences page are not that accurate in
> interpreting a dial up connection.
>
> The problem is the BOINC has never been great at respecting these settings, so
> I (and I'm sure I'm not the only one) actually have my settings set the same
> way an always on connection would be. I control my connection by disabling
> network access from the GUI.
>
> What is needed is an additional setting. "I connect with a modem" or something
> like that. This way the CC can easily tell whether or not the user is on dial
> up.

I know JM VII has asked for some new parameters, but to my knowledge he has not yet gotten permission for them.

>
ID: 7947 · Report as offensive     Reply Quote
bass4lhc

Send message
Joined: 28 Sep 04
Posts: 43
Credit: 249,962
RAC: 0
Message 7949 - Posted: 3 Jun 2005, 22:48:49 UTC - in response to Message 7920.  
Last modified: 3 Jun 2005, 23:54:58 UTC

>
> Why the rules are not:
> First: obey the resource shares
> and second (!) : is there a deadline-conflict
>
> only a suggestion.
>
and a perfect suggestion.

we are donating our cpu time, we have the choice how to share this time if we run more boinc projects.

i run seti and lhc. i have work for both but only seti runs. there is in no way a reason for the panic mode in the client software, the seti wu's do not have to be returned in about 9 days.
i have set the client software with a preference for lhc. still i run much more seti then lhc due to the nature of this project. so long term or short term dept should make no difference, if i have lhc wu's they should run.

the size of the cache we donate to these project should never make a difference to our choices. we donate the space on our hd's!

switching between projects is on 1 hour, as recommended. but does not happen.

so we have all these choices, but nobody listens to them.

(and that's what happens if we are seen as users. this does not happen to donators.)

boinc should listen to littleBouncer's suggestion. it is to say the least "user"friendly.

by the way, with the introduction of boinc we were promissed we would never get wu's which our computers could not get ready on time.

i can suspend projects to reach my goal, if we all would do this boinc will be disrupted maybe?

bass4lhc/bass4seti
ID: 7949 · Report as offensive     Reply Quote
Profile JigPu

Send message
Joined: 1 Sep 04
Posts: 26
Credit: 600,998
RAC: 0
Message 7982 - Posted: 6 Jun 2005, 8:01:54 UTC - in response to Message 7949.  
Last modified: 6 Jun 2005, 8:03:10 UTC

> i run seti and lhc. i have work for both but only seti runs. there is in no
> way a reason for the panic mode in the client software, the seti wu's do not
> have to be returned in about 9 days.
>
If your cache is set too large (which only takes a setting greater than 2 or 3 days =(), BOINC will panic due to possible deadline issues. I think it's WAY too conservative about it's panicking, but the idea (prevent as many WUs from deadling when the cache is too large by running the one with the earliest deadline first) is sound provided it were implemented better and panicked a lot less.

> i have set the client software with a preference for lhc. still i run much
> more seti then lhc due to the nature of this project. so long term or short
> term dept should make no difference, if i have lhc wu's they should run.
>
They should indeed, and will once it comes to "their turn". Once the work is on your computer, it will eventually be crunched. Because of other projects combined with the panic mode it may be sitting at the very end of the queue, but they'll get their chance. If BOINC wern't in panic mode, it would be using the round-robin scheduling (which would finish your LHC fairly quickly) that 4.2 and earlier versions used.

> the size of the cache we donate to these project should never make a
> difference to our choices. we donate the space on our hd's!
>
It should indeed not, and in reality it does not. I could set a huge cache, and BOINC would let me. However, when deadlines are taken into account there is a practical limit on how large we should size our cache. Obviously downloading 100 days of work won't work for projects with 2 week deadlines... However, downloading 7 days of work (with a 2 week deadline) on a 24/7 cruncher shouldn't matter at all (even though with the current client it unfortunatly does =(... THIS is what I'd like to see fixed)

> switching between projects is on 1 hour, as recommended. but does not happen.
>
I haven't paid enough attention to my BOINC client to know (I'm set to switch every 30 minutes), but if it's not honoring it, that's definatly a problem that should be brought up...

> so we have all these choices, but nobody listens to them.
>
> (and that's what happens if we are seen as users. this does not happen to
> donators.)
>
> boinc should listen to littleBouncer's suggestion. it is to say the least
> "user"friendly.
>
Can't agree more =) Hopefully with BOINC being open source, this will change as more of the dontators modify the client, making it more donator-centric.

Puffy
ID: 7982 · Report as offensive     Reply Quote
Heffed

Send message
Joined: 2 Sep 04
Posts: 71
Credit: 8,657
RAC: 0
Message 8052 - Posted: 11 Jun 2005, 12:54:57 UTC - in response to Message 7982.  

> I haven't paid enough attention to my BOINC client to know (I'm set to switch
> every 30 minutes), but if it's not honoring it, that's definatly a problem
> that should be brought up...

It won't in earliest deadline mode.
ID: 8052 · Report as offensive     Reply Quote
John McLeod VII
Avatar

Send message
Joined: 2 Sep 04
Posts: 165
Credit: 146,925
RAC: 0
Message 8053 - Posted: 11 Jun 2005, 18:43:50 UTC - in response to Message 8052.  

> > I haven't paid enough attention to my BOINC client to know (I'm set to
> switch
> > every 30 minutes), but if it's not honoring it, that's definatly a
> problem
> > that should be brought up...
>
> It won't in earliest deadline mode.
>
It will honor resource shares over the long term even if the CPU enters EDF. However, the whole point of EDF is to get work done and reported by the deadline, and this means that over the short term, resource shares cannot be honored.


BOINC WIKI
ID: 8053 · Report as offensive     Reply Quote

Message boards : Number crunching : shorter deadline please!


©2022 CERN