Message boards : Number crunching : @Markku - Estimated WU Completion
Message board moderation

To post messages, you must log in.

1 · 2 · Next

AuthorMessage
Profile The Gas Giant

Send message
Joined: 2 Sep 04
Posts: 309
Credit: 715,258
RAC: 0
Message 8784 - Posted: 21 Jul 2005, 23:35:18 UTC
Last modified: 21 Jul 2005, 23:35:52 UTC

I haven't seen a really long wu in quite a while, in fact the longest has been about 50% of the estimated wu completion time. Would it be possible to change the following lines in the wu data info to 50% of their current value (that's of course if these are the figures used by boinc to calculate the estimated wu completion times).

rsc_fpops_est = 30000000000000.000000
rsc_fpops_bound = 300000000000000.000000
rsc_memory_bound = 30000000.000000
rsc_disk_bound = 15000000.000000

This really is an easy change and would go a long way to overcoming wu caching issues for folks still on dial up or people with multiple computers at home, but without a network connection 24/7.

Live long and crunch.

Paul
(S@H1 8888)
BOINC/SAH BETA
ID: 8784 · Report as offensive     Reply Quote
John McLeod VII
Avatar

Send message
Joined: 2 Sep 04
Posts: 165
Credit: 146,925
RAC: 0
Message 8810 - Posted: 22 Jul 2005, 16:26:49 UTC

4.70+ have a correction factor that will over time get the estimates right per machine - project combination. It also reports back to the project server for each machine.

While it is simple to change the FPOPS Est (and it is indeed used to calculate the initial estimate) not every machine takes the estimate time based on the benchmarks and the FPOPS estimate.


BOINC WIKI
ID: 8810 · Report as offensive     Reply Quote
Profile The Gas Giant

Send message
Joined: 2 Sep 04
Posts: 309
Credit: 715,258
RAC: 0
Message 8824 - Posted: 22 Jul 2005, 21:37:57 UTC
Last modified: 22 Jul 2005, 21:38:42 UTC

May has well try and get it about right in the first place! For such a simple modification, it would have saved some people a bucket load of time micro-managing their boinc caches! In any case, what happens the first time you attach to a project? The estimated completion time will still be way out. This is a simple change that overcomes this and really would take a shorter amount of time to implement than it did for me to write this reply.

Live long and crunch.

Paul.
ID: 8824 · Report as offensive     Reply Quote
John McLeod VII
Avatar

Send message
Joined: 2 Sep 04
Posts: 165
Credit: 146,925
RAC: 0
Message 8833 - Posted: 23 Jul 2005, 2:14:14 UTC - in response to Message 8824.  

> May has well try and get it about right in the first place! For such a simple
> modification, it would have saved some people a bucket load of time
> micro-managing their boinc caches! In any case, what happens the first time
> you attach to a project? The estimated completion time will still be way out.
> This is a simple change that overcomes this and really would take a shorter
> amount of time to implement than it did for me to write this reply.
>
> Live long and crunch.
>
> Paul.
>
I agree, and if it is too far out, the consequences will not be fixed for a long time, however, not every machine will match the estimate perfectly (in some cases by a factor of 2 either way). The estimate should reflect a reasonable average.


BOINC WIKI
ID: 8833 · Report as offensive     Reply Quote
Profile The Gas Giant

Send message
Joined: 2 Sep 04
Posts: 309
Credit: 715,258
RAC: 0
Message 8881 - Posted: 24 Jul 2005, 4:51:29 UTC

Be nice if Markku could reply.....

And I'm yet to be shown a computer that actually takes longer than 50% of the estimated wu completion time (even though both JM7 and Markku have indicated that they exist), especially in the latest few batches of wu's.

Live long and crunch.

Paul.
ID: 8881 · Report as offensive     Reply Quote
John McLeod VII
Avatar

Send message
Joined: 2 Sep 04
Posts: 165
Credit: 146,925
RAC: 0
Message 8902 - Posted: 25 Jul 2005, 1:56:24 UTC - in response to Message 8881.  

> Be nice if Markku could reply.....
>
> And I'm yet to be shown a computer that actually takes longer than 50% of the
> estimated wu completion time (even though both JM7 and Markku have indicated
> that they exist), especially in the latest few batches of wu's.
>
> Live long and crunch.
>
> Paul.
>
I have one that I believe takes around 80% of the estimated time.


BOINC WIKI
ID: 8902 · Report as offensive     Reply Quote
Jayargh

Send message
Joined: 24 Oct 04
Posts: 79
Credit: 257,762
RAC: 0
Message 8905 - Posted: 25 Jul 2005, 3:16:21 UTC - in response to Message 8902.  
Last modified: 25 Jul 2005, 3:25:18 UTC

> > Be nice if Markku could reply.....
> >
> > And I'm yet to be shown a computer that actually takes longer than 50% of
> the
> > estimated wu completion time (even though both JM7 and Markku have
> indicated
> > that they exist), especially in the latest few batches of wu's.
> >
> > Live long and crunch.
> >
> > Paul.
> >
> I have one that I believe takes around 80% of the estimated time.
>
And yet I have a host that does an average of 23% of estimate....(hence my cache is screwed, have 1.5 days of work and Boinc setting is 10 days...Whats wrong with this picture?).... I hope they release Boinc 4.7x soon(due to Boinc actually supposed to adjust time to completion on ready to run workunits) because it doesn't look like admin will adjust soon.... Chrulle was supossed to look at this 2 months ago according to a response by Markku at same time frame :(
ID: 8905 · Report as offensive     Reply Quote
Profile The Gas Giant

Send message
Joined: 2 Sep 04
Posts: 309
Credit: 715,258
RAC: 0
Message 8907 - Posted: 25 Jul 2005, 4:48:23 UTC - in response to Message 8902.  
Last modified: 25 Jul 2005, 4:55:09 UTC

> > Be nice if Markku could reply.....
> >
> > And I'm yet to be shown a computer that actually takes longer than 50% of
> the
> > estimated wu completion time (even though both JM7 and Markku have
> indicated
> > that they exist), especially in the latest few batches of wu's.
> >
> > Live long and crunch.
> >
> > Paul.
> >
> I have one that I believe takes around 80% of the estimated time.
>
Whose estimated time John? The new estimate with BOINC V4.7x or the pure unadulterated estimate you get with the unoptimised BOINC V4.45? I really do not understand why you appear to be against the projects trying to get the estimate a little better than it currently is.

Can you give us some more information regarding the host it is on and the work unit link? [edit: I just looked at your hosts - oh my you are pushing the bounds of old and slow machines on BOINC!]

Live long and crunch.

Paul.


ID: 8907 · Report as offensive     Reply Quote
Profile Chrulle

Send message
Joined: 27 Jul 04
Posts: 182
Credit: 1,880
RAC: 0
Message 8914 - Posted: 25 Jul 2005, 10:44:45 UTC

No, what i looked at was the deadline. It goes through the database and tries to calculate a deadline that will give us the results faster.

Markku wont be answering the thread for a while. He is of on summer vacation interrailing around europe, and we only have him for a couple of hours a week anyway. His "real" job is studying back in Finland.

I will try to take a look at the WU estimate, but my advice is to get one of the new clients (4.70+).



Chrulle
Research Assistant & Ex-LHC@home developer
Niels Bohr Institute
ID: 8914 · Report as offensive     Reply Quote
Profile Fuzzy Hollynoodles
Avatar

Send message
Joined: 13 Jul 05
Posts: 51
Credit: 10,626
RAC: 0
Message 8923 - Posted: 25 Jul 2005, 22:31:47 UTC - in response to Message 8914.  
Last modified: 25 Jul 2005, 22:32:45 UTC

> No, what i looked at was the deadline. It goes through the database and tries
> to calculate a deadline that will give us the results faster.
>
> Markku wont be answering the thread for a while. He is of on summer vacation
> interrailing around europe, and we only have him for a couple of hours a week
> anyway. His "real" job is studying back in Finland.
>
> I will try to take a look at the WU estimate, but my advice is to get one of
> the new clients (4.70+).
>

I just installed the 4.72 client, and my LHC WU's are still estimated to 11.45.54 hours, which is ridiculous as I've never finished a 1,000,000 turn WU in more than about 4 hours and many times less, much less!

And these estimated times makes my client think that my Einstein WU's can't be finished within the deadline, so sometimes it override the round robin, so it crunches Einstein WU's solely! Even there are more than 2 days till deadline and they easily can be finished with a normal round robin!


"I'm trying to maintain a shred of dignity in this world" - Me
ID: 8923 · Report as offensive     Reply Quote
John McLeod VII
Avatar

Send message
Joined: 2 Sep 04
Posts: 165
Credit: 146,925
RAC: 0
Message 8924 - Posted: 25 Jul 2005, 22:56:27 UTC - in response to Message 8923.  

> > No, what i looked at was the deadline. It goes through the database and
> tries
> > to calculate a deadline that will give us the results faster.
> >
> > Markku wont be answering the thread for a while. He is of on summer
> vacation
> > interrailing around europe, and we only have him for a couple of hours a
> week
> > anyway. His "real" job is studying back in Finland.
> >
> > I will try to take a look at the WU estimate, but my advice is to get one
> of
> > the new clients (4.70+).
> >
>
> I just installed the 4.72 client, and my LHC WU's are still estimated to
> 11.45.54 hours, which is ridiculous as I've never finished a 1,000,000 turn WU
> in more than about 4 hours and many times less, much less!
>
> And these estimated times makes my client think that my Einstein WU's can't be
> finished within the deadline, so sometimes it override the round robin, so it
> crunches Einstein WU's solely! Even there are more than 2 days till deadline
> and they easily can be finished with a normal round robin!
>
The client only starts collecting the data after you install 4.72. It will move estimates down cautiously, but up very quickly.


BOINC WIKI
ID: 8924 · Report as offensive     Reply Quote
Profile The Gas Giant

Send message
Joined: 2 Sep 04
Posts: 309
Credit: 715,258
RAC: 0
Message 8934 - Posted: 26 Jul 2005, 7:10:47 UTC

And the up can go way over the original estimate even though it may have been 1 minute past the current estimate....I'll report this to the alpha mailing list. A bug might exist in the upwards movement of the estimate.

Paul.
ID: 8934 · Report as offensive     Reply Quote
Heffed

Send message
Joined: 2 Sep 04
Posts: 71
Credit: 8,657
RAC: 0
Message 8936 - Posted: 26 Jul 2005, 10:24:08 UTC - in response to Message 8934.  

If I've ever had one go over the revised estimate, it jumps to exactly the amount of time it took the longer WU.
ID: 8936 · Report as offensive     Reply Quote
Peter Cheung

Send message
Joined: 29 Sep 04
Posts: 7
Credit: 2,316,474
RAC: 2
Message 8957 - Posted: 27 Jul 2005, 2:52:31 UTC

It seems that the estimated to-completion time of the new batch of WUs (names start with wjun1D_...) is reduced.
For the previous WUs, the estimated time is 20~21 hours in my machine (which would finish the WU in about 7 hrs), but for the new ones the estimated to-completion time is about 13:45.
Yet it is still twice the actual running time.

ID: 8957 · Report as offensive     Reply Quote
Profile Chrulle

Send message
Joined: 27 Jul 04
Posts: 182
Credit: 1,880
RAC: 0
Message 8964 - Posted: 27 Jul 2005, 11:50:59 UTC

Yes, i had a chat with the scientists and they have revised the estimate a bit.

Chrulle
Research Assistant & Ex-LHC@home developer
Niels Bohr Institute
ID: 8964 · Report as offensive     Reply Quote
Profile The Gas Giant

Send message
Joined: 2 Sep 04
Posts: 309
Credit: 715,258
RAC: 0
Message 8966 - Posted: 27 Jul 2005, 12:33:31 UTC - in response to Message 8964.  

> Yes, i had a chat with the scientists and they have revised the estimate a
> bit.
>
Thanks Chrulle....much better place to start. I'll bitch about it again in 1 month when we have had a good opportunity to see how this change has affected things. But all good so far! Again thanks.

Live long and crunch.

Paul.
ID: 8966 · Report as offensive     Reply Quote
Profile Paul D. Buck

Send message
Joined: 2 Sep 04
Posts: 545
Credit: 148,912
RAC: 0
Message 8977 - Posted: 27 Jul 2005, 18:49:03 UTC

I am waiting for the next recommended release where BOINC "learns" reality. The only fly in the ointment is LHC@Home with the variable number of turns.

Though I did have a thought. why not make sixtrack1, 2, and 3 ... all of them the same binary. Then assign the WU to the appropriate exe? Or is the time project level only???

Gotta ask JM VII ...
ID: 8977 · Report as offensive     Reply Quote
Profile Chrulle

Send message
Joined: 27 Jul 04
Posts: 182
Credit: 1,880
RAC: 0
Message 8978 - Posted: 27 Jul 2005, 19:07:37 UTC

Hmm, not sure i understand.
The variable number of turns are tuned in the input files, and the time estimate(well, actually the fops estimate) is based on the number of turns we ask to be done.

Chrulle
Research Assistant & Ex-LHC@home developer
Niels Bohr Institute
ID: 8978 · Report as offensive     Reply Quote
Gaspode the UnDressed

Send message
Joined: 1 Sep 04
Posts: 506
Credit: 118,619
RAC: 0
Message 8979 - Posted: 27 Jul 2005, 19:47:56 UTC - in response to Message 8977.  

> I am waiting for the next recommended release where BOINC "learns" reality.
> The only fly in the ointment is LHC@Home with the variable number of turns.
>

The picture is more involved than this, given that any unit potentially can terminate in a few seconds or minutes if the beam is unstable. LHC@Home is not the only project with this sort of variable execution. MDIV units run by a closed Alpha project that I know Paul is aware of can vary according the the requirements of the input files. The project's home page suggests that time estimates may be 'wildly inaccurate'. What's worse, the MDIV units neither update their progress meters, nor write checkpoint data. I can't see how BOINC 4.7+ can learn anything effective from either SixTrack or MDIV


Gaspode the UnDressed
http://www.littlevale.co.uk
ID: 8979 · Report as offensive     Reply Quote
Profile Paul D. Buck

Send message
Joined: 2 Sep 04
Posts: 545
Credit: 148,912
RAC: 0
Message 8982 - Posted: 28 Jul 2005, 1:22:11 UTC

Yeah, I am aware of the "sudden death" problem. But, I am just ruminating that over time a 10K job takes say 10 min ... who cares the accuracy for that. But, I don't want the 10K jobs to pull down a good estimate for the 100K an 1M models.

Again, as these things go, in the LONG run it is all who cares. And I am fully into all 5 of my projects so, I don't have the "usual" problems.

Chrulle,

I was just thinking "out loud" ... pros and cons ...

ID: 8982 · Report as offensive     Reply Quote
1 · 2 · Next

Message boards : Number crunching : @Markku - Estimated WU Completion


©2024 CERN