Message boards : Number crunching : Super user ?????
Message board moderation

To post messages, you must log in.

AuthorMessage
Dimmerjas

Send message
Joined: 16 Jul 05
Posts: 4
Credit: 9,801
RAC: 0
Message 10195 - Posted: 16 Sep 2005, 8:07:19 UTC

Take a look at this:
http://lhcathome.cern.ch/results.php?hostid=28337

He won't run out of work. But will he be abel to make it in time?
A Super User or someone who delay the completion of job's as fast as possible instead?

Dimmerjas
ID: 10195 · Report as offensive     Reply Quote
Profile adrianxw

Send message
Joined: 29 Sep 04
Posts: 187
Credit: 705,487
RAC: 0
Message 10197 - Posted: 16 Sep 2005, 8:12:37 UTC

I have seen this happen before, indeed, it happened to me once at Predictor. It was no deliberate act, some quirk of the fates got my core client and the server into a weird state and I was "sent" about 100 wu's, none of which arrived here, and no communication errors were flagged.

Wave upon wave of demented avengers march cheerfully out of obscurity into the dream.
ID: 10197 · Report as offensive     Reply Quote
Dimmerjas

Send message
Joined: 16 Jul 05
Posts: 4
Credit: 9,801
RAC: 0
Message 10199 - Posted: 16 Sep 2005, 8:43:01 UTC

Well! - If this can happen for other users, and for other project's running using Boinc, then maby Boinc need some kind of ajustment? Like a limit for number of jobs sent to one user for each project using Boinc.

Dimmerjas
ID: 10199 · Report as offensive     Reply Quote
Profile Thierry Van Driessche
Avatar

Send message
Joined: 1 Sep 04
Posts: 157
Credit: 82,604
RAC: 0
Message 10201 - Posted: 16 Sep 2005, 9:19:33 UTC - in response to Message 10199.  
Last modified: 16 Sep 2005, 9:21:07 UTC

<blockquote>Well! - If this can happen for other users, and for other project's running using Boinc, then maby Boinc need some kind of ajustment? Like a limit for number of jobs sent to one user for each project using Boinc.

Dimmerjas</blockquote>
There is a limit that is used. It is the daily quota which is per CPU and per day. This is a project depending number. For example here it is set at 100, for E@H it is set at 8.
Best greetings from Belgium
Thierry
ID: 10201 · Report as offensive     Reply Quote
J D K

Send message
Joined: 27 Aug 05
Posts: 50
Credit: 24,055
RAC: 0
Message 10204 - Posted: 16 Sep 2005, 14:55:29 UTC
Last modified: 16 Sep 2005, 15:09:56 UTC

Looked him over good, last contact 7 sept, processes WU in about 20,000 sec using GenuineIntel Intel(R) Pentium(R) 4 CPU 1.80GHz, has on hand 538 WUs, so I feel he will not be able to finish them unless time stands still lol.......
ID: 10204 · Report as offensive     Reply Quote
Dimmerjas

Send message
Joined: 16 Jul 05
Posts: 4
Credit: 9,801
RAC: 0
Message 10207 - Posted: 16 Sep 2005, 15:37:05 UTC - in response to Message 10204.  

<blockquote>Looked him over good, last contact 7 sept, processes WU in about 20,000 sec using GenuineIntel Intel(R) Pentium(R) 4 CPU 1.80GHz, has on hand 538 WUs, so I feel he will not be able to finish them unless time stands still lol....... </blockquote>

That's my point! Why keep on sending work to users with that many Wu's in wait. When they won't be abel to return them in time. Due to a slow CPU. Or when the CPU is not used on Boinc-project at all. As in this case it seems. 540 jobs in total. 2 jobs returned. 538 in wait. The LHC-limit of 8 Wu's a day is in use here I may say. And the only thing the Boinc Manager is used for?

Dimmerjas

ID: 10207 · Report as offensive     Reply Quote
Profile Paul D. Buck

Send message
Joined: 2 Sep 04
Posts: 545
Credit: 148,912
RAC: 0
Message 10209 - Posted: 16 Sep 2005, 15:55:24 UTC

Just a couple of points...

The "ghost" work unit issue problem has been addressed to some extent with the updated client and server side code. But, if you are not using both, well, this won't be prevented.

Many projects don't like to update the server side code that often because it is not always an easy thing to do. CPDN just did an update and they are still finding things that are not working well ... a week after the fact ... with resource limits of people, it is hard to say go do this too ...

Even with the latest fixes, it is not at all certain that there will not be more and different problems down the line that allow this to happen in the future.
ID: 10209 · Report as offensive     Reply Quote
Profile adrianxw

Send message
Joined: 29 Sep 04
Posts: 187
Credit: 705,487
RAC: 0
Message 10213 - Posted: 16 Sep 2005, 21:37:08 UTC

And there are crunchers that will not load the later core clients because they object profoundly to the way the development has gone.

Wave upon wave of demented avengers march cheerfully out of obscurity into the dream.
ID: 10213 · Report as offensive     Reply Quote
Profile Paul D. Buck

Send message
Joined: 2 Sep 04
Posts: 545
Credit: 148,912
RAC: 0
Message 10220 - Posted: 17 Sep 2005, 3:58:59 UTC - in response to Message 10213.  

<blockquote>And there are crunchers that will not load the later core clients because they object profoundly to the way the development has gone.</blockquote>
For those in that case, within a year or so, they will be leaving, or be severely restricted as to the project. MOST LIKELY.

At some point in time, the projects are going to raise the floor past 4.3x to a min of 4.4x or later. I hate to see anyone leave ... but, I don't see how to avoid it.

I suppose if we could figure out what the main objection is, this could be changed, but then again, maybe not ...
ID: 10220 · Report as offensive     Reply Quote
Profile adrianxw

Send message
Joined: 29 Sep 04
Posts: 187
Credit: 705,487
RAC: 0
Message 10223 - Posted: 17 Sep 2005, 8:11:54 UTC
Last modified: 17 Sep 2005, 8:13:11 UTC

It has been said many times. 4.25 gave people a lot more control over the way their machines were working, the later clients introduced a series of measures designed to help the "load and forget" crunchers, (quite rightly so), but took away the control possibilities that the earlier cores gave to the people who understand BOINC, their projects, and their computers likely availibility.

I have absolutely no objections to the core client downloading and installing a "load and forget it" set of defaults, but I want to be able to switch them off, or the ability to modify them.

What is so difficult to understand about that? There are informed and uninformed crunchers, the later core clients treat everyone as dumb.

I know what I need and when I need it to keep my computer busy and productive without missing deadlines. I know when I am going to have internet access and when not. I know when I am going to be away and when not. I know when I am planning hardware or software upgrades and when not. BOINC will never know this.

Wave upon wave of demented avengers march cheerfully out of obscurity into the dream.
ID: 10223 · Report as offensive     Reply Quote
GX10120
Avatar

Send message
Joined: 13 Sep 05
Posts: 14
Credit: 1,606
RAC: 0
Message 10224 - Posted: 17 Sep 2005, 9:10:12 UTC

lol that's crazy. and i dont see a point of download tons of works and unable finish in time. best for all is to have thier modem connect to internet all time and have thier PC run for 24hours. the client would download new work, if the recent running work is about 30-50% done. for overheated CPU user, just use Thread Master. a nice program wont decrease the process speed and nicely reduced CPU speed.

Yet, thread master is for user who had enough knowledge of window registry.
ID: 10224 · Report as offensive     Reply Quote
Profile adrianxw

Send message
Joined: 29 Sep 04
Posts: 187
Credit: 705,487
RAC: 0
Message 10241 - Posted: 17 Sep 2005, 16:54:02 UTC
Last modified: 17 Sep 2005, 16:58:42 UTC

>>> i dont see a point of download tons of works and unable finish in time.

Good grief, read what I said won't you? Here for example...

>>> without missing deadlines.

I DO have my system permenantly connected, I do run my systems 24/7, I do download one unit at a time. I do have my systems adequately cooled, so no, I do not need some third party throttling software.

What I do not want, is when I need to take a server offline for a day, to find BOINC fiddling around with what it will and won't download based on it's best guess as to what I am up too - I know what I am up too.

If you're going to call me crazy, come back with something better then that, huh? As it stands, it looks like you just did not read or understand what I said and did a typical knee-JERK response.

@ Paul,

I suspect when forced to use an LTD client, what most of us will do is not drop out of BOINC, we will stop it from time to time and edit the .xml files to get it to do what we want. A few dozen lines of code will make a little program to wipe LTD.

The point really is, why should we have too?

Wave upon wave of demented avengers march cheerfully out of obscurity into the dream.
ID: 10241 · Report as offensive     Reply Quote
GX10120
Avatar

Send message
Joined: 13 Sep 05
Posts: 14
Credit: 1,606
RAC: 0
Message 10258 - Posted: 18 Sep 2005, 5:36:02 UTC
Last modified: 18 Sep 2005, 5:51:59 UTC

@adrianxw
Im talking "crazy" about the first Dimmerjas post, isnt that stupid/waste? There's nothing to do with you. And third-party thingy is for someone who wants to read. Those are not the things are pointed at you.

Clarify enough?

>>If you're going to call me crazy, come back with something better then that, huh? As it stands, it looks like you just did not read or understand what I said and did a typical knee-JERK response.

So want me to call you a JERK. fine
ID: 10258 · Report as offensive     Reply Quote
Profile Paul D. Buck

Send message
Joined: 2 Sep 04
Posts: 545
Credit: 148,912
RAC: 0
Message 10263 - Posted: 18 Sep 2005, 6:24:42 UTC - in response to Message 10241.  

<blockquote>I suspect when forced to use an LTD client, what most of us will do is not drop out of BOINC, we will stop it from time to time and edit the .xml files to get it to do what we want. A few dozen lines of code will make a little program to wipe LTD.</blockquote>
This is where I get lost in the argument. I have been running 4.43 and later clients on all my systems and I cannot understand how they are not working. One of the prime goals was to implement my resource share. BOINC now does this, with these later versions more accurately than ever before. Even better, with the 4.7x versions we have improved "correcting" behavior for run-time estimates. And, scheduler hand-shaking to make sure we get the Work assigned.

<blockquote>The point really is, why should we have too?</blockquote>
Well, the alternative is to code a change so that the client will schedule the way you would like and that the system will allow the choice. Even if UCB does not add it to the baseline, roll your own. Or use the LTD tool...

But, as I said, I can't quite get my arms around why the later versions are considered "broken". Heck, you even have the new buttons for "forcing" modes of operation manually ...
ID: 10263 · Report as offensive     Reply Quote
Profile adrianxw

Send message
Joined: 29 Sep 04
Posts: 187
Credit: 705,487
RAC: 0
Message 10270 - Posted: 18 Sep 2005, 9:19:38 UTC

@ GX10120

I am genuinely sorry if you did not mean your post as a reply to mine, but there is no way I could have known that. If you are responding to a post which is not the one above yours, it is better to add quotes from the post to which you are responding, or "@"'s otherwise misuderstanding like this will happen.

@ Paul

You see the dilema? You, the developers and others cannot understand how it can be considered broken, and we cannot understand how you can possibly think it not!

I, and others, have as simplistically as possible tried to explain what is wrong, where it started to go wrong, our accepted understanding of why it went wrong, (catering for load and forget users), and how it should be fixed, there is a precis of it in this very thread.

You are obviously aware that there are a substantial number of people that think like this. There are similar debates to this one on most of the projects messages boards and wider on the wires.

BOINC should download and install as a "load and forget it" thing. At the same time, it should not force people into taking a back seat while it does what it wants to do with their computers based on what it thinks it knows.

The informed, hands on cruncher will always be a better scheduler then BOINC.

(*** stupid example warning ***)
Today, I may want to crunch more Predictor for my own or my teams reasons. I want to be able to do that simply without BOINC's limited heuristics "correcting my behaviour" by stomping on my settings.

Now, assuming all the projects servers are running normally, (one can dream), I can do that by visiting each of the projects sites, fiddling with the quotas, then updating all of them. Why though, since the scheduling is not done at the project sites, it is done here?

Yet if I use the core client here to effect the changes I want, the newer clients will make adjustment underneath what I have done to penalise the project I wanted to promote. It will get less time for the rest of the week until the "debt" has been cleared. This is completely counter to what I wanted.
(/*** stupid example ***)

Post is getting too long, I'll shut up now.


Wave upon wave of demented avengers march cheerfully out of obscurity into the dream.
ID: 10270 · Report as offensive     Reply Quote
Profile Keck_Komputers

Send message
Joined: 1 Sep 04
Posts: 275
Credit: 2,652,452
RAC: 0
Message 10272 - Posted: 18 Sep 2005, 9:45:14 UTC

I also personally dislike the new CPU scheduler/queue handling. But my idea of how to fix it is too extreme so I don't worry about it too much. I just hope the max_projects_on_hand preference is put in which would help alot in my opinion.

I would like to go back to the 3.xx (and earlier) scheduler with JM7's LTD calculations. This setup behaves very similar to being in EDF mode at all times. The queue size can be more accurately set since it doesn't even try to get work from all projects all the time. It can handle an unlimited number of projects with no problems on any computer that can run the individual projects. The thing that will keep it from being a reality is too many people wanting the faster swithing between projects. Also too many people can't understand that the resource share is met on a longer time frame.
BOINC WIKI

BOINCing since 2002/12/8
ID: 10272 · Report as offensive     Reply Quote
Profile Paul D. Buck

Send message
Joined: 2 Sep 04
Posts: 545
Credit: 148,912
RAC: 0
Message 10277 - Posted: 18 Sep 2005, 15:45:31 UTC - in response to Message 10270.  

<blockquote>You see the dilema? You, the developers and others cannot understand how it can be considered broken, and we cannot understand how you can possibly think it not!</blockquote>
Oh my yes. I can see the dilema. I am not as insensitive as to that, contrary to rumor :)

<blockquote>I, and others, have as simplistically as possible tried to explain what is wrong, where it started to go wrong, our accepted understanding of why it went wrong, (catering for load and forget users), and how it should be fixed, there is a precis of it in this very thread.</blockquote>
My difficulty is that I don't see the loss of control that you say happens. That is where I am having problems. I grant, that I am a "fire and forget" type for the most part so the way BOINC schedules is not of much concern to me. As long as it basically does what I want which is pretty much equal time for all projects on all systems.

But, I though that if you did the suspend that the debt for that project was also not changed. So, by doing project and or work unit suspends you should be able to make the system respond as you like. Or so it seems to *ME*. I guess the examples and discussions have just not made any sense to me at all.

As it turns out, I am running one 4.27 system and yet I really don't see any noticable difference in processing on that one as contrasted with the 4.45 and 4.72 (PowerMac) versions. As I am always interested in understanding, can you explain it again? At the moment all I have is this understanding that you feel that the system does not let you have control, and yet, I don't understand how you think that you lost the control in the first place.


<blockquote>Post is getting too long, I'll shut up now.
</blockquote>
Electrons are cheap ... :)
ID: 10277 · Report as offensive     Reply Quote
Profile adrianxw

Send message
Joined: 29 Sep 04
Posts: 187
Credit: 705,487
RAC: 0
Message 10280 - Posted: 18 Sep 2005, 19:56:49 UTC

>>> Electrons are cheap ... :)

I'm in Denmark, nothing is cheap.

The use case I gave above as a silly example is just that, a silly example, but, as a use case, that is a daily request from team managers. I am trying feeble use cases to try to make the point.

I have to be at the airport in a few hours. If this is still relevent in a week or so, I'll get back to it. Hell, I'd love to get this sorted.



Wave upon wave of demented avengers march cheerfully out of obscurity into the dream.
ID: 10280 · Report as offensive     Reply Quote
Profile Chrulle

Send message
Joined: 27 Jul 04
Posts: 182
Credit: 1,880
RAC: 0
Message 10361 - Posted: 21 Sep 2005, 20:41:32 UTC - in response to Message 10280.  

<blockquote>>>> Electrons are cheap ... :)

I'm in Denmark, nothing is cheap.
</blockquote>

Except: "Hygge" :-)

Hope it didn't rain too much.

Chrulle
Research Assistant &amp; Ex-LHC@home developer
Niels Bohr Institute
ID: 10361 · Report as offensive     Reply Quote

Message boards : Number crunching : Super user ?????


©2024 CERN