Message boards :
Number crunching :
Too low credits granted in LHC
Message board moderation
Previous · 1 · 2 · 3 · 4 · 5 · Next
Author | Message |
---|---|
Send message Joined: 25 Jan 11 Posts: 179 Credit: 83,858 RAC: 0 |
Cross project credit equality is definitely impossible when projects are allowed to give whatever projects they want. We've been trying that for years and it hasn't worked, mostly because renegade projects refuse to do the decent thing and comply. CreditNew is in its infancy. It obviously isn't going to work perfectly in its infancy and it might not work perfectly 2 years from now but I think one day it will work far better than what we have now. |
Send message Joined: 25 Jan 11 Posts: 179 Credit: 83,858 RAC: 0 |
If there is no flocking happening then nobody will object to cross project parity, will they. |
Send message Joined: 1 Dec 05 Posts: 62 Credit: 11,398,274 RAC: 261 |
At last, some sanity. Ya just can't spend that crap on anything!!!Pick |
Send message Joined: 28 Nov 05 Posts: 31 Credit: 115,957 RAC: 0 |
CWe've been trying that for years and it hasn't worked, mostly because renegade projects refuse to do the decent thing and comply Oh my. That's fighting talk right there :) Who is this "we" of which you speak? "Comply" is a very nasty word. The beatings will continue until you comply etc etc. Trying to figure out who you are ... Stooge for DA? Let's see some stats: How many projects are you / have you crunched? What authority do you have to speak for the whole BOINC community (beacause, believe me, this *is* a community)? You could just be a troll of course :) Al. |
Send message Joined: 12 Jul 11 Posts: 857 Credit: 1,619,050 RAC: 0 |
Well this is surely a fascinating and conroversial subject. I don't really have time to think about it right now but it is going to keep me awake (maybe better than nightmares!). I had been thinking of offering a concrete award like a CERN visit. This would clearly be a total disaster as people being people they would do anything to win the prize. Mark you I could define my own credit system (keeping it secfret of course) but perhaps based on helpful posts, number of tasks, total CPU time, etc etc. we shall see. Right now the priority is to sort out numerical compatibility and a reliable system with well informed volunteers. My colleague Frank Schmidt suggesated I offer a PC testing service! If your result is rejected your PC is not working correctly. First I have to analyse rejected results though in case I have overlooked some 1 ULP difference somewhere. We shall see. (Incidentally the current rejection rate is down to < 0.5% as compared to 2.5% in the past.H have you overclocked your CPU??? I already found a few failing mavchines at CERN including one in the central batch computing cluster. Tired but happy. Goodnight. Eric. |
Send message Joined: 25 Jan 11 Posts: 179 Credit: 83,858 RAC: 0 |
CWe've been trying that for years and it hasn't worked, mostly because renegade projects refuse to do the decent thing and comply Are you a renegade project? Do you do indecent things? Have you refused to comply? If not then why do those words anger you? Why do you make that your business when it appears not to be your business? Do you wish to fight? Who is this "we" of which you speak? We are CreditNew. You will be assimilated. Resistance is futile. "Comply" is a very nasty word. The beatings will continue until you comply etc etc. We are not Dick Cheney. We do not beat. We assimilate. You will be assimilated. Resistance is futile. Trying to figure out who you are ... Stooge for DA? Let's see some stats: Stats are irrelevant. You will be assimilated regardless of stats. What authority do you have to speak for the whole BOINC community (beacause, believe me, this *is* a community)? I have no authority yet I speak for the collective. You could just be a troll of course :) And so could you be a troll. Whatever. Trolls, geeks, doofuses, big stats, low stats, no stats, credit whores, makes no difference. All will be assimilated. All will be decent. All your smiles will be assimilated too. Resistance is futile. |
Send message Joined: 25 Jan 11 Posts: 179 Credit: 83,858 RAC: 0 |
Mark you I could define my own credit system If your credit system is better than CreditNew it will be assimilated and used for the good of the collective. |
Send message Joined: 3 Dec 05 Posts: 11 Credit: 991,804 RAC: 41 |
>> Wow, I didn't realize that I'd be opening up such a can of worms with the credit issue! > What makes you think you opened the can? It's been open for a long time. Just standing up and taking my lumps. :-) One thing I do want to mention before continuing with our message exchange, is that the written word does not convey the human aspect of a face-to-face conversation. I ask questions because I do not know the answer, and not to be confrontational or adversarial. I make suggestions without knowing if the same suggestion has already been made. As you are reading this, you can't see that I'm sitting here scratching my head while staring off into space thinking about the conversation. I can assure you that I'm not sitting here red faced and fuming, with a large heavy object, ready to beat you soundly about the head and shoulders until you come around to my way of thinking. My intention is to contribute to the team spirit by putting my thoughts and ideas on the table. Needless to say, just because a thought pops out of my head doesn't mean that anyone is obligated to put it to use. On the other hand, sometimes the solutions to seemingly complex problems come from the most unlikely sources. Such as my IT department at work, troubleshooting a server failure for 5 hours, and swapping out motherboards, power supplies, RAM, CPU's... Then, they were jolted back into reality by a fork truck operator that drove by singing the words to a Glade air freshener commercial - "Plug it in, plug it in!". They had checked at the beginning to verify that the computer was plugged in, but, they hadn't noticed that the battery charger and radio plugged into the same outlet were not working either. They spent several thousand dollars in manpower and hardware troubleshooting what turned out to be a tripped circuit breaker. That was a poor example, but the basic point was supposed to be that you can not solve the problem until you drill down to the root cause. So, putting on our collective thinking caps, what's the root cause? I think you came really close, possibly without realizing it: > All that amounts to is equal pay for equal work, a principle long established and accepted by most. > Cross project credit equality is definitely impossible when projects are allowed to give whatever projects they want. It appears to me that cross project credit equality is impossible, because we have no way of determining what constitutes equal pay for equal work, because there is not yet any definition of what 'work' actually consists of. Now, take a deep breath, count to 10... am I close? Could it be as simple as that? We're on the third version of credit grants, so we've replaced the power supply, motherboard, and CPU. Could a lack of a definition for 'work' be the tripped circuit breaker causing the problem? >> Yes, there should be some type of limit set > Now you contradict what you just said about it being the admin's decision. Umm... er... OK, I'll concede that point, and withdraw my suggestion that the project admins assign the credit grants. Once again, you've provided something that made me rethink things. > Who's going to determine what a task is worth? Dave Anderson is going to do that AHA! Who better to define 'work' than the BOINC founder? Once we have a definition of work, it can now be applied equally across all projects. Let me catch a number... hang on... got it! 10,000 Gflops. OK, Dave decides that 1 unit of work consists of 10,000 Gflops, and 1 credit is granted for each unit of work completed. Work has just been defined, and the credit grant has been assigned. You have been assimilated. CreditNewerThanNew is now in effect. Rogue projects have just been un-rogued. Cheaters have just been tripped up. The project administrators are out of the loop, but they can still control number of tasks per CPU, number of tasks in queue, number of active tasks, all that good stuff. > the amount of granted credit can be scaled based on the percentage of the work unit that was processed. In other words, if 20 percent of the WU was processed, then 1 point is granted, 50 percent gets 2.5 points, and so on. > You're talking about a deterministic algorithm, an algorithm whose progress can be determined prior to running or counted while running. That's not always possible. There is no way to determine if 20 percent was processed, 50 percent or whatever. > Computers don't compute pie in the sky and pipe dreams. They compute numbers. OK, here's a chance for me to pick up some more education. :-) Checkpoints? When an application is interrupted, it needs to know where to pick up from when it gets CPU time again. Don't laugh too loud if I'm about to embarrass myself, but, if the application knows how long the WU is, knows what position it reached at the last checkpoint, and can successfully resume from the same point, shouldn't it be able to take those two numbers and compute the percentage of the WU that's been completed? If the WU is 561K in length, and the last checkpoint was at 178K, then the WU is 31.7 percent complete? Time for me to put this down for a bit. I hope I was able to stimulate some productive thoughts without tromping aimlessly on raw nerves. :-) I appreciate the conversation, thanks! |
Send message Joined: 3 Dec 05 Posts: 11 Credit: 991,804 RAC: 41 |
I think you may have misunderstood what the question was that I was asking here: I read the article on 'creditnew', and to be perfectly honest, about 90 percent of it shot right past me at about warp 7. I saw the parts about computer speed, processing time, etc. But I have to ask, why is the CPU benchmark given in flops? On the server status pages, why is the available computing power given in Gflops and Tflops? When a project page shows the 'user of the day', why does it say 'Person X is contributing Y Gflops'? And before I go too much further, yes, I realize that 'flops' is floating point operations per second, second being a measure of time. The question was why are flops used as the standard measure everywhere except credit grants? Back in my days of Z80 assembly programming, I routinely went through the instruction repertoire calculating the number of CPU cycles needed per instruction, just to make sure that a routine would meet the polling time requirement. How many cycles to perform the instruction fetch, and how many cycles to load either one or two bytes from RAM into the registers, and do I need a near or far load instruction, each with a different number of cycles? And, what is the memory latency in the system? Also, will I need to reprogram the PIO chip for a different data type? Not to mention interrupt handling - an MI (Maskable interrupt) involved about 50 cycles to dismiss if I recall correctly, while an NMI (Non-Maskable nterrupt) shot the whole process out the window. An NMI was the usual response to the DMA controller tri-stating the CPU data and address busses and handing control over to the FDC. When this happened, the CPU executed NOP's continuously until a CPU reset occurred in response to the 20ms heartbeat interrupt, or, the CPU entered a halt state until the DMA released the halt line. If you had a 50ms polling requirement, your routine clocked in at 38 ms, but, the FDC had the busses for 1.5 seconds, you were dead in the water with no chance of survival. As far as floating point operations, what accuracy are you looking for? How many positions will the data need to be shifted to meet the accuracy requirement? Each data shift requires a set number of cycles to perform, so it's important to know how many shifts are needed. Also, are we doing simple shifts, logical shifts, or rotates? In the Z80, not all registers were capable of 16 bit shifts and rotates, some were only capable of 8 bit operations. A 16 bit shift in a 16 bit register involved 6 prefetch cycles and 31 execution cycles. Meanwhile, a 16 bit shift in an 8 bit register pair required two separate 8 bit shifts, along with setting a bit in the flag register, and including that flag in the second 8 bit shift. Each 8 bit shift involved 6 prefetch cycles and 18 execution cycles, while the flag set/test/restore required 5 prefetch and 9 execution cycles. So, yes, I know what a flop is. I have some knowledge about machine cycles, instructions and operations. What I was asking is why aren't we uniformly maintaining that standard of measurement for the credit grants? |
Send message Joined: 17 Sep 04 Posts: 40 Credit: 293,269 RAC: 0 |
University Flashback! Now I remember why I loathed Assembler Programming. -ChinookFöhn |
Send message Joined: 25 Jan 11 Posts: 179 Credit: 83,858 RAC: 0 |
It appears to me that cross project credit equality is impossible, because we have no way of determining what constitutes equal pay for equal work, because there is not yet any definition of what 'work' actually consists of. I think most will agree that we have a definition of what work is. It's the integer and floating point operations and logical operations that occur in the CPU. Lots of people claim the definition of work should include I/O to RAM and disk, as well as downloading/uploading operations. I can't disagree with that expanded definition but it does introduce accounting problems. Counting/measuring/estimating the float and int ops alone is very difficult and very prone to inaccuracy, some applications more than others, but trying to count I/O and network traffic an app performs too is even harder, IMHO. I'm happy with just counting/estimating/measuring the int and float ops but nobody has found a good way to do that so far. Who's going to determine what a task is worth? Dave Anderson is going to do that Again, it's not the definition of work that is the problem. What I meant when I said "Dave Anderson is going to that" is that his algorithm is going to count/estimate/measure int and float ops, logical ops and whatever other units of work he decides his algorithm will count/estimate/measure. I don't know exactly how he is going to do that but he's the lead programmer, he's no dummy, so let's give him a shot at it. Once the ops and whatever other units of work are counted/measured/estimated there is already a generally accepted formula that can be used to turn those counts/measurements/estimates into credits. If the old formula is not acceptable then a new one can be negotiated and used. OK, here's a chance for me to pick up some more education. :-) Checkpoints? When an application is interrupted, it needs to know where to pick up from when it gets CPU time again. Don't laugh too loud if I'm about to embarrass myself, but, if the application knows how long the WU is, knows what position it reached at the last checkpoint, and can successfully resume from the same point, shouldn't it be able to take those two numbers and compute the percentage of the WU that's been completed? If the WU is 561K in length, and the last checkpoint was at 178K, then the WU is 31.7 percent complete? Where are youi getting those numbers from? 561K in length... what does that mean? 561K operations? If you're referring to operations then yes, at 178K the WU would be 31.7% complete BUT you can't estimate the total number of operations most WUs require ahead of time (before the WU runs) and you can't tell how many of those operations most WUs have done at any given time. That doesn't apply to EVERY computer program but it applies to those that are as complex as the ones that run under BOINC, usually. Time for me to put this down for a bit. I hope I was able to stimulate some productive thoughts without tromping aimlessly on raw nerves. :-) I hope my answers have been productive and somewhat accurate. My nerves aren't raw and you haven't trampled on them. |
Send message Joined: 25 Jan 11 Posts: 179 Credit: 83,858 RAC: 0 |
So, yes, I know what a flop is. I have some knowledge about machine cycles, instructions and operations. You certainly do! That is plain to see :-) What I was asking is why aren't we uniformly maintaining that standard of measurement for the credit grants? I'm sure Dave Anderson would like to do just that. If that could be done the problem would be mostly solved as I see it. If every science app that runs under BOINC were coded in assembly then a very accurate count of the flops and iops could be determined prior to run time IF you could know before hand how times loops would be iterated and exactly where the code would go before it exits. How many times does it enter this/that block? How many times does this loop iterate? Those answers have to be known prior to run time but they usually aren't. The problem becomes even worse when apps are compiled/assembled from C, C++, FORTRAN, etc. |
Send message Joined: 25 Jan 11 Posts: 179 Credit: 83,858 RAC: 0 |
we are on boinc version 6.11.0 with validator for sixtrack rewritten from They are definitely on NewCredit. I know because their current credit system is cheatable with the same method as every other project on CreditNew. If they weren't on CreditNew then the method I used to cheat on the latest task I returned would not have worked. I won't do it again and I won't share the method with anyone. Igor, the credits are not calculated by the client software. They are calculated by the server based on numbers reported by the client. If you want to adjust the credit on my last task to something more reasonable please do so. |
Send message Joined: 24 Oct 04 Posts: 1114 Credit: 49,501,728 RAC: 4,157 |
Naomi Rankin does like the color of cherry's when they are picked. NC or CN Volunteer Mad Scientist For Life |
Send message Joined: 3 Dec 05 Posts: 11 Credit: 991,804 RAC: 41 |
Sorry I didn't get back to you last night... between a long day at work, the grand kids screaming in the background, and then my reply to the message being 'lost' somewhere, I decided to wait till today and try again. What I was asking is why aren't we uniformly maintaining that standard of measurement for the credit grants? I think that's where I started yesterday, and yes, I agree. I'm sure Dave would like to put this all behind him also. The languages you mentioned aren't in my area of expertise, unfortunately. I can't put any code on the table for that reason. About the best I can do is to propose a semi-detailed concept, and hand it over to the people that have their hands on the code on a daily basis. I'm happy with just counting/estimating/measuring the int and float ops but nobody has found a good way to do that so far. That particular point has resulted in my brain turning into a large sized pretzel. By selecting any task in progress in my task queue, and displaying the properties (sorry - can't manage to copy it over to the message for reference... it a SETI enhanced 6.03 WU) I see two particular items that jump off the screen and bite me in the knee. Those two items being 'Estimated task size' and 'Fraction done'. The estimate of the task size comes from... somewhere? Is it assigned by the WU generator? And, is the calculation for that hard coded? What I meant when I said "Dave Anderson is going to that" is that his algorithm is going to count/estimate/measure int and float ops, logical ops and whatever other units of work he decides his algorithm will count/estimate/measure. I don't know exactly how he is going to do that but he's the lead programmer, he's no dummy, so let's give him a shot at it. Emphasis added by yours truly. :-) I chose the SETI WU because I have a fair degree of confidence that Dave has been instrumental in most, if not all aspects of generating/distributing/processing and granting credits for completion. I'm also fairly confident that the method of estimating the task size and fraction done has been sufficiently refined over the years that it has reached the point where it 'works for SETI', therefore, it 'works for Dave'. This causes me to stand up and argue that we have a time tested and proven method that can be applied across all projects. The question would then become how to ensure that all projects follow SETI's lead and use the same method? Remove any option(s) to use any method other than the hard coded algorithm. By doing that, all projects would have a uniform method of estimating task size, a uniform method of reporting the fraction done, and therefore have a uniform method of establishing credit grants. Since the project servers know set parameters (size, credits, etc) before assigning the WU, then the client claimed credit becomes irrelevant. I'm going to have to stop now for dinner. I'll go ahead and post this now, and check back in a bit, hopefully before I forget the rest of my thoughts! :-) |
Send message Joined: 14 Jul 05 Posts: 35 Credit: 71,636 RAC: 0 |
For the first time in months my LHC credit is going UP = My computer(s) (not me) are hopefully actually achieving something worth while. I personally let my comps crunch for projects I believe in not the amount of credit per WU, per FLOP, per Second or what ever. It's not the speed, but the quality - Until I get a faster computer. |
Send message Joined: 25 Jan 11 Posts: 179 Credit: 83,858 RAC: 0 |
The problem becomes even worse when apps are compiled/assembled from C, C++, FORTRAN, etc. Oh no, wasn't asking for code. I was just trying to make the point that it's even harder counting flops before runtime when the application is compiled from C, FORTRAN, etc. because compilers and math libraries can do some funny things sometimes. I'm happy with just counting/estimating/measuring the int and float ops but nobody has found a good way to do that so far. In the properties there is another interesting number... Estimated app speed in GFLOPS/sec. I have no idea what that's used for but I bet it has (or will have in the future) something to do with credit calculations. I don't know which process on the server inserts those numbers in the WU, could be WU generator. There may or may not be a calculation for those numbers, I'm not sure. Some projects might run some kind of size and speed calculation at some point, others probably just make the best estimate they can. I run 1 project that gives exactly the same size and speed estimate to every task they issue but the actual run time can be anywhere from 15 minutes to 6 hours. I don't think that means they are bad estimators or whatever calculation they might be doing is flawed. I think it is because for their application and tasks it's impossible to know, prior to runtime, how many ops the task is going to take. I'm also fairly confident that the method of estimating the task size and fraction done has been sufficiently refined over the years that it has reached the point where it 'works for SETI', therefore, it 'works for Dave'. This causes me to stand up and argue that we have a time tested and proven method that can be applied across all projects. There may be a proven method that works well for SETI applications and tasks but that same method might fail miserably on other projects' apps and tasks. I wouldn't assume anything there. The question would then become how to ensure that all projects follow SETI's lead and use the same method? Remove any option(s) to use any method other than the hard coded algorithm. By doing that, all projects would have a uniform method of estimating task size, a uniform method of reporting the fraction done, and therefore have a uniform method of establishing credit grants. Since the project servers know set parameters (size, credits, etc) before assigning the WU, then the client claimed credit becomes irrelevant. But the servers don't have set parameters to work with. That's the problem. How Dave is going to solve that problem or whether he will ever be able to solve that problem remains to be seen. It's going to take a lot of diplomacy to sell it to projects that already have their credit granting method worked out. New projects, on the other hand, seem very agreeable to CreditNew. It saves them a lot of time because it's all coded for them and if it doesn't work well they can just point the finger at Dave and say "Dave might fix it in the next server update". |
Send message Joined: 3 Dec 05 Posts: 11 Credit: 991,804 RAC: 41 |
I think I'm back. :-) I did some more mental pretzelization during dinner, and kind of refined my thoughts. I think my stumbling blocks are based on what I view to be two basic issues. First issue being what I perceive to be a lack of a solid definition of 'work'. I know you feel that there is a definition, but I just don't get a warm fuzzy with the current definition. I'll give my thoughts and reasoning on this shortly. The second issue being task length calculations for the purpose of determining the amount of work done. OK, first issue - definition of work. As I had mentioned earlier, it appears to me that the flop is the standard applied across BOINC. but, when the subject of credit grants is raised, suddenly there are several creeping unknowns thrown in to complicate and confuse - time, processor speed, memory access/latency, CPU vs GPU, who screams the loudest, rogue projects, cheaters, etc. Since credit is granted for performing work, then, in order to to have universal standard to determine how much credit to grant for performing that work, there must first be a universal standard to define the work that is being performed. If you can bear with me for a moment while I pull some numbers out of the air, a 'policy determination' from BOINC might look like this: [ In order to address concerns regarding the processing and granting of credit for Work Units, the staff of UC Berkley Science Lab, BOINC Administration and BOINC Development teams ("BOINC Admin") have adopted the following standards: 1 - The basic measure of computer processing resources expended by any type of electronic computing device will be the Unit of Work (UW). One UW consists of 0.25 Gflops. The 'Estimated Task Size' as reported by the Work Unit Generation software authored, compiled, and issued by BOINC Admin is the sole measure of work required to complete a Work Unit. 2 - One credit will be granted for each UW that is successfully completed and that has passed validation by the project that generated and issued the original Work Unit. In the event that any project administration team has any concerns regarding any aspect of Task Size estimates or Credit Granting will present their concern to BOINC Admin for review. If BOINC Admin determines that the concern is valid, BOINC Admin will decide what, if any, corrective action will be taken. ] That would give us a standard definition of work, a standard to use for granting credit, and a standard point of authority for resolving disputes. As far as the task length calculations, that's been sorta kinda halfway addressed. Maybe. Still thinking on that. The problem remaining is how to report the actual work done if the task size estimate was way off, and be 'cheat resistant' at the same time. That's the biggest contributing factor to my mental hernia right at the moment. Let me think on it some more and try to post an idea or two tomorrow. Chat at ya later! |
Send message Joined: 3 Dec 05 Posts: 11 Credit: 991,804 RAC: 41 |
After writing my message offline, then logging on and posting it, I discovered your reply. Just my luck... :-) I'm also fairly confident that the method of estimating the task size and fraction done has been sufficiently refined over the years that it has reached the point where it 'works for SETI', therefore, it 'works for Dave'. This causes me to stand up and argue that we have a time tested and proven method that can be applied across all projects. After you read message that I just posted, that pretty much summarizes the rough spot that I'm trying to get a handle on. You had mentioned concerns about the number of times a particular routine might get called, varying numbers of loop iterations, and that sort of thing. All of which are valid points. I'm trying to come up with a workable method of reporting that back to the projects in such a way that the information can be put to good use, while at the same time remaining 'cheat resistant'. Maybe one or more fields that are coded against a randomly generated one-time-key from the project servers? Hmmmm... Got any aspirin? :-) |
Send message Joined: 3 Dec 05 Posts: 11 Credit: 991,804 RAC: 41 |
In the properties there is another interesting number... Estimated app speed in GFLOPS/sec. I have no idea what that's used for but I bet it has (or will have in the future) something to do with credit calculations. I think that's used by the server scheduler. I saw that also, and just happened to notice that although it's a different number for each computer, each individual computer has the same number for every WU from every project. Stepping cautiously out on a limb, I think the scheduler uses your CPU benchmark to generate that number, and then uses that number as a gross check against the WU deadline before sending the WU. |
©2024 CERN