LHC@home

Hello,

one of my teammates has just asked this on our board:

If you look at his results, and add up the granted credit, it'll give you 49.49.

if you look at all the other pages here (not the external stats pages of course), where this should be listed, it's always only 36.58
Computer summary
Account Data
Team List

How could this happen?
Is the data also just updated once a day, and not from the main data base itself?

---

Grüße vom Sänger

Strange!
Well.....before I put in my first WU my RAC was 0.53.....and I had done no work at that time. So....perhaps there is a fault somewhere. It must be wrong or as you say updated infrequently.

---

If you look at the 12.91 credit's workunit results, you'll see that it's an 'invalid' result.

34.77+ .23+ 1.58 = 36.58

Maybe this is related to running updated Boinc server software, and those of us with a lot of credits don't notice the odd 'invalid' workunit not being credited because we aren't looking at the project as closely as we used to.

---

I'm not the LHC Alex. Just a number cruncher like everyone else here.

> If you look at the 12.91 credit's workunit results, you'll see that it's an
> 'invalid' result.
>
> 34.77+ .23+ 1.58 = 36.58
>
> Maybe this is related to running updated Boinc server software, and those of
> us with a lot of credits don't notice the odd 'invalid' workunit not being
> credited because we aren't looking at the project as closely as we used to.

It was granted 12.9058951654525 credits, and as I've been told in the other thread, that common here. And granted is granted, so it has to be added. If nothing is added, nothing is granted.

Why write some fake granted credit in the table, when it's in fact not granted?

---

Grüße vom Sänger

>
> Why write some fake granted credit in the table, when it's in fact not
> granted?
>

I think it's the nature of software development where a numbers of programmers are working on the code. Changes get made to software or systems and sometimes the regular users don't notice.

Boinc is used by a large number of projects, but I think this is the only project that gives partial credit for 'close' results. Maybe one of those SETI guys optimized the boinc server code to add numbers faster or something.

Maybe the easy fix is for LHC to not give out partial credit.

---

I'm not the LHC Alex. Just a number cruncher like everyone else here.

> I think it's the nature of software development where a numbers of programmers
> are working on the code. Changes get made to software or systems and
> sometimes the regular users don't notice.
>
> Boinc is used by a large number of projects, but I think this is the only
> project that gives partial credit for 'close' results. Maybe one of those
> SETI guys optimized the boinc server code to add numbers faster or something.
>
> Maybe the easy fix is for LHC to not give out partial credit.

I'm not into programming or database handling, I just saw this inconsistencies and pointed to them. I didn't want to rant, just to know why.

I personally would prefer to get nothing for invalid results. As I said in the other thread: granted credit, no matter what amount, is fo me a sign of proper work. If it's less then expected, I'm unlucky, but I would presume that I had a valid result. I would never look whether I had valids or invalids with all of them granted, and a lot of invalid results is imho a sign of a broken hardware beneath my table.

So what I know now is this: There is something broken in the credit allocation, and it's catched attention because it's more visible for people with little credits at all. That's reasonable, and as it is weekend, I don't expect a solution before monday ;)

Happy crunching! (and I have to put the Wiki link in my sig as well ;)

---

Grüße vom Sänger

Maybe the table that sources the summaries is updated at a different time to that which holds the results?

---

Wave upon wave of demented avengers march cheerfully out of obscurity into the dream.

> Maybe the table that sources the summaries is updated at a different time to
> that which holds the results?
>

I just had a peek in the FAQ, where it says, they update once every 12h. The situation is still the same now, 18h after my initial post (and 22h after he made his post on the S.G Board). If you take a look at the WU, the last result came in 15 Jul 2005 16:26:39 UTC, give it another 12h to validate (and that's quite long imho), and it's 16 Jul 2005 02:26:39 UTC, that's 32h since granting. So enough time has elapsed now to make it update.

The problem comes with this inconsistent handling of invalid results.

Either invalid is invalid thus no granted credit, or there is a new category "not-quite-valid-but-not-that-bad-as-well", where half the amount is granted. But then granted is granted, and not just some fake value in the table, to put you at rest, and lets you expect everything is fine, but don't add this to your score. That's deceit, and I don't even see some advantage for the project from this.

---

Grüße vom Sänger

> Either invalid is invalid thus no granted credit, or there is a new category
> "not-quite-valid-but-not-that-bad-as-well", where half the amount is granted.
> But then granted is granted, and not just some fake value in the table, to put
> you at rest, and lets you expect everything is fine, but don't add this to
> your score. That's deceit, and I don't even see some advantage for the project
> from this.


Deceit? That seems like a pretty harsh word to be throwing around. BOINC isn't perfect (yet) and there are going to be inconsistencies between projects. I personally don't think anyone at CERN is trying to be deceitful, and from the good work I've seen accomplished by the limited staffers assigned to this project, I think you're being just a little bit unfair.

---

www.clintcollins.org - spouting off at the speed of site

>> That's deceit, and I don't even see some advantage for the project from this.

Early in the project there were problems with variation in the final results. This was down to the sensitivity of the SixTrack algorithms to slight differences in FPU functions, and resulted in a wide variation in final values. Some results matched perfectly, some varied over a small range and some didn't match even across ten or more WUs. There was some disquiet among participants that through no fault of their own they were processing WUs which were then lost.

To address this, CERN implemented a validator that granted some credit, even if the results could not form a quorum. This result looks like it might be a side effect of that. The benefit to the participant was to receive some credit for their work, and the benefit to the project was to keep participants on the project who might have left otherwise. No deceit took place. The validator function was extensively discussed at the time, and is described in the FAQ.

Which is worse? Credit for valid work even if the results can't be used, or no credit if the result is 'invalid' through no fault of the participant?

Giskard

---

Gaspode the UnDressed
http://www.littlevale.co.uk

> Deceit? That seems like a pretty harsh word to be throwing around. BOINC isn't
> perfect (yet) and there are going to be inconsistencies between projects. I
> personally don't think anyone at CERN is trying to be deceitful, and from the
> good work I've seen accomplished by the limited staffers assigned to this
> project, I think you're being just a little bit unfair.

Sorry for a possible non perfect wording, but I'm no native speaker.
But...

The decision to grant credit even for non-valid results is one made by the project. I may not like it, but can live with it.

What I can't live with is fake granted credit. In the case of OTN it looks like this on the surface:
Credit wasn't granted in reality, it was just written to his table to keep him quiet, but not added to his totaly. Unfortunately he had little credit at all to see this happen.

That's the surface, but as I don't expect something as evil as this from the team, I think it's a glitch, that has to be (and can be) repaired.


> Which is worse? Credit for valid work even if the results can't be used,
> or no credit if the result is 'invalid' through no fault of the participant?

Worst is credit just written as granted to the participants results table, but in reality not granted. If that is a real policy, I'll stand even with the harsh word "deceit".

IMHO invalid is invalid is zero credit. If this leads to too much invalid results, the validator and/or sixtrack is broken and has to be adjusted.

BTW:
There has been some new credit added to his table and his totals, but the 12.91 granted for result 1554963 are still missing.

---

Grüße vom Sänger

>>If this leads to too much invalid results, the validator and/or sixtrack is broken and has to be adjusted.

I agree. Sixtrack was broken, at least for this project. It has now been fixed, which is why LHC went through several months of beta test. Maybe the validator can now be 'fixed' too, although it isn't broken, just doing what it's told.

For what it's worth, Sixtrack has been around for a long time and is a very well respected application, and thoroughly trusted. It is one of the benchmark applications used to optimise Fortran compilers. The problem was not Sixtrack, but the compiler used originally that caused the problem.

Giskard.

---

Gaspode the UnDressed
http://www.littlevale.co.uk

Hello to the project team!

Still no change and no satisfying answer :(

> If you look at his results, and add up the granted credit, it'll give you 49.49

edit: it's now 142,34 ( incl. 1.58 of 1 result, that is purged by now)


>
> if you look at all the other pages here (not the external stats pages of
> course), where this should be listed, it's always only 36.58

edit: it's now 129.43 (incl. 1.58 of 1 result, that is purged by now)

> Computer summary
> Account Data
> Team List
>
> How could this happen?

This question still stands!

As I said before:
No real big problem with partial grants, but a real big one with pretended granting just to keep them quiet.


edit:
An "OK, we see it's broken, we're working on it" will be sufficient for now.

---

Grüße vom Sänger

> How could this happen?

Because the result with id 1554936 has been flagged with validate state: Invalid in the database. Its credit has therefore not been added to his total.

---

Chrulle
Research Assistant & Ex-LHC@home developer
Niels Bohr Institute

> > How could this happen?
>
> Because the result with id 1554936 has been flagged with validate state:
> Invalid in the database. Its credit has therefore not been added to his
> total.

If the result is invalid, why has credit been granted?

Claimed is OK, the machine doesn't know it better at that stadium, but granted credit is just that: granted!!

If it's taken away out of sight from the person who got it granted, I call this deceit.

If invalid results don't add anything, why is there something in the column?
Why is this fake value stated in there?
What is this fake granted credit good for?
Why not just write 0.00?

Or, if it's intended to give those close, but not right, something, as has been said in former answers, why not add it?

If invalid will not be added, it has to be stated as granted credit 0.00.
If some granted credit is listed in the table, it has to be added.

Everything else is either a glitch, programming error, whatever, I don't mind, it can happen, it's been found fortunately and can be repaired.

Or, if not considered as such but considered as "work as designed", it's deceit, betrayal, cowardice....whatever, definitely not my project any more.

I believe it's the first, I really do, and I think it will be repaired soon, as I can't believe, the second is a real possibility with a project like LHC@Home.


edit:
Not to be taken false here, if I get more invalids here in this still beta stadium, it's OK, so what, it's just credit.
But with this kind of presentation I will never know, because I will never see anything invalid, unless I look close at every single result after validation.
To know, I've got lot's of invalids is of value as well: I'll know, there's something wrong with my puter, I should take a closer look.

---

Grüße vom Sänger

Hello again!

Weekend is nearing again, and still no satisfying answer regarding this issue.

Is the policy of granting half the credits for close but invalid results still in place?

Will these granted credits be added to the totals?

Is there a solution for past results (like his)?

It's impossible to look up the (in)valid result anymore, as the database has been cleaned as designed. And the follow-up of such things is getting harder because of this purges as well, as you can't just add-up all results any longer, but have to keep track somehow manually. But that's no problem, as the glitch is identified and can be dealt with imho.

---

Grüße vom Sänger

And another week goes by without any sufficient answer from the team :(

I'm getting a bit upset, and begin to suspect there is none, at least none that could be made public without embarrasment.

My questions again:

What is the current project policy regarding invalid results?

Is the policy of granting partial credit still in place?

If so, are these credits realy granted, or just some cover up for not granted?


My opinion:

Invalid is invalid is 0.00 credits.

Invalid results will only be recognised by participants, if nothing is granted, and it is of great value to know, if ones puters just crunch invalids.

Lot's of invalid mean, that this CPU/OS setup is not suitable for this projects needs. I can either fix it, or leave. That's fine, no harm done (as is imho with fake credit).

If too little participants can reach the desired results, the project team on the other hand has to fix it's problems, as something has to be wrong here.

---

Grüße vom Sänger

I'm no project dev, but here is what I suspect is going on:

The validator was written by the guys at Berkeley along with the rest of the BOINC package. The validator as written is designed to compare returned results, and grant credit to those which form a quorum (a set of several agreeing results). The validator is sufficient for most of the projects, and so any problems with it would be low on the priority list (when compared to bugs that effect all projects).

It has been known for some time that Sixtrack has some issues with generating consistant results among various CPUs. Early in the project, it was decided to enable the validator's "homegenous redundancy" option, which forces work to be passed out only to similar (the exact same?) CPUs. Obviously this isn't a great solution, since it means that results processed on an Intel Celeron D for example can only be processed by other Celeron Ds, greatly reducing the speed of validation.

After quite a while where homogenous redundancy was the norm for LHC, it was decided to try to fix Sixtrack so that it's results were more consistant, allowing homogenous redundancy to be disabled. I believe they were sucessful, but only marginally so -- some CPUs still had issues validating correctly against others. They decided to disable homogenous redundancy anyway since it would validate correctly most of the time, though there was still the odd case where it wouldn't.

As a partial fix for this, LHC modified the validator so that it would generate partial credit for "semi-valid" results. The validator itself would consider such a result invalid (since it does not match with the quorum), but would grant credit anyway (since it's probably actually valid). The problem with this hack as you found though is that credits from invalid results (which would NORMALLY be zero) are not added into the grand total as Chrulle pointed out, so even though they're "granted", it's not reflected.

Why did LHC opt to use partial credit instead of really fixing the validator? Who knows. Perhaps it would have taken far too long, perhaps the Berkeley guys are working on it and gave LHC this validator as a temporary version, perhaps the real fix is much harder than this hack to implement... Like I said, I'm not a project dev, so I haven't a clue.

I highly doubt that this is a case of "deceit", but just an honest oversight as you mentioned (the oversight being that results flagged invalid aren't totaled up, regardless of the partial granted credit).


Puffy

---

We believe we have found the reason for the missing credit.

It is not a validator bug (which by the way is written by us). Around the time of the missing credit the database chrashed. the recovery led to a number of WUs being assimilated before the results had come back, this meant that it was impossible to do real credit granting through the validator. Markku then went through those results by hand and granted them credit, but we(Markku and I) believe that he forgot to add those extra credits to the users total.

---

Chrulle
Research Assistant & Ex-LHC@home developer
Niels Bohr Institute

<blockquote>And another week goes by without any sufficient answer from the team :(

I'm getting a bit upset, and begin to suspect there is none, at least none that could be made public without embarrasment.

My questions again:

What is the current project policy regarding invalid results?

Is the policy of granting partial credit still in place?

If so, are these credits realy granted, or just some cover up for not granted?


My opinion:

Invalid is invalid is 0.00 credits.

Invalid results will only be recognised by participants, if nothing is granted, and it is of great value to know, if ones puters just crunch invalids.

Lot's of invalid mean, that this CPU/OS setup is not suitable for this projects needs. I can either fix it, or leave. That's fine, no harm done (as is imho with fake credit).

If too little participants can reach the desired results, the project team on the other hand has to fix it's problems, as something has to be wrong here.

</blockquote>

As obviously nothing has changed (see here), and answers to my direct questions were avoided, I bump this one up again.

---

Grüße vom Sänger