Message boards : ATLAS application : credits for runtime, not for cputime ?
Message board moderation

To post messages, you must log in.

1 · 2 · Next

AuthorMessage
Klaus

Send message
Joined: 27 Aug 15
Posts: 27
Credit: 6,577,749
RAC: 1,617
Message 31003 - Posted: 24 Jun 2017, 6:09:22 UTC

To decrease my cpu-temperature i decrease the cpu usage from 90 % to 45 %.
The runtime and credits for tasks are increasing. My average credit doesn't decrease since 10 days!
My question: credits for runtime, not for cputime ?
ID: 31003 · Report as offensive     Reply Quote
Profile HerveUAE
Avatar

Send message
Joined: 18 Dec 16
Posts: 123
Credit: 37,495,365
RAC: 0
Message 31031 - Posted: 25 Jun 2017, 4:30:05 UTC - in response to Message 31003.  

From what I can see, you have only decreased the CPU throttling, so you can still run 8-core tasks on your 8-core computer.
Indeed the credit allocation is based on run time, and not CPU time (that was mentioned by David Cameron in a thread in old ATLAS project). So it seems that the reduced efficiency in completing a single task is not affecting the credit allocation!
We are the product of random evolution.
ID: 31031 · Report as offensive     Reply Quote
Klaus

Send message
Joined: 27 Aug 15
Posts: 27
Credit: 6,577,749
RAC: 1,617
Message 31032 - Posted: 25 Jun 2017, 5:34:30 UTC

In BOINC-manager/opinions/calculation settings i reduced the value for use cpu-time less than ... This was the only change i made. Where you can see this, i don't know. Perhaps you can believe it.
My profession is engeniering and I do not understand why there are the same credits for calculating a WU in 1 h or in 2 to 7 h (cpu-time is the same). Scientists need a lot of results in a short time, so I think it is better to give more credits for a short calculation-time.
(In the last 40 years my income depends on results, not on beeing there)
This is to think over.
ID: 31032 · Report as offensive     Reply Quote
computezrmle
Volunteer moderator
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 15 Jun 08
Posts: 1818
Credit: 122,901,984
RAC: 76,283
Message 31033 - Posted: 25 Jun 2017, 6:08:12 UTC - in response to Message 31032.  

In BOINC-manager/opinions/calculation settings i reduced the value for use cpu-time less than ... This was the only change i made. Where you can see this, i don't know. Perhaps you can believe it.

From your log:
2017-06-23 12:25:58 (4820): Setting CPU throttle for VM. (45%)



... (In the last 40 years my income depends on results, not on beeing there)
This is to think over.


If you rent a flat you pay it monthly regardless if you sleep there or not.
If you rent a car you pay per day regardless if you drive it or leave it at the parking site.
If a BOINC project allocates your computing resources you get "paid" in credits regardless of the project's efficiency.
ID: 31033 · Report as offensive     Reply Quote
Klaus

Send message
Joined: 27 Aug 15
Posts: 27
Credit: 6,577,749
RAC: 1,617
Message 31034 - Posted: 25 Jun 2017, 6:50:57 UTC

Thanks for the the extra lesson.
You are right in all points.
But I get "paid" in credits regardless of my computer's efficiency.
This is to think over.
ID: 31034 · Report as offensive     Reply Quote
computezrmle
Volunteer moderator
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 15 Jun 08
Posts: 1818
Credit: 122,901,984
RAC: 76,283
Message 31035 - Posted: 25 Jun 2017, 8:04:48 UTC - in response to Message 31034.  

Discussions about credit calculation are always very emotional.
I have a very personal opinion about it that I don't want to share here.

Reduced to a very simple technical perspective you may read the BOINC documentation.
This may at least make it clear why it cannot be solved on the fly.
ID: 31035 · Report as offensive     Reply Quote
Klaus

Send message
Joined: 27 Aug 15
Posts: 27
Credit: 6,577,749
RAC: 1,617
Message 31036 - Posted: 25 Jun 2017, 10:22:52 UTC

For credids can nobody buy anything, so calculation of credit is no mainproblem.
Reading the BOINC documentation shows to me, that the fairness is a highly complex problem.
I'm not satisfied, but I do not know a solution for everybody.
When my roomtemperature decrease, when there are tasks in the queue i will speed up my cpu as previously.
ID: 31036 · Report as offensive     Reply Quote
djoser
Avatar

Send message
Joined: 30 Aug 14
Posts: 145
Credit: 10,847,070
RAC: 0
Message 31163 - Posted: 28 Jun 2017, 18:17:55 UTC - in response to Message 31036.  

Do i understand this right?

Let's say i would want to max out my credit for Atlas-tasks.
So the best would be to limit Atlas to crunch only single core tasks not multicore, right?


For credids can nobody buy anything...


That's not quite correct! See my signature for more infos.

Regards, djoser.
Why mine when you can research? - GRIDCOIN - Real cryptocurrency without wasting hashes! https://gridcoin.us
ID: 31163 · Report as offensive     Reply Quote
computezrmle
Volunteer moderator
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 15 Jun 08
Posts: 1818
Credit: 122,901,984
RAC: 76,283
Message 31171 - Posted: 29 Jun 2017, 6:46:28 UTC - in response to Message 31163.  

Do i understand this right?

Let's say i would want to max out my credit for Atlas-tasks.
So the best would be to limit Atlas to crunch only single core tasks not multicore, right?

In your case probably not as your 2-core setup seems to be nearly perfect.
At least on this host:
https://lhcathome.cern.ch/lhcathome/show_host_detail.php?hostid=10486800

If you calculate the ratio cputime/(ncores*walltime) you get values between 96.5 % and 98.0 % for your last valid ATLAS results.
There would not be much room for a performance boost if you would run 2 concurrent VMs in a 1-core setup.
Instead it would lock roughly 2 GB more RAM that can be used now by your OS.
ID: 31171 · Report as offensive     Reply Quote
djoser
Avatar

Send message
Joined: 30 Aug 14
Posts: 145
Credit: 10,847,070
RAC: 0
Message 31172 - Posted: 29 Jun 2017, 7:33:31 UTC - in response to Message 31171.  

There would not be much room for a performance boost if you would run 2 concurrent VMs in a 1-core setup.


Thanks for your answer!

But if credits are granted for runtime, not cputime then running two single-core tasks concurrently should give twice the amount of credits than one two-core task, correct?

Or what am i missing here?

djoser.
Why mine when you can research? - GRIDCOIN - Real cryptocurrency without wasting hashes! https://gridcoin.us
ID: 31172 · Report as offensive     Reply Quote
Crystal Pellet
Volunteer moderator
Volunteer tester

Send message
Joined: 14 Jan 10
Posts: 1046
Credit: 6,601,227
RAC: 256
Message 31175 - Posted: 29 Jun 2017, 9:14:17 UTC - in response to Message 31172.  

Or what am i missing here?

djoser.

Nothing.

You're right. It's up to you to configure your system(s) to your needs.

ATLAS is the best sub-project in the LHC-project for fixed credits, at least for tasks/jobs from the same batch.
ID: 31175 · Report as offensive     Reply Quote
computezrmle
Volunteer moderator
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 15 Jun 08
Posts: 1818
Credit: 122,901,984
RAC: 76,283
Message 31181 - Posted: 29 Jun 2017, 12:02:11 UTC - in response to Message 31172.  

There would not be much room for a performance boost if you would run 2 concurrent VMs in a 1-core setup.


Thanks for your answer!

But if credits are granted for runtime, not cputime then running two single-core tasks concurrently should give twice the amount of credits than one two-core task, correct?

Or what am i missing here?

djoser.

Without looking into the server code I guess that ATLAS solves this issue by calculating a new flops value for the host:
new_flops = original_flops * n_cpus

This way the n_cpus gets represented in the credit calculation algorithm.

It may have been Laurence or David Cameron who mentioned in an old thread that credit calculation is based on walltime.
This ensures that volunteers get rewarded for providing their resources even if the application hangs in an idle loop.
Sorry I can't find the reference at the moment.
ID: 31181 · Report as offensive     Reply Quote
David Cameron
Project administrator
Project developer
Project scientist

Send message
Joined: 13 May 14
Posts: 354
Credit: 12,041,535
RAC: 3,823
Message 31196 - Posted: 30 Jun 2017, 11:28:43 UTC

There is a long discussion on this topic on the old ATLAS@Home forum

In conclusion: running single-core tasks gives you more credit, but it's not easy to fill a machine with single-core tasks because BOINC requires the full n-cores memory for each one (even if it doesn't use it).
ID: 31196 · Report as offensive     Reply Quote
marmot
Avatar

Send message
Joined: 5 Nov 15
Posts: 143
Credit: 6,301,268
RAC: 0
Message 33176 - Posted: 30 Nov 2017, 15:12:00 UTC
Last modified: 30 Nov 2017, 15:39:31 UTC

Labor is one of the cornerstones for valuation of services/item production and CPU labor(cycles) is what LHC, or any other BOINC project, is receiving.

If LHC is using 8 CPU's and only giving credit for 1 CPU when the other 7 could be used by other projects that are also offering credit, then the payment system is predatory.

I know the argument can be made that the efficiency in performance means faster throughput and so payment is equivalent but... that's not what my data sets from actual measuring is proving.

Single core ATLAS is giving out 1500% to 2000% greater credit per core as the 8 core jobs and even 100% more than the most efficient 4 core.

For comparison, the 2 core Theory performed ~10% better than 1 core WU's and the 8 core Theory WU's (while having 33% CPU inefficiencies) still paid only ~15% worse than a 1 core WU. All tests done on the same machine and compared to another identically configured machine with similar results.

If the multi-core implementations of Theory are able to pay nearly equivalent credit per CPU core claimed then ATLAS should be able to do the same and the 8 core jobs are an order of magnitude worse in payment per core.

I was shocked to see that it's approximately 9x better to run 8 single core ATLAS than to run 3x 8 multicore ATLAS.
ID: 33176 · Report as offensive     Reply Quote
David Cameron
Project administrator
Project developer
Project scientist

Send message
Joined: 13 May 14
Posts: 354
Credit: 12,041,535
RAC: 3,823
Message 33198 - Posted: 3 Dec 2017, 21:52:36 UTC - in response to Message 33176.  

As far as I understand credit is based relative runtime compared to the estimated runtime, not on absolute runtime. Therefore if you have been running 8-core tasks for some time then suddenly switch to single core, they run 8 times longer than expected so they get massive credit. But if you keep running single core the estimated runtime adjusts up over time and so the credit decreases.

Also I think Theory tasks work differently in that they have a fixed runtime independent of the number of cores - the WU keep pulling tasks until the time is up. ATLAS tasks have a pre-determined number of events to process so have a different runtime for different core numbers.
ID: 33198 · Report as offensive     Reply Quote
Greger

Send message
Joined: 9 Jan 15
Posts: 149
Credit: 431,596,822
RAC: 0
Message 44653 - Posted: 4 Apr 2021, 10:52:20 UTC

Need to bring up this old thread as i have notice this issue remain unsolved and host does abuse it more each day. I have posted PM to admins on site to look up abnormal behavior from host and accounts attached to these.

As you say the credit would adjust by time to host but we see that host are generated and burned out after fetched some work and then re-send result back.
This may be a part of abuse of this "creditnew" and project may have additional way on top of it but looks like it could be abused as it is today.

So issue remain that task could be tricked still today and system could make 10 times what they should get or even more from generating several boinc instances and get high credit of runtime both for theory and atlas and it would not stop there is also additional ways to fiddling with runtime and cputime host offline before resend to server.
I have so far seen 2 accounts here that abused it hard and that is with same system to open up many hosts with same result with high runtime but low cputime. Task looks clean on task and unit and generate hits-file but workunits don't have no wingman to compare with and how credits are generated is wrong.
Theory task remain fixed to 2 days on these host but last week they manage to go from 1/10 of cpu time to equal as the runtime even so jobs remain low.
Atlas is mt task and tricked to 8 threads and device peak flops is set but running only with 1 thread.

This is negative to project in long run and should be investigated. We need to change how credits is counted and apply cpu usage into counting would step forward
I have put message to admins here and to other project affected also talked to devs and Gridcoin community to review credit systems. This track back to "creditnew" and possible issues of this system. A re-work on new or custom made credit solution may be needed.

I have step out until there have been some re-work on this and would like admins to take a look on this and project could be greylisted until it has been fixed
ID: 44653 · Report as offensive     Reply Quote
djoser
Avatar

Send message
Joined: 30 Aug 14
Posts: 145
Credit: 10,847,070
RAC: 0
Message 44654 - Posted: 4 Apr 2021, 21:12:20 UTC - in response to Message 44653.  
Last modified: 4 Apr 2021, 21:56:10 UTC

I have step out until there have been some re-work on this and would like admins to take a look on this and project could be greylisted until it has been fixed

I think there will be no re-work of this matter ever. Here is why:

First of all the amount of work done by BOINC-Users is just a small fraction of all work being done by WLCG internally (although it's said to be compareable to a TIER-2 site). I think they just keep up the infrastrucure for BOINC because it's free computing power to do simulations. And BOINC-Crunchers do nothing else but simulations for CERN (besides Sixtrack).

Secondly the credit issue just affects Gridcoin participants, which again are only a small fraction of BOINC-Crunchers. Normal BOINC-Crunchers don't really care about credits.

So why should they waste time (and therefore money) to fix something that only matters to a few cryptocurrency nerds? Don't get me wrong, i'm one of those nerds, but even i think this matter is not really important. I'm in for the science, not for the Gridcoins!
Why mine when you can research? - GRIDCOIN - Real cryptocurrency without wasting hashes! https://gridcoin.us
ID: 44654 · Report as offensive     Reply Quote
Harri Liljeroos
Avatar

Send message
Joined: 28 Sep 04
Posts: 547
Credit: 29,882,520
RAC: 11,721
Message 44655 - Posted: 4 Apr 2021, 21:31:22 UTC

I think that the runtime was selected instead of cpu-time because of tasks like this come around every now and then: https://lhcathome.cern.ch/lhcathome/workunit.php?wuid=159721656 This is a CMS task and not Atlas but same applies here. Normally CMS tasks run about 12-13 hours and are terminated at 18 hours if they run longer. Now this task had the runtime of 18 hours but cputime only about 7 hours. For some reason the task stopped progressing before it should have. These kind of tasks were abundant in the beginning (and still some users bump into them) and caused a rage when several hours of calculation were wasted and practically no credit was given for that. So to compensate these faulty tasks credit was decided to give based on runtime and not cputime. This unfortunately results in what you have described.
ID: 44655 · Report as offensive     Reply Quote
Greger

Send message
Joined: 9 Jan 15
Posts: 149
Credit: 431,596,822
RAC: 0
Message 44795 - Posted: 23 Apr 2021, 14:12:59 UTC

This is unfortunately that there is no mechanism to check task if operate as it should. Only thing i could come up with would be trickle info with wrapper and possibly make client to cancel task if any issues job occurred. It would probably not be easy to code perfect system but sure it could be better.

I would like to announce that this project have been greylisted to Gridcoin until until it this is solved.

It is sad but necessary action for now. I have spent many cpu hours to project and had good time to contribute to it. I hope we could get it solved add get back soon.

https://github.com/gridcoin-community/Gridcoin-Tasks/issues/218
ID: 44795 · Report as offensive     Reply Quote
Greger

Send message
Joined: 9 Jan 15
Posts: 149
Credit: 431,596,822
RAC: 0
Message 44796 - Posted: 23 Apr 2021, 14:44:22 UTC - in response to Message 44654.  

I have step out until there have been some re-work on this and would like admins to take a look on this and project could be greylisted until it has been fixed

I think there will be no re-work of this matter ever. Here is why:

First of all the amount of work done by BOINC-Users is just a small fraction of all work being done by WLCG internally (although it's said to be comparable to a TIER-2 site). I think they just keep up the infrastructure for BOINC because it's free computing power to do simulations. And BOINC-Crunchers do nothing else but simulations for CERN (besides Sixtrack).

Secondly the credit issue just affects Gridcoin participants, which again are only a small fraction of BOINC-Crunchers. Normal BOINC-Crunchers don't really care about credits.

So why should they waste time (and therefore money) to fix something that only matters to a few cryptocurrency nerds? Don't get me wrong, i'm one of those nerds, but even i think this matter is not really important. I'm in for the science, not for the Gridcoins!


Maybe not and i do agree to that. Boinc may not be big part to LHC in most simulation but there work into build it for boinc and maintain it to open up for people to volunteer support big or small inspired me. The possibility to support with my system on complex workloads and able to use native is good move from them. Now if we gpu support it would be even better and boinc could have great power to add on project. It is good that have been in work to add Atlas-long and now gpu application.

It sad that it would be pointed to credit to value contribution and not fair shared to to real job done. It just that it is open for users to trick systems to gain 10x more. Credits is somewhat useful for non-gridcoin users also to compare contribution and that would be something that would stay. Some would be badge hunters and others would focus on low user-id and testing in beta. At end You would do good and have the honor as pure volunteer.

I like community as it is to Gridcoin and good feelings to to contribute project but also get something back. I would still encourage users to contribute to project and i would probably also do so if gpu application is added but would be more to test and run it in small scaled not 24/7 on all systems.

I hope boinc server devs or project admin could improve boinc credit options and focus on this issue. There have been several issues unsolved to native application but more to virtualbox. These jobs to longrunners have been better to Theory but remain still and each new users would experience it and troubleshoot them.
ID: 44796 · Report as offensive     Reply Quote
1 · 2 · Next

Message boards : ATLAS application : credits for runtime, not for cputime ?


©2021 CERN