Message boards : Theory Application : Some WUs with fractional core usage?
Message board moderation

To post messages, you must log in.

AuthorMessage
Mr P Hucker
Avatar

Send message
Joined: 12 Aug 06
Posts: 418
Credit: 5,667,249
RAC: 48
Message 42300 - Posted: 27 Apr 2020, 14:45:40 UTC

I seem to get some Theory WUs that say something weird like 0.667C, but in Windows task manager I can see they use 1 whole core, just like the other Theories. Any reason for this?
ID: 42300 · Report as offensive     Reply Quote
Harri Liljeroos
Avatar

Send message
Joined: 28 Sep 04
Posts: 674
Credit: 43,162,254
RAC: 15,940
Message 42301 - Posted: 27 Apr 2020, 14:56:40 UTC - in response to Message 42300.  

Where do you see this?
ID: 42301 · Report as offensive     Reply Quote
Mr P Hucker
Avatar

Send message
Joined: 12 Aug 06
Posts: 418
Credit: 5,667,249
RAC: 48
Message 42302 - Posted: 27 Apr 2020, 15:36:53 UTC - in response to Message 42301.  

Where do you see this?


I use Boinctasks, which shows it in the "Use" column. Anything on one CPU core has nothing in that column. Things like Atlas show 6C to show it's using 6 cores. Things running on GPUs like Einstein show "0.5C + 1ATI", meaning half a CPU core and 1 ATI graphics card.

In the Boinc Manager, I guess (as I don't normally look at it) it would be in the "Status" column, which similarly shows "Running" for 1 CPU core, or anything else is described as "Running (0.5 CPUs + 1 AMD/ATI GPUs)".
ID: 42302 · Report as offensive     Reply Quote
Jim1348

Send message
Joined: 15 Nov 14
Posts: 602
Credit: 24,371,321
RAC: 0
Message 42303 - Posted: 27 Apr 2020, 16:01:37 UTC - in response to Message 42300.  
Last modified: 27 Apr 2020, 16:03:32 UTC

I seem to get some Theory WUs that say something weird like 0.667C, but in Windows task manager I can see they use 1 whole core, just like the other Theories. Any reason for this?

I have seen the same thing in Linux (Ubuntu 18.04). BOINCTasks will show a fractional core, or greater than one (e.g., 1.3 cores). As I recall, the "top" command shows only full cores, but I am not running it now.
I suppose it is just how virtual cores are split up by the OS for this type of work unit. Aren't they really multi-core?
ID: 42303 · Report as offensive     Reply Quote
Harri Liljeroos
Avatar

Send message
Joined: 28 Sep 04
Posts: 674
Credit: 43,162,254
RAC: 15,940
Message 42304 - Posted: 27 Apr 2020, 16:51:13 UTC - in response to Message 42302.  

Where do you see this?


I use Boinctasks, which shows it in the "Use" column. Anything on one CPU core has nothing in that column. Things like Atlas show 6C to show it's using 6 cores. Things running on GPUs like Einstein show "0.5C + 1ATI", meaning half a CPU core and 1 ATI graphics card.

In the Boinc Manager, I guess (as I don't normally look at it) it would be in the "Status" column, which similarly shows "Running" for 1 CPU core, or anything else is described as "Running (0.5 CPUs + 1 AMD/ATI GPUs)".

I am using BoincTasks as well, but I have never seen something like that. I just scrolled thru the History tab as well but nothing like that has visited my hosts in the past couple of weeks.
ID: 42304 · Report as offensive     Reply Quote
Mr P Hucker
Avatar

Send message
Joined: 12 Aug 06
Posts: 418
Credit: 5,667,249
RAC: 48
Message 42305 - Posted: 27 Apr 2020, 18:18:18 UTC - in response to Message 42303.  

I have seen the same thing in Linux (Ubuntu 18.04). BOINCTasks will show a fractional core, or greater than one (e.g., 1.3 cores). As I recall, the "top" command shows only full cores, but I am not running it now.
I suppose it is just how virtual cores are split up by the OS for this type of work unit. Aren't they really multi-core?


Atlas is multicore and downloads a WU equal to the number of cores you have assigned to Boinc. But Theory are single core. I managed to get a few of these fractional core ones a week or two ago, and it was taking notice of the number, as it ran more WUs than I had cores.
ID: 42305 · Report as offensive     Reply Quote
Mr P Hucker
Avatar

Send message
Joined: 12 Aug 06
Posts: 418
Credit: 5,667,249
RAC: 48
Message 42306 - Posted: 27 Apr 2020, 18:27:27 UTC - in response to Message 42304.  

I am using BoincTasks as well, but I have never seen something like that. I just scrolled thru the History tab as well but nothing like that has visited my hosts in the past couple of weeks.


I have attached mine below. About half of them are fractional cores.
https://www.dropbox.com/s/d4nifr21lt9v3ni/fractional.jpg?dl=0

P.S. it moaned my history was too large when I viewed the history of all projects on all computers. I guess that's due to the Milkyway Seperation WUs that take only 25 seconds per graphics card - one host downloads 900 WUs at a time. I wonder why they don't make them larger. It used to be even worse - they're now in bundles of 4 or 5, they used to take only 6 seconds.
ID: 42306 · Report as offensive     Reply Quote
Harri Liljeroos
Avatar

Send message
Joined: 28 Sep 04
Posts: 674
Credit: 43,162,254
RAC: 15,940
Message 42308 - Posted: 27 Apr 2020, 19:43:54 UTC - in response to Message 42306.  

P.S. it moaned my history was too large when I viewed the history of all projects on all computers. I guess that's due to the Milkyway Seperation WUs that take only 25 seconds per graphics card - one host downloads 900 WUs at a time. I wonder why they don't make them larger. It used to be even worse - they're now in bundles of 4 or 5, they used to take only 6 seconds.

I have seen those complaints also when there was a lot of short tasks from sixtrack and Seti. So I just reduced the number of days to store in History. I have two host (Windows) and both have their own BoincTask running.
ID: 42308 · Report as offensive     Reply Quote
Mr P Hucker
Avatar

Send message
Joined: 12 Aug 06
Posts: 418
Credit: 5,667,249
RAC: 48
Message 42309 - Posted: 27 Apr 2020, 20:06:34 UTC - in response to Message 42308.  

I have seen those complaints also when there was a lot of short tasks from sixtrack and Seti. So I just reduced the number of days to store in History. I have two host (Windows) and both have their own BoincTask running.


I left it at 14 days and selected one computer before clicking history. That seemed to work. Even on the computer that does crazy amounts of WUs with 3 large GPUs on Milkyway and Einstein.
ID: 42309 · Report as offensive     Reply Quote
Jim1348

Send message
Joined: 15 Nov 14
Posts: 602
Credit: 24,371,321
RAC: 0
Message 42311 - Posted: 27 Apr 2020, 21:49:17 UTC - in response to Message 42305.  

Atlas is multicore and downloads a WU equal to the number of cores you have assigned to Boinc. But Theory are single core. I managed to get a few of these fractional core ones a week or two ago, and it was taking notice of the number, as it ran more WUs than I had cores.

From memory, I think you see the fractional cores only if you are running native Theory and native ATLAS together. They somehow split up the work between them.
You may not see it if you are running either by itself.
ID: 42311 · Report as offensive     Reply Quote
Mr P Hucker
Avatar

Send message
Joined: 12 Aug 06
Posts: 418
Credit: 5,667,249
RAC: 48
Message 42312 - Posted: 27 Apr 2020, 22:08:27 UTC - in response to Message 42311.  
Last modified: 27 Apr 2020, 22:10:01 UTC

Atlas is multicore and downloads a WU equal to the number of cores you have assigned to Boinc. But Theory are single core. I managed to get a few of these fractional core ones a week or two ago, and it was taking notice of the number, as it ran more WUs than I had cores.

From memory, I think you see the fractional cores only if you are running native Theory and native ATLAS together. They somehow split up the work between them.
You may not see it if you are running either by itself.


I do get Atlases aswell, but I've seen the fractional Theory when I have no Atlases in the buffer. When I get a few of them at once, it runs more WUs than I have cores allocated, despite each one using an entire core. So I'm wondering why they say they need half or two thirds of a core, when in fact they need a whole core just like any other Theory WU. Mildly irritating as it ends up with insufficient cores to support my GPUs and slows that down.

I don't think they could split the work between them, as far as I know each virtual machine is entirely seperate and can't talk to each other.
ID: 42312 · Report as offensive     Reply Quote
Jim1348

Send message
Joined: 15 Nov 14
Posts: 602
Credit: 24,371,321
RAC: 0
Message 42315 - Posted: 27 Apr 2020, 23:43:16 UTC - in response to Message 42312.  

I don't think they could split the work between them, as far as I know each virtual machine is entirely seperate and can't talk to each other.

The OS can schedule threads as it wants I think, though I am not the expert on that.
But you often see GPUs using less than a core.
ID: 42315 · Report as offensive     Reply Quote
Mr P Hucker
Avatar

Send message
Joined: 12 Aug 06
Posts: 418
Credit: 5,667,249
RAC: 48
Message 42316 - Posted: 28 Apr 2020, 0:31:39 UTC - in response to Message 42315.  

The OS can schedule threads as it wants I think, though I am not the expert on that.


I assume you mean the virtual OS inside the virtual machine? I don't think they can see each other. If I'm running two Theory tasks, each is a seperate instance of Linux, under whatever OS your machine booted from.

But you often see GPUs using less than a core.


That's different. GPUs request assistance of the CPU for instructions they cannot handle themselves. Depending on how powerful your CPU and GPU are, one will be maxed out and one won't. If the GPU isn't maxed out, you're better running two or more tasks on it at once, then it can use two or more CPU cores.
ID: 42316 · Report as offensive     Reply Quote
Harri Liljeroos
Avatar

Send message
Joined: 28 Sep 04
Posts: 674
Credit: 43,162,254
RAC: 15,940
Message 42321 - Posted: 28 Apr 2020, 10:24:01 UTC

This value cannot control how much of a CPU a task will actually use. Application will use as much of CPU as it needs. This value controls how many tasks Boinc can run simultaneously until the core limit kicks in. You can overrule the server values in an app_config.xml and force Boinc to reserve 1 CPU core for each Theory task if you like. Here's the manual for it: https://boinc.berkeley.edu/wiki/Client_configuration#Project-level_configuration
ID: 42321 · Report as offensive     Reply Quote
Mr P Hucker
Avatar

Send message
Joined: 12 Aug 06
Posts: 418
Credit: 5,667,249
RAC: 48
Message 42322 - Posted: 28 Apr 2020, 11:32:28 UTC - in response to Message 42321.  
Last modified: 28 Apr 2020, 11:45:11 UTC

This value cannot control how much of a CPU a task will actually use. Application will use as much of CPU as it needs. This value controls how many tasks Boinc can run simultaneously until the core limit kicks in. You can overrule the server values in an app_config.xml and force Boinc to reserve 1 CPU core for each Theory task if you like. Here's the manual for it: https://boinc.berkeley.edu/wiki/Client_configuration#Project-level_configuration


Yes I guess I could do that, but I was wondering why they've set it to a fraction, since that will never possibly happen. The WU is entirely CPU dependant, so nothing could make it need less than 1 core! There's nothing else it would be waiting on.

So to make the setting, I do this? I hate config files and never get things right.

app_config.xml in LHC project folder:

<app_config>
  <report_results_immediately/>
  <app_version>
    <app_name>Theory</app_name>
    <avg_ncpus>1</avg_ncpus>
  </app_version>
</app_config>

ID: 42322 · Report as offensive     Reply Quote
computezrmle
Volunteer moderator
Volunteer developer
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 15 Jun 08
Posts: 2386
Credit: 223,002,758
RAC: 136,195
Message 42325 - Posted: 28 Apr 2020, 12:33:00 UTC

Once upon a time a famous sculptor was asked how to create a perfect statue from a raw marble block.
He answered: "The statue is already there. Just cut away anything not being part of it."

It's nearly as easy to create a perfect app_config.xml based on the skeleton from the already mentioned page.


The skeleton:
<app_config>
   [<app>
      <name>Application_Name</name>
      <max_concurrent>1</max_concurrent>
      [<report_results_immediately/>]
      [<fraction_done_exact/>]
      <gpu_versions>
          <gpu_usage>.5</gpu_usage>
          <cpu_usage>.4</cpu_usage>
      </gpu_versions>
    </app>]
   ...
   [<app_version>
       <app_name>Application_Name</app_name>
       [<plan_class>mt</plan_class>]
       [<avg_ncpus>x</avg_ncpus>]
       [<ngpus>x</ngpus>]
       [<cmdline>--nthreads 7</cmdline>]
   </app_version>]
   ...
   [<project_max_concurrent>N</project_max_concurrent>]
   [<report_results_immediately/>]
</app_config>



The result:
<app_config>
   <app>
      <name>Theory</name>
   </app>
   <app_version>
      <app_name>Theory</app_name>
      <plan_class>vbox64_theory</plan_class>
      <avg_ncpus>1</avg_ncpus>
   </app_version>
   <report_results_immediately/>
</app_config>




The fractional values appear to be a result of a BOINC bug and using an app_config.xml is currently the only solution to keep the balance among the projects.
:-(
ID: 42325 · Report as offensive     Reply Quote
Mr P Hucker
Avatar

Send message
Joined: 12 Aug 06
Posts: 418
Credit: 5,667,249
RAC: 48
Message 42326 - Posted: 28 Apr 2020, 12:50:35 UTC - in response to Message 42325.  
Last modified: 28 Apr 2020, 12:52:00 UTC

Once upon a time a famous sculptor was asked how to create a perfect statue from a raw marble block.
He answered: "The statue is already there. Just cut away anything not being part of it."


I've heard that before, but it was carving a horse out of a block of wood. One probably copied the other.

It's nearly as easy to create a perfect app_config.xml based on the skeleton from the already mentioned page.


It is when you understand programming. I don't, not that well anyway. For example, I have no idea why you put in the plan class, or why I have to say "Theory" twice. I'll take your word for it and use what you wrote.

The fractional values appear to be a result of a BOINC bug and using an app_config.xml is currently the only solution to keep the balance among the projects.
:-(


What is this bug? Should I report it in the Boinc forums? I thought the server clearly defined how many CPU cores were needed for each WU?
ID: 42326 · Report as offensive     Reply Quote
maeax

Send message
Joined: 2 May 07
Posts: 2071
Credit: 156,182,852
RAC: 104,928
Message 42327 - Posted: 28 Apr 2020, 13:34:36 UTC - in response to Message 42326.  

It is when you understand programming. I don't, not that well anyway.

You can say the sky is blue.
You can also say, the sky is NOT black and NOT white and NOT green.
As computezrmle wrote,
we have the possibility to work with a app_config in LHCatHome.
ID: 42327 · Report as offensive     Reply Quote
Mr P Hucker
Avatar

Send message
Joined: 12 Aug 06
Posts: 418
Credit: 5,667,249
RAC: 48
Message 42328 - Posted: 28 Apr 2020, 13:39:14 UTC - in response to Message 42327.  
Last modified: 28 Apr 2020, 13:39:58 UTC

It is when you understand programming. I don't, not that well anyway.

You can say the sky is blue.
You can also say, the sky is NOT black and NOT white and NOT green.
As computezrmle wrote,
we have the possibility to work with a app_config in LHCatHome.


Yes, I have put in computezrmle's solution. Everything in life is a workaround, and I learn what I need to get what I want :-)
ID: 42328 · Report as offensive     Reply Quote

Message boards : Theory Application : Some WUs with fractional core usage?


©2024 CERN