Message boards : Theory Application : Move TheoryN Back Into Theory.
Message board moderation

To post messages, you must log in.

1 · 2 · 3 · 4 · Next

AuthorMessage
Profile Laurence
Project administrator
Project developer

Send message
Joined: 20 Jun 14
Posts: 336
Credit: 237,918
RAC: 0
Message 40443 - Posted: 14 Nov 2019, 10:29:57 UTC

Now that the Theory VM app is running using the same model as the native Theory app, are there any objections to moving the native app (TheoryN ) into Theory? From what I can see the credit given is comparable.
ID: 40443 · Report as offensive     Reply Quote
Crystal Pellet
Volunteer moderator
Volunteer tester

Send message
Joined: 14 Jan 10
Posts: 934
Credit: 6,286,290
RAC: 664
Message 40444 - Posted: 14 Nov 2019, 11:50:19 UTC - in response to Message 40443.  

I know from ATLAS there is a big difference in needed disk space between the VBox- and native version.
I suppose this will be the same with Theory, so you will have to choose the higher of the 2 rsc_disk_bounds of the 2 applications.
ID: 40444 · Report as offensive     Reply Quote
Profile Laurence
Project administrator
Project developer

Send message
Joined: 20 Jun 14
Posts: 336
Credit: 237,918
RAC: 0
Message 40445 - Posted: 14 Nov 2019, 12:32:51 UTC - in response to Message 40444.  

Do you know which version is preferred? The Vbox version or the native version?
ID: 40445 · Report as offensive     Reply Quote
Crystal Pellet
Volunteer moderator
Volunteer tester

Send message
Joined: 14 Jan 10
Posts: 934
Credit: 6,286,290
RAC: 664
Message 40446 - Posted: 14 Nov 2019, 13:48:07 UTC - in response to Message 40445.  

Do you know which version is preferred? The Vbox version or the native version?
The value for native is far too low for the VBox-version even without making a snapshot when suspending, so use the 2000000000 from the VBox-version.
ID: 40446 · Report as offensive     Reply Quote
Profile Laurence
Project administrator
Project developer

Send message
Joined: 20 Jun 14
Posts: 336
Credit: 237,918
RAC: 0
Message 40447 - Posted: 14 Nov 2019, 14:42:45 UTC - in response to Message 40446.  

Do you know which version is preferred? The Vbox version or the native version?
The value for native is far too low for the VBox-version even without making a snapshot when suspending, so use the 2000000000 from the VBox-version.

Sorry I meant if you have VBox installed, do you know whether the client will get a VBox task or a native one?
ID: 40447 · Report as offensive     Reply Quote
Crystal Pellet
Volunteer moderator
Volunteer tester

Send message
Joined: 14 Jan 10
Posts: 934
Credit: 6,286,290
RAC: 664
Message 40448 - Posted: 14 Nov 2019, 14:48:37 UTC - in response to Message 40447.  

Sorry I meant if you have VBox installed, do you know whether the client will get a VBox task or a native one?
I don't have VBox installed on my Linux-VM. I created that guest-VM, just to test the native applications and run VBox on the Windows host.
Someone else may answer that question.
ID: 40448 · Report as offensive     Reply Quote
computezrmle
Volunteer moderator
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 15 Jun 08
Posts: 1428
Credit: 73,019,068
RAC: 105,551
Message 40449 - Posted: 14 Nov 2019, 15:08:36 UTC - in response to Message 40447.  

... if you have VBox installed, do you know whether the client will get a VBox task or a native one?

Untested, but ...
since Theory native (appid=17) and Theory vbox (appid=13) have different appids the client gets what is checked at the preferences page.
This is in contrast to ATLAS where both apps are served under the same appid.
ID: 40449 · Report as offensive     Reply Quote
Profile Laurence
Project administrator
Project developer

Send message
Joined: 20 Jun 14
Posts: 336
Credit: 237,918
RAC: 0
Message 40450 - Posted: 14 Nov 2019, 15:14:20 UTC - in response to Message 40448.  

Someone else may answer that question.

I have added the native app as a beta app to test. Here is what I found so far. Without having VBox installed the native app is tried and fails as I don't have CVMFS installed either. When VBox is installed it oscillates between the VBox and native app. [/list]
ID: 40450 · Report as offensive     Reply Quote
computezrmle
Volunteer moderator
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 15 Jun 08
Posts: 1428
Credit: 73,019,068
RAC: 105,551
Message 40451 - Posted: 14 Nov 2019, 15:31:55 UTC

Although Theory native is a bit more efficient there are a lot of pros to focus on the vbox version.
ID: 40451 · Report as offensive     Reply Quote
BITLab Argo

Send message
Joined: 16 Jul 05
Posts: 24
Credit: 35,251,537
RAC: 0
Message 40452 - Posted: 14 Nov 2019, 15:41:30 UTC - in response to Message 40450.  

I think you are asking about converting Theory to use the Atlas model.

...When VBox is installed it oscillates between the VBox and native app.
My experience from Atlas:
* If both VBox and "native" (i.e. its dependencies) are installed and working, the client will oscillate for a while, eventually preferring whichever option gets it the most credits
* If VBox is missing but "native" is OK, the server won't issue Vbox tasks (which is fine) and the client will run native tasks (which is fine)
* If VBox is OK but the (Linux) host is missing the "native" dependencies, the client will fetch a mixture of tasks but the native ones will fail, wasting resources and not so fine. Worse (though this is based on experience with an older client version) the client seemed to then prioritise the failing native version instead of the successful Vbox one. So I think there needs to be a way for Linux users to opt out of the native app.
ID: 40452 · Report as offensive     Reply Quote
Profile Laurence
Project administrator
Project developer

Send message
Joined: 20 Jun 14
Posts: 336
Credit: 237,918
RAC: 0
Message 40453 - Posted: 14 Nov 2019, 15:44:50 UTC - in response to Message 40451.  

Although Theory native is a bit more efficient there are a lot of pros to focus on the vbox version.

I think I have found a solution where the scheduler will prefer the VBox version if you have it installed. If this machine keeps getting the VBox version, it means it worked. Is there away for someone to override this behavior on the client to so that you can request the native app even if you have VBox installed? I see there is a dont_use_vbox option in the cc_config.xml.
ID: 40453 · Report as offensive     Reply Quote
computezrmle
Volunteer moderator
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 15 Jun 08
Posts: 1428
Credit: 73,019,068
RAC: 105,551
Message 40454 - Posted: 14 Nov 2019, 16:19:36 UTC

BITLab Argo wrote:
I think you are asking about converting Theory to use the Atlas model.

If this is the goal I
- didn't get it yet
- would suggest NOT to do so as it might conflict with other projects (cosmology) that require VirtualBox.


@Laurence
Be so kind as to clarify what the goal is.
ID: 40454 · Report as offensive     Reply Quote
Profile Laurence
Project administrator
Project developer

Send message
Joined: 20 Jun 14
Posts: 336
Credit: 237,918
RAC: 0
Message 40455 - Posted: 14 Nov 2019, 18:35:50 UTC - in response to Message 40452.  

I think you are asking about converting Theory to use the Atlas model.

Basically yes in the sense that the native app and vm app are both available. Thanks for describing your experience, it correlates with what I observed.
ID: 40455 · Report as offensive     Reply Quote
Profile Laurence
Project administrator
Project developer

Send message
Joined: 20 Jun 14
Posts: 336
Credit: 237,918
RAC: 0
Message 40456 - Posted: 14 Nov 2019, 18:38:57 UTC - in response to Message 40454.  


@Laurence
Be so kind as to clarify what the goal is.

It is a situation rather than a goal. If the vm and native app are both available, how should they be prioritized under different situations, then how do we implement what we want?
ID: 40456 · Report as offensive     Reply Quote
maeax

Send message
Joined: 2 May 07
Posts: 916
Credit: 33,703,146
RAC: 6,710
Message 40457 - Posted: 14 Nov 2019, 18:53:13 UTC - in response to Message 40444.  
Last modified: 14 Nov 2019, 18:56:23 UTC

I know from ATLAS there is a big difference in needed disk space between the VBox- and native version.
I suppose this will be the same with Theory, so you will have to choose the higher of the 2 rsc_disk_bounds of the 2 applications.

Yes, my data-transfer is half of them before (300-400 Gbyte now up to one TByte before) for a month.
Atlas-native and Theory-native together.
Both are Linux-VM and have cache and need no new download for a lot of Data.
Is it possible to let -native running under a other Hypervisor than Virtualbox?
Every task need a new start of the inside Linux in -native.
-native have a good stability for doing the work.
Have no idea which concept is better.
Running also cosmology with a other container-concept (Boinc to docker).
When Atlas and Theory (later CMS and/or Alice) have the same software is a good idea, if it is possible to realize.
ID: 40457 · Report as offensive     Reply Quote
Profile Laurence
Project administrator
Project developer

Send message
Joined: 20 Jun 14
Posts: 336
Credit: 237,918
RAC: 0
Message 40465 - Posted: 15 Nov 2019, 8:49:13 UTC - in response to Message 40457.  
Last modified: 15 Nov 2019, 8:49:57 UTC


Yes, my data-transfer is half of them before (300-400 Gbyte now up to one TByte before) for a month.
Atlas-native and Theory-native together.
Both are Linux-VM and have cache and need no new download for a lot of Data.
Is it possible to let -native running under a other Hypervisor than Virtualbox?
Every task need a new start of the inside Linux in -native.
-native have a good stability for doing the work.
Have no idea which concept is better.
Running also cosmology with a other container-concept (Boinc to docker).
When Atlas and Theory (later CMS and/or Alice) have the same software is a good idea, if it is possible to realize.

It would be possible to run on other Hypervisors but I don't see there to be much benefit. Hyper-V on Windows may be better for that platform but is only available in the Professional editions. I think containers is a good direction for BOINC applications. Fundamentally Theory, Atlas and Cosmology are all using the same Linux container technology. Hopefully in the near future there will be easier ways to run Linux containers on Windows such as WSL but again as this requires Hyper-V, may only be available in the Professional editions.

Edit: Thoughts and ideas on things like this are always welcome.
ID: 40465 · Report as offensive     Reply Quote
Profile Laurence
Project administrator
Project developer

Send message
Joined: 20 Jun 14
Posts: 336
Credit: 237,918
RAC: 0
Message 40470 - Posted: 15 Nov 2019, 12:55:32 UTC - in response to Message 40453.  

Although Theory native is a bit more efficient there are a lot of pros to focus on the vbox version.

I think I have found a solution where the scheduler will prefer the VBox version if you have it installed. If this machine keeps getting the VBox version, it means it worked. Is there away for someone to override this behavior on the client to so that you can request the native app even if you have VBox installed? I see there is a dont_use_vbox option in the cc_config.xml.


I think that I have found a solution. The projected_flops_scale attribute in the plan class can be used to favour one class over another. In the app_config.xml on the client, you can define an plan class to use. So on machines with out VBox they will get the native app. With VBox they will favour the VBox app and if you have VBox but want the native app, you can configure your client to do this using the following in the app_config.xml:

<app_config>
   <app_version>
       <app_name>Theory</app_name>
       <plan_class>native_theory</plan_class>
   </app_version>
</app_config>

ID: 40470 · Report as offensive     Reply Quote
Profile Laurence
Project administrator
Project developer

Send message
Joined: 20 Jun 14
Posts: 336
Credit: 237,918
RAC: 0
Message 40472 - Posted: 15 Nov 2019, 14:13:29 UTC - in response to Message 40470.  


I think that I have found a solution. The projected_flops_scale attribute in the plan class can be used to favour one class over another. In the app_config.xml on the client, you can define an plan class to use. So on machines with out VBox they will get the native app. With VBox they will favour the VBox app and if you have VBox but want the native app, you can configure your client to do this using the following in the app_config.xml:

<app_config>
   <app_version>
       <app_name>Theory</app_name>
       <plan_class>native_theory</plan_class>
   </app_version>
</app_config>


The app_config.xml solution didn't work for me. If anyone knows how to get the client to request that native version over the VBox version, please let me know.
ID: 40472 · Report as offensive     Reply Quote
Profile Laurence
Project administrator
Project developer

Send message
Joined: 20 Jun 14
Posts: 336
Credit: 237,918
RAC: 0
Message 40473 - Posted: 15 Nov 2019, 14:20:32 UTC - in response to Message 40472.  
Last modified: 15 Nov 2019, 14:20:50 UTC


The app_config.xml solution didn't work for me. If anyone knows how to get the client to request that native version over the VBox version, please let me know.

Setting dont_use_vbox in the cc_config.xml works but this affects all apps/projects.
<cc_config>
  <options>
    <dont_use_vbox>1</dont_use_vbox>
  </options>
</cc_config>

ID: 40473 · Report as offensive     Reply Quote
BITLab Argo

Send message
Joined: 16 Jul 05
Posts: 24
Credit: 35,251,537
RAC: 0
Message 40480 - Posted: 16 Nov 2019, 15:11:22 UTC - in response to Message 40473.  

The app_config.xml solution didn't work for me. If anyone knows how to get the client to request that native version over the VBox version, please let me know.

The Web GUI has a per-locale "Run test applications" tickbox (that Atlas already requires for its native application); this allows opt-in/opt-out of the native app but will have side-effects on other projects/sub-projects. Could you instead prefer the native app at the server but let users opt out, in which case they'd get the Vbox app (if they have Vbox installed)?

The thing is that the volunteers hit by my third bullet point are those for whom native apps won't run, e.g. because they can't get CVMFS to work. That might suggest that they are less technically able, so expecting them to hand-edit some obscure XML file seems a bit unfriendly!
ID: 40480 · Report as offensive     Reply Quote
1 · 2 · 3 · 4 · Next

Message boards : Theory Application : Move TheoryN Back Into Theory.


©2020 CERN