Message boards : ATLAS application : LHC@Home consolidation - ATLAS
Message board moderation

To post messages, you must log in.

1 · 2 · Next

AuthorMessage
David Cameron
Project administrator
Project developer
Project scientist

Send message
Joined: 13 May 14
Posts: 387
Credit: 15,314,184
RAC: 0
Message 27935 - Posted: 24 Nov 2016, 8:35:23 UTC

Dear all,

We are working on a plan to integrate ATLAS@Home into the consolidated LHC@Home project. Since ATLAS works in a slightly different way from the other LHC@Home apps it will be the last to be migrated, so please continue to crunch ATLAS tasks from the usual place for now. We will post more news here as it happens.

David for the ATLAS@Home team
ID: 27935 · Report as offensive     Reply Quote
Jim Wilkins

Send message
Joined: 22 Aug 06
Posts: 22
Credit: 466,060
RAC: 0
Message 28127 - Posted: 15 Dec 2016, 20:12:47 UTC - in response to Message 27935.  

I looked at the LHc apps and Atlas is in the collection of apps. So, am I able to run Atlas from LHC?

Thanks,
Jiim
ID: 28127 · Report as offensive     Reply Quote
Jim Wilkins

Send message
Joined: 22 Aug 06
Posts: 22
Credit: 466,060
RAC: 0
Message 28128 - Posted: 15 Dec 2016, 20:17:31 UTC - in response to Message 28127.  

Sorry. I looked at the the Applications page and it appears that Atlas has not been introduces yet.
ID: 28128 · Report as offensive     Reply Quote
Profile HerveUAE
Avatar

Send message
Joined: 18 Dec 16
Posts: 123
Credit: 37,495,365
RAC: 0
Message 28720 - Posted: 29 Jan 2017, 18:37:51 UTC

Anybody has an idea of when the consolidation of ATLAS@Home within LHC@Home is planned?
I would love to help in any of the on-going tests, but have not been getting any ATLAS test WUs yet.
We are the product of random evolution.
ID: 28720 · Report as offensive     Reply Quote
Profile Yeti
Volunteer moderator
Avatar

Send message
Joined: 2 Sep 04
Posts: 453
Credit: 193,369,412
RAC: 10,065
Message 28722 - Posted: 29 Jan 2017, 19:40:25 UTC - in response to Message 28720.  

Anybody has an idea of when the consolidation of ATLAS@Home within LHC@Home is planned?
I would love to help in any of the on-going tests, but have not been getting any ATLAS test WUs yet.

It is already ongoing, but there is a problem with the Server not getting the tasks out to interested users


Supporting BOINC, a great concept !
ID: 28722 · Report as offensive     Reply Quote
Profile HerveUAE
Avatar

Send message
Joined: 18 Dec 16
Posts: 123
Credit: 37,495,365
RAC: 0
Message 28724 - Posted: 30 Jan 2017, 4:08:55 UTC

It is already ongoing, but there is a problem with the Server not getting the tasks out to interested users

It explains why I have not received any task since configuring my LHC@Home preferences 5 days ago to receive ATLAS test applications.
So I guess there is no planned date yet for bringing ATLAS@Home down.
ID: 28724 · Report as offensive     Reply Quote
Profile Yeti
Volunteer moderator
Avatar

Send message
Joined: 2 Sep 04
Posts: 453
Credit: 193,369,412
RAC: 10,065
Message 28725 - Posted: 30 Jan 2017, 7:15:36 UTC - in response to Message 28724.  

It explains why I have not received any task since configuring my LHC@Home preferences 5 days ago to receive ATLAS test applications.

Did you set "Allow Beta-Applications" ?


Supporting BOINC, a great concept !
ID: 28725 · Report as offensive     Reply Quote
PHILIPPE

Send message
Joined: 24 Jul 16
Posts: 88
Credit: 239,917
RAC: 0
Message 28733 - Posted: 30 Jan 2017, 17:13:20 UTC - in response to Message 28725.  
Last modified: 30 Jan 2017, 18:02:31 UTC

I was in the same case of HerveUAE ,
but fortunately i received today one task (single core ? or 1-core multicore?),i had to abort it because my computer became unresponsive ...
i think it was 1-core multicore except if single core needs more memory in LHC than in ATLAS...

2017-01-30 12:47:28 (5928): vboxwrapper (7.7.26196): starting
2017-01-30 12:47:28 (5928): Status Report: Detected vboxsvc.exe. (PID = '3280')
2017-01-30 12:47:33 (5928): Feature: Checkpoint interval offset (409 seconds)
2017-01-30 12:47:33 (5928): Detected: VirtualBox COM Interface (Version: 5.1.14)
2017-01-30 12:47:33 (5928): Detected: Sandbox Configuration Enabled
2017-01-30 12:47:34 (5928): Detected: Minimum checkpoint interval (900.000000 seconds)
2017-01-30 12:47:34 (5928): Successfully copied 'init_data.xml' to the shared directory.
2017-01-30 12:47:35 (5928): Create VM. (boinc_2dc2b26f10baff2b, slot#4)
2017-01-30 12:47:35 (5928): Setting Memory Size for VM. (3300MB)
2017-01-30 12:47:35 (5928): Setting CPU Count for VM. (1)

set-up :
Run test application
Max # jobs 1
Max # CPUs 1
ID: 28733 · Report as offensive     Reply Quote
Profile HerveUAE
Avatar

Send message
Joined: 18 Dec 16
Posts: 123
Credit: 37,495,365
RAC: 0
Message 28734 - Posted: 30 Jan 2017, 17:28:17 UTC

I checked "Run test applications?" to Yes, which I guess is the same.
ID: 28734 · Report as offensive     Reply Quote
Profile Yeti
Volunteer moderator
Avatar

Send message
Joined: 2 Sep 04
Posts: 453
Credit: 193,369,412
RAC: 10,065
Message 28736 - Posted: 30 Jan 2017, 19:17:04 UTC - in response to Message 28734.  

I checked "Run test applications?" to Yes, which I guess is the same.

YES


Supporting BOINC, a great concept !
ID: 28736 · Report as offensive     Reply Quote
Terrible T

Send message
Joined: 1 Nov 05
Posts: 8
Credit: 597,196
RAC: 0
Message 28738 - Posted: 30 Jan 2017, 20:49:51 UTC

Got some 10core WU's, noticed they use a lot of memory, especially near the end of calculations, ~12GB ('in use' in Resource monitor). This on top of the VM memory (16GB ('Standby') in my case.

Also run time is appr. 50min, where the manager shows 17min before starting.

anybody similar numbers?
ID: 28738 · Report as offensive     Reply Quote
David Cameron
Project administrator
Project developer
Project scientist

Send message
Joined: 13 May 14
Posts: 387
Credit: 15,314,184
RAC: 0
Message 28740 - Posted: 30 Jan 2017, 21:30:37 UTC - in response to Message 28738.  

That is expected - the memory ATLAS tasks require is given by the formula 2.5GB + 0.8GB * ncores, i.e. 10.5GB for 10-cores.

Have you set Max # CPUs in your preferences? As noted in the other thread we plan to limit the max to 8 because more than that gives terrible performance.
ID: 28740 · Report as offensive     Reply Quote
Terrible T

Send message
Joined: 1 Nov 05
Posts: 8
Credit: 597,196
RAC: 0
Message 28743 - Posted: 30 Jan 2017, 23:20:37 UTC - in response to Message 28740.  

Yes had Max#CPU at 10 cores for first WU, then increased to 11.
re memory should I understand that the total required memory is
10.5GB for the VM's AND 10.5GB 'direct' memory?
Also noted WU progress slowing down , 70% at 20mins; 80% at 30 mins; and 90% 45mins.
Also noted only 1 core full used, the other 9 around 10-15%.
ID: 28743 · Report as offensive     Reply Quote
Profile Yeti
Volunteer moderator
Avatar

Send message
Joined: 2 Sep 04
Posts: 453
Credit: 193,369,412
RAC: 10,065
Message 28744 - Posted: 31 Jan 2017, 5:32:24 UTC - in response to Message 28743.  

re memory should I understand that the total required memory is
10.5GB for the VM's AND 10.5GB 'direct' memory?

Nope, it is 1x 10.5 GB


Supporting BOINC, a great concept !
ID: 28744 · Report as offensive     Reply Quote
PHILIPPE

Send message
Joined: 24 Jul 16
Posts: 88
Credit: 239,917
RAC: 0
Message 28748 - Posted: 31 Jan 2017, 16:40:33 UTC
Last modified: 31 Jan 2017, 16:42:08 UTC

Today i received 2 LHC ATLAS 1-core multicore.

I followed the starting of the 2 tasks with windows task manager to see how it makes my computer unresponsive.
I don't have enough ram to manage this multicore...

It seems that the criteria in calcul preference : no use more than 90% is not repected during the running.

the use of ram memory grows up to 50% to 97% before i aborted the tasks.
my host has only 4GBytes RAM memory.
By default , 3.3Bytes are allocated to the VM ,but it imply that the wu spreads over the 90%*4GB = 3.6GBytes because of the memory needed by the host even when it's iddle.
There's a conflict between these two parameters.
A test before launching the wu is maybe necessary to avoid the trouble.(disk use and memory use ~ 100%).
I don't have app_config ,
my set_up is the same as yesterday:
Run test application
Max # jobs 1
Max # CPUs 1
ID: 28748 · Report as offensive     Reply Quote
PHILIPPE

Send message
Joined: 24 Jul 16
Posts: 88
Credit: 239,917
RAC: 0
Message 28749 - Posted: 31 Jan 2017, 18:36:03 UTC - in response to Message 28748.  

Next wu , same behavior,

mouse more responsive but after 30 min ,computer out of control.

processor (80% global- 26% for LHC)
memory 97%
disk 100%

Under heavy load (97% use of memory),computer succeeded to perform during 30 min,but disk was too much used.

one-core multicore is not adapted for my host.(RAM memory ~3.7 GBytes)
Do you intend to do single core task as you did in ATLAS ?or is this feature given up to be more efficient in the process task of this project?
ID: 28749 · Report as offensive     Reply Quote
Profile HerveUAE
Avatar

Send message
Joined: 18 Dec 16
Posts: 123
Credit: 37,495,365
RAC: 0
Message 28750 - Posted: 31 Jan 2017, 18:50:48 UTC

one-core multicore is not adapted for my host.(RAM memory ~3.7 GBytes)

Philippe, you seem familiar with ATLAS@Home and might have seen this post on trying to run 1-core multi-core WUs with only 4Gbytes of RAM: http://atlasathome.cern.ch/forum_thread.php?id=640&postid=5922#5922
Others have tried, with difficulties as well.
ID: 28750 · Report as offensive     Reply Quote
David Cameron
Project administrator
Project developer
Project scientist

Send message
Joined: 13 May 14
Posts: 387
Credit: 15,314,184
RAC: 0
Message 28751 - Posted: 31 Jan 2017, 19:40:42 UTC - in response to Message 28750.  
Last modified: 31 Jan 2017, 19:42:22 UTC

I think the memory required for the current tasks is way over-estimated towards the lower end of number of cores. If I check completed tasks the average peak memory for a 1-core WU is around 1.7GB and never goes above 2GB.

I've changed the formula to 1.4GB + 0.8 * ncores, so a 1-core WU will require 2.2GB of RAM and this should work in a 4GB PC. I'll monitor for memory failures in case this new formula is too small. Let me know how you get on.

Do you intend to do single core task as you did in ATLAS ?or is this feature given up to be more efficient in the process task of this project?


I would prefer to keep one app to make things simpler but make sure that this app can run ok for people who only want to run single core.
ID: 28751 · Report as offensive     Reply Quote
PHILIPPE

Send message
Joined: 24 Jul 16
Posts: 88
Credit: 239,917
RAC: 0
Message 28753 - Posted: 31 Jan 2017, 20:57:07 UTC - in response to Message 28751.  

Thanks for your answer , David , i imagine you are very busy ,with this consolidation.
So ,for this new trial ,the computer freezed after 15 min.
I shutdown and reboot ,after 15 min exactly the same story:
Windows task manager stop , and i hear the disk continiously being active ,without any possibility to click or type on the keyboard.

In ATLAS, i succeeded to run multicore by using an app_config.xml.
But i understood, reducing the amount of memory allocated to the wu (2500 instead of 3300), i allowed the wu to begin the run in the area of 2500 MBytes RAM.
But there was a risk , because if the wu needed more memory to be finished,the wu used other part of ram memory, shared by the other process.Thus sometimes i noticed wu were validated with an error because of internal conflicts (and hours of processing were lost unfortunately).

So i think you can't go down more than a certain limit.You have to guarantee to all the crunchers who intend to run these tasks a good probability to end correctly.(even if we are aware that each wu is different and behave ramdomsly).

Do your best but without taking too much risk.Being in LHC now ,allow people to choose easier tasks, adapted to their hardware configuration and their wishes.

Yes HerveUAE , i know this thread ,i posted in it ,life is not always simple,and science experimental also.But each day we hope we advance.This is the most important...
ID: 28753 · Report as offensive     Reply Quote
Profile Yeti
Volunteer moderator
Avatar

Send message
Joined: 2 Sep 04
Posts: 453
Credit: 193,369,412
RAC: 10,065
Message 28755 - Posted: 31 Jan 2017, 21:11:02 UTC - in response to Message 28751.  

I would prefer to keep one app to make things simpler but make sure that this app can run ok for people who only want to run single core.

Please remember that MultiCore-WUs do not work on all PCs.

I have a PC that is doing fine with SingleCore, but every MultiCoreWU is interrupted with an error that no one could explain.

And I know there are more PCs out that can only run SingleCoreWUs.

So this leads to these possibilities:

  1. Ignore PCs that can't run MultiCoreWUs.
  2. Help to find what is real the problem, but this will be Time-Consuming




Supporting BOINC, a great concept !
ID: 28755 · Report as offensive     Reply Quote
1 · 2 · Next

Message boards : ATLAS application : LHC@Home consolidation - ATLAS


©2024 CERN