Message boards : ATLAS application : Atlas Simulation 1.01 (Vbox64) will not finish
Message board moderation

To post messages, you must log in.

Previous · 1 · 2 · 3

AuthorMessage
computezrmle
Volunteer moderator
Volunteer developer
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 15 Jun 08
Posts: 2401
Credit: 225,502,623
RAC: 124,995
Message 38199 - Posted: 9 Mar 2019, 23:14:02 UTC - in response to Message 38193.  

Interesting..but wouldn't that cause a conflict of resources (cpu) with ATLAS needing all the cores I allow BOINC to have and the other projects also wanting to use the total amount of cores I have allocated at the same time?

Simply limit the resources (here: CPUs) for each client.
Have you ever looked through the menu options of your BOINC client or read the manual?
https://boinc.berkeley.edu/wiki/Client_configuration
ID: 38199 · Report as offensive     Reply Quote
greg_be

Send message
Joined: 28 Dec 08
Posts: 318
Credit: 4,203,902
RAC: 3,788
Message 38200 - Posted: 10 Mar 2019, 0:29:37 UTC - in response to Message 38199.  

Interesting..but wouldn't that cause a conflict of resources (cpu) with ATLAS needing all the cores I allow BOINC to have and the other projects also wanting to use the total amount of cores I have allocated at the same time?

Simply limit the resources (here: CPUs) for each client.
Have you ever looked through the menu options of your BOINC client or read the manual?
https://boinc.berkeley.edu/wiki/Client_configuration


It's 1am here in the EU now. I will read more about this stuff when I get up.
When I started, I had a low end PC and Rosetta was my only project, so there was no need to do stuff like what is shown on that page.
Only in the past few years have I upgraded to stronger and more complex CPU's and projects and never needed to do stuff like what we talk about now.
So this is all new to me, because I have never "tweaked" BOINC before.
ID: 38200 · Report as offensive     Reply Quote
greg_be

Send message
Joined: 28 Dec 08
Posts: 318
Credit: 4,203,902
RAC: 3,788
Message 38201 - Posted: 10 Mar 2019, 0:31:23 UTC - in response to Message 38198.  

... native ATLAS is the only CPU job running).
So I suspended all three that were running.

Independent from BOINC.
ATLAS native must not be suspended/resumed as it will always start from the scratch.
David Cameron explained somewhere (don't find it ATM) that this is by design of the scientific app.

If you run ATLAS native inside your own VM, then you may suspend the VM instead.


What is the definition of "your own" VM? My Windows ATLAS stuff runs the "external" VM. I can look at the VM info via the application itself on the desktop.
ID: 38201 · Report as offensive     Reply Quote
greg_be

Send message
Joined: 28 Dec 08
Posts: 318
Credit: 4,203,902
RAC: 3,788
Message 38202 - Posted: 10 Mar 2019, 0:39:05 UTC

Update

Still running first task. 10:21 into it now. 95.13%
1 ATLAS second takes around 10 real time seconds now.
Still 31:40 to go.
My guess based on what I hope is the correct calculations...30 mins x 60 to get seconds and then multiplied by 10 to get what "real" time seconds are involved and then dividing by 60 to minutes and 60 again to get hours, says that it will be about another 5 hours.

So it looks like about a multiplication factor of 5 on estimated time. IE 3:25 is really about 15.5 hrs more or less. That's pretty wild. I knew ATLAS was a time hog and memory hog, but wow! 15 hrs!
ID: 38202 · Report as offensive     Reply Quote
Jim1348

Send message
Joined: 15 Nov 14
Posts: 602
Credit: 24,371,321
RAC: 0
Message 38203 - Posted: 10 Mar 2019, 0:49:57 UTC - in response to Message 38198.  

ATLAS native must not be suspended/resumed as it will always start from the scratch.
David Cameron explained somewhere (don't find it ATM) that this is by design of the scientific app..

Possibly so. I did not check to see what was going on in detail. However, they uploaded immediately after I resumed them, so I suspect they just did not suspend at all.
ID: 38203 · Report as offensive     Reply Quote
Jonathan

Send message
Joined: 25 Sep 17
Posts: 99
Credit: 3,233,696
RAC: 5,560
Message 38204 - Posted: 10 Mar 2019, 2:43:08 UTC - in response to Message 38202.  

greg_be, how many cores is the Atlas task using that you are talking about above? I have a Ryzen 1700 and it is completing Atlas tasks in under two hours. My time estimate is currently 1:41 for upcoming work units. I wouldn't expect your time to completing to be that far off from mine.
You can view how many processors are assigned to the virtual machine and how much ram is assigned by opening Virtual Box, clicking on the running virtual item and looking at what is reported under the system section of the window.

If you click on the green, show arrow, it will bring up the running virtual machines terminal window. If it is blank, I just hit the Shift key to wake it up. The first terminal window you can't log into but you hit Alt+F2 to watch the events information and Alt+F3 to see TOP related information like processor and memory usage. You would select Machine then Detach GUI to close the running virtual machine window. The machine is still running and this only closes the terminal view window.
ID: 38204 · Report as offensive     Reply Quote
greg_be

Send message
Joined: 28 Dec 08
Posts: 318
Credit: 4,203,902
RAC: 3,788
Message 38206 - Posted: 10 Mar 2019, 9:16:11 UTC - in response to Message 38204.  
Last modified: 10 Mar 2019, 9:20:43 UTC

greg_be, how many cores is the Atlas task using that you are talking about above? I have a Ryzen 1700 and it is completing Atlas tasks in under two hours. My time estimate is currently 1:41 for upcoming work units. I wouldn't expect your time to completing to be that far off from mine.
You can view how many processors are assigned to the virtual machine and how much ram is assigned by opening Virtual Box, clicking on the running virtual item and looking at what is reported under the system section of the window.

If you click on the green, show arrow, it will bring up the running virtual machines terminal window. If it is blank, I just hit the Shift key to wake it up. The first terminal window you can't log into but you hit Alt+F2 to watch the events information and Alt+F3 to see TOP related information like processor and memory usage. You would select Machine then Detach GUI to close the running virtual machine window. The machine is still running and this only closes the terminal view window.



You got to be kidding1 your trying to compare your machine to mine? You have double the computing power of me. 8 cores 16 thread vs 4 cores and 8 threads. I am only .2 GHZ faster than you. But your double my capacity in computing power! Your speed rating according to Userbenchmark is 59/1147 vs my 178/1147. That's like comparing a street racer car to a factory car. Just no comparison.

Due to overheat concerns (long story) I am restricting my system to 95% capacity and 90% run time on the cpu's and 7 cores. I would guess 4 physical and 3 threads or the other way around. Memory is 95% of total memory in BOINC and VM is using 9300MB (default setting)

Now the other thing to note, that task last night was suspended by BOINC to run other tasks, so that appears to have affected the way it was working. I am on a new task, uninterrupted and 6:25 into it at 84.7% with a theorized 1:09 to finish.

The task is running at 5-8 real time seconds per time remaining second.
ID: 38206 · Report as offensive     Reply Quote
Jonathan

Send message
Joined: 25 Sep 17
Posts: 99
Credit: 3,233,696
RAC: 5,560
Message 38207 - Posted: 10 Mar 2019, 10:30:33 UTC - in response to Message 38206.  

You misunderstand how the the Atlas tasks work with multiple cores. Did you take a look at post 38191 above by computezrmie. Here is a link to a Ryzen 1400 with completed tasks. Look at the run times. Do you have more processors assigned to running virtual machines than you actually have?

https://lhcathome.cern.ch/lhcathome/results.php?hostid=10570592&offset=0&show_names=0&state=0&appid=14
ID: 38207 · Report as offensive     Reply Quote
greg_be

Send message
Joined: 28 Dec 08
Posts: 318
Credit: 4,203,902
RAC: 3,788
Message 38210 - Posted: 10 Mar 2019, 13:31:59 UTC - in response to Message 38207.  

You misunderstand how the the Atlas tasks work with multiple cores. Did you take a look at post 38191 above by computezrmie. Here is a link to a Ryzen 1400 with completed tasks. Look at the run times. Do you have more processors assigned to running virtual machines than you actually have?

https://lhcathome.cern.ch/lhcathome/results.php?hostid=10570592&offset=0&show_names=0&state=0&appid=14



I'll get back to you on that. I am a bit distracted with other things right now and fighting off a nasty coughing round. Hard to focus on really tech stuff right now.
ID: 38210 · Report as offensive     Reply Quote
greg_be

Send message
Joined: 28 Dec 08
Posts: 318
Credit: 4,203,902
RAC: 3,788
Message 38212 - Posted: 10 Mar 2019, 16:41:51 UTC - in response to Message 38210.  
Last modified: 10 Mar 2019, 16:44:06 UTC

You misunderstand how the the Atlas tasks work with multiple cores. Did you take a look at post 38191 above by computezrmie. Here is a link to a Ryzen 1400 with completed tasks. Look at the run times. Do you have more processors assigned to running virtual machines than you actually have?

https://lhcathome.cern.ch/lhcathome/results.php?hostid=10570592&offset=0&show_names=0&state=0&appid=14



I'll get back to you on that. I am a bit distracted with other things right now and fighting off a nasty coughing round. Hard to focus on really tech stuff right now.



The computer you are comparing me to while the CPU is the same. The FPS on that machine is greater than mine, which I think means hes OC'ing the machine. I can't do that because of Rosetta and a few other projects are sensitive to OC. The FPS on that machine is Measured floating point speed 3.91 billion ops/sec and the Measured integer speed is 14.39 billion ops/sec.

My machine: Measured floating point speed 1.17 billion ops/sec
Measured integer speed 6.2 billion ops/sec

So again, you can't compare the two machines rate of processing, because one is tweaked and the other is on default. AND....you can not tell if they are running all 8 or 7 or whatever threads. I have 8 "cores" and am using only 7. That person has 8 and maybe using 8 and has tweaked the speed of the chip.
ID: 38212 · Report as offensive     Reply Quote
greg_be

Send message
Joined: 28 Dec 08
Posts: 318
Credit: 4,203,902
RAC: 3,788
Message 38213 - Posted: 10 Mar 2019, 16:53:03 UTC
Last modified: 10 Mar 2019, 16:56:17 UTC

Looking back at my "completed" tasks, I see that they average typically 7-8,000 seconds run time and CPU time is in the 40,000 range. That was on the old ones. The new stuff is also averaging in on that same time frame.

So now I have to look back and find that section about customizing ATLAS.
As someone has pointed out in another thread....and it seems to be the case, ATLAS has to run uninterrupted. My problems with completing the tasks came from the interruptions and possibly the AV program.
ID: 38213 · Report as offensive     Reply Quote
greg_be

Send message
Joined: 28 Dec 08
Posts: 318
Credit: 4,203,902
RAC: 3,788
Message 38214 - Posted: 10 Mar 2019, 17:04:34 UTC - in response to Message 38192.  

... Then maybe I could dedicate a second host to exclusive ATLAS tasks.

Why not just a second BOINC client on the same host?
There are lot's of howtos around.
To find them you may ask your favorite search engine.



I can't deal with this right now since I have ATLAS tasks already lined up to process. I will just keep doing like I am doing now and let them run out with no interference from other programs. When I have time next weekend, I will look at doing this. My weekdays are very long with the middle of the week being the worst. (11 hrs onsite).

But thanks to you and the others I understand better what is going on with ATLAS and why my machine was taking so long to process them.
ID: 38214 · Report as offensive     Reply Quote
bronco

Send message
Joined: 13 Apr 18
Posts: 443
Credit: 8,438,885
RAC: 0
Message 38215 - Posted: 10 Mar 2019, 18:51:14 UTC - in response to Message 38213.  

Looking back at my "completed" tasks, I see that they average typically 7-8,000 seconds run time and CPU time is in the 40,000 range. That was on the old ones. The new stuff is also averaging in on that same time frame.

So now I have to look back and find that section about customizing ATLAS.
As someone has pointed out in another thread....and it seems to be the case, ATLAS has to run uninterrupted. My problems with completing the tasks came from the interruptions and possibly the AV program.
I was one of two who told you ATLAS VBox tasks reset the event counter to zero if suspended and should therefore be run uninterrupted. I was wrong. As computezrmie pointed out, only ATLAS native tasks reset the event counter, ATLAS VBox does not. My apologies for feeding you that misinformation. For my penance I will be forfeiting yet another barrel of our finest Alberta dilbit.

I do maintain you'll have less trouble from ATLAS VBox tasks if you run them uninterrupted though I suspect others will point out that they pause them regularly with no ill side effects.
ID: 38215 · Report as offensive     Reply Quote
greg_be

Send message
Joined: 28 Dec 08
Posts: 318
Credit: 4,203,902
RAC: 3,788
Message 38216 - Posted: 10 Mar 2019, 20:57:30 UTC - in response to Message 38215.  
Last modified: 10 Mar 2019, 20:58:03 UTC

Looking back at my "completed" tasks, I see that they average typically 7-8,000 seconds run time and CPU time is in the 40,000 range. That was on the old ones. The new stuff is also averaging in on that same time frame.

So now I have to look back and find that section about customizing ATLAS.
As someone has pointed out in another thread....and it seems to be the case, ATLAS has to run uninterrupted. My problems with completing the tasks came from the interruptions and possibly the AV program.
I was one of two who told you ATLAS VBox tasks reset the event counter to zero if suspended and should therefore be run uninterrupted. I was wrong. As computezrmie pointed out, only ATLAS native tasks reset the event counter, ATLAS VBox does not. My apologies for feeding you that misinformation. For my penance I will be forfeiting yet another barrel of our finest Alberta dilbit.

I do maintain you'll have less trouble from ATLAS VBox tasks if you run them uninterrupted though I suspect others will point out that they pause them regularly with no ill side effects.


I'll take your barrel of dilbit! Got toss in a half case of Maple Syrup as well.
Well for my machine, the paused unit took 15 hrs to complete and some others that were paused regularly by BOINC either caused the problem I started with here or took over 36hrs to complete.
Untouched units are taking 9-10 to finish. They slow down at the end once in the 80% range.
Then its 7-8 realtime seconds to every BOINC second. That is consistent with ATLAS on my system.
I disabled further ATLAS tasks until I can get my head around how to create a separate client and make directories. I'll burn through what I have in queue now to get my points back up to where they should be.
Then its back to regular computing once ATLAS is gone.
ID: 38216 · Report as offensive     Reply Quote
greg_be

Send message
Joined: 28 Dec 08
Posts: 318
Credit: 4,203,902
RAC: 3,788
Message 38221 - Posted: 11 Mar 2019, 18:46:03 UTC

Something whacky happened..I don't know how another CPU project got activated, but it did.
I shut it down and got ATLAS back and running, everything seemed ok for awhile, but now at 90 some odd percent, while BOINC still counts off the time and the percent and the usual sluggish remaining time vs real time is happening, the log has not update with new target information for 20 mins now.
Not sure what happened. The log refreshes, but no new information comes through.
ID: 38221 · Report as offensive     Reply Quote
greg_be

Send message
Joined: 28 Dec 08
Posts: 318
Credit: 4,203,902
RAC: 3,788
Message 38224 - Posted: 11 Mar 2019, 22:59:28 UTC - in response to Message 38221.  

Something whacky happened..I don't know how another CPU project got activated, but it did.
I shut it down and got ATLAS back and running, everything seemed ok for awhile, but now at 90 some odd percent, while BOINC still counts off the time and the percent and the usual sluggish remaining time vs real time is happening, the log has not update with new target information for 20 mins now.
Not sure what happened. The log refreshes, but no new information comes through.


Note: Due to the interruption by another project switching from GPU to CPU (damn I need to get that second host going, but no time right now) this task I am running now will run another 3hrs before finishing.
Looks like log file caught up finally.

Anyway....dropped ATLAS for now until I can get this host thing set up. All remaining tasks will be finished (2 more).
ID: 38224 · Report as offensive     Reply Quote
mmonnin

Send message
Joined: 22 Mar 17
Posts: 55
Credit: 10,223,976
RAC: 309
Message 38227 - Posted: 12 Mar 2019, 11:40:12 UTC - in response to Message 38224.  
Last modified: 12 Mar 2019, 11:40:28 UTC

Something whacky happened..I don't know how another CPU project got activated, but it did.
I shut it down and got ATLAS back and running, everything seemed ok for awhile, but now at 90 some odd percent, while BOINC still counts off the time and the percent and the usual sluggish remaining time vs real time is happening, the log has not update with new target information for 20 mins now.
Not sure what happened. The log refreshes, but no new information comes through.


Note: Due to the interruption by another project switching from GPU to CPU (damn I need to get that second host going, but no time right now) this task I am running now will run another 3hrs before finishing.
Looks like log file caught up finally.

Anyway....dropped ATLAS for now until I can get this host thing set up. All remaining tasks will be finished (2 more).


I run a separate GPU BOINC client on all my PCs.
https://www.overclock.net/forum/18056-boinc-guides-tutorials/1628924-guide-setting-up-multiple-boinc-instances.html
ID: 38227 · Report as offensive     Reply Quote
Previous · 1 · 2 · 3

Message boards : ATLAS application : Atlas Simulation 1.01 (Vbox64) will not finish


©2024 CERN