Message boards : Number crunching : Projects use very little virtual memory?
Message board moderation

To post messages, you must log in.

1 · 2 · Next

AuthorMessage
Profile meckano
Avatar

Send message
Joined: 17 Sep 04
Posts: 150
Credit: 20,315
RAC: 0
Message 11248 - Posted: 7 Nov 2005, 1:30:58 UTC
Last modified: 7 Nov 2005, 1:33:43 UTC

I changed from 0% (zero%) to 50%,
and did not find much of a difference.

The only project with constant result times are einstein though.
- mine are always between 21 000 and 21 200, except for one which was 21 333.33
- if you check my times there, the first one with the change is Nov.6'05
-----------------------
Click to see my tag
My tag
SNAFU'ed? Turn the Page! :D
ID: 11248 · Report as offensive     Reply Quote
Travis DJ

Send message
Joined: 29 Sep 04
Posts: 196
Credit: 207,040
RAC: 0
Message 11254 - Posted: 7 Nov 2005, 7:56:40 UTC - in response to Message 11248.  
Last modified: 7 Nov 2005, 7:57:37 UTC

Keep in mind, Virtual Memory, while being a good thing which prevents applications from running out of "RAM" when needed, is a BadThing™ when it comes to actual performance. You really don't want to have insufficient physical RAM which would make LHC (or other projects) revert to Virtual Memory to compensate. Both your computers should be fine - 256-512MB is good enough for someone who isn't doing a lot of gaming, programming, or running several large programs at once. :)

(BBCode doesn't handle extended characters very well .. grr .. that was supposed to be a (TM) .. )

ID: 11254 · Report as offensive     Reply Quote
Profile Doug Worrall

Send message
Joined: 13 Jul 05
Posts: 7
Credit: 9,163
RAC: 0
Message 11261 - Posted: 7 Nov 2005, 23:20:07 UTC - in response to Message 11254.  

Keep in mind, Virtual Memory, while being a good thing which prevents applications from running out of "RAM" when needed, is a BadThing™ when it comes to actual performance. You really don't want to have insufficient physical RAM which would make LHC (or other projects) revert to Virtual Memory to compensate. Both your computers should be fine - 256-512MB is good enough for someone who isn't doing a lot of gaming, programming, or running several large programs at once. :)

(BBCode doesn't handle extended characters very well .. grr .. that was supposed to be a (TM) .. )

Thanks Travis,
This answered another quetion I had.
Happy Crunching
Doug

ID: 11261 · Report as offensive     Reply Quote
Profile meckano
Avatar

Send message
Joined: 17 Sep 04
Posts: 150
Credit: 20,315
RAC: 0
Message 11263 - Posted: 7 Nov 2005, 23:34:30 UTC - in response to Message 11261.  
Last modified: 7 Nov 2005, 23:34:58 UTC

So I shall assume that virtual memory is only used when:
1) not enough memory is available. I guess that should be stated somewhere by Boinc, addit: as it sounds important.
2) when wu's are pre-empted to run whatever else you use the computer for.

Mind you, virtual memory and a save are both on the hard drive, so I don't get the whole picture here.
-----------------------
Click to see my tag
My tag
SNAFU'ed? Turn the Page! :D
ID: 11263 · Report as offensive     Reply Quote
Travis DJ

Send message
Joined: 29 Sep 04
Posts: 196
Credit: 207,040
RAC: 0
Message 11266 - Posted: 8 Nov 2005, 1:51:01 UTC
Last modified: 8 Nov 2005, 1:56:39 UTC

To answer your questions:

1) Yes. WinNT/2K/XP Task manager can tell you how much is allocated, but not necessarily if it's "In Use"
2) I believe when a project is preempted, it is put on "pause," a checkpoint is written to the hard drive, then removed from memory. (Your user settings per project also determine this behavior, i.e. "Leave project in memory while preempted")

Memory in a 32-bit WindowsNT-Based Enviornment, 101:

All 32-bit CPUs have a maximum addressable memory of 4GB.
All 32-bit Windows NT-Based Operating systems can address up to 4GB RAM.
(Windows 9x/Me can only support 512MB and is not discussed here)
It can be "Physical" memory such as RAM, or...
It can be "Virtual" memory as in any storage space used to save live data.
Your PC must boot using Physical memory/You can't boot without Physical RAM.

It is possible for a WinXP computer with 256MB Physical RAM to run an application which requires much larger amounts of "memory", up to 4GB. It does so by using a "Swapfile" where Pages of memory in RAM are sent to another device temporarily to make room for other data to be worked on. The most logical device for this purpose is your hard drive. Naturally since your HDD is extremely slow compared to RAM, your performance degrades dramatically the more Virtual Memory (beyond what is Physically available) has to be used.

What happens with BOINC if projects are allowed to be left in memory while preempted, upon RAM running low Windows will remove the least used Pages from RAM and move it to the hard drive's swapfile. When those pages are needed once more for recall and processing, they're transferred back to RAM where it works most efficiently. What's key to understand is the swapfile is a logical extention of "memory" and that's how it differs from a checkpoint. Virtual memory enables multiple apps to be loaded simultaneously and assures memory won't run out, up to 4GB. The apps run in their own virtual environment separate from all other apps; each sees its own area of memory as if it were the only program running on the PC. Windows does all the hard work so apps (or the user for that matter) don't have to.

This process applies to all applications (including .dll files, services, others) in WindowsNT based OSes. There's more to it and I'm sure you could Google up a whitepaper on Virtual Memory if you really really want to know the innerds of it. Hopefully this explains for you.. if not it was a great test of my knowledge. :)



ID: 11266 · Report as offensive     Reply Quote
Profile meckano
Avatar

Send message
Joined: 17 Sep 04
Posts: 150
Credit: 20,315
RAC: 0
Message 11268 - Posted: 8 Nov 2005, 2:06:56 UTC - in response to Message 11266.  
Last modified: 8 Nov 2005, 2:40:28 UTC

Wow! Nice answer, thank you!

I'm going to 100% the virtual on the 256M memory, win98.

Appreciated! :)

addit: well, while I was into it, I changed the mult. from 15 to 17 ;) Now clocking higher than my 3200+/errr-oc'ed-2500+ again.

addit2: 17 too high :( down to 16.5 :) (oooo the heat lol)
-----------------------
Click to see my tag
My tag
SNAFU'ed? Turn the Page! :D
ID: 11268 · Report as offensive     Reply Quote
Travis DJ

Send message
Joined: 29 Sep 04
Posts: 196
Credit: 207,040
RAC: 0
Message 11282 - Posted: 9 Nov 2005, 3:15:12 UTC - in response to Message 11268.  
Last modified: 9 Nov 2005, 3:15:41 UTC

EEK! Win98? hehe

It's mostly the same.. :) Happy crunching. Be careful about OC.. if you push too hard your client could return results which the validator may not accept due to errors. See the Boinc Wiki about overclocking.
ID: 11282 · Report as offensive     Reply Quote
Profile meckano
Avatar

Send message
Joined: 17 Sep 04
Posts: 150
Credit: 20,315
RAC: 0
Message 11293 - Posted: 9 Nov 2005, 14:45:16 UTC - in response to Message 11282.  

Thanks.
This box is winxp with 2500 as 3200 and has been for years.
:)
-----------------------
Click to see my tag
My tag
SNAFU'ed? Turn the Page! :D
ID: 11293 · Report as offensive     Reply Quote
Profile meckano
Avatar

Send message
Joined: 17 Sep 04
Posts: 150
Credit: 20,315
RAC: 0
Message 11295 - Posted: 9 Nov 2005, 17:25:13 UTC - in response to Message 11293.  
Last modified: 9 Nov 2005, 17:25:27 UTC


I didn't find anything about o-c'ing, except 3 questions with no write-ups.
-----------------------
Click to see my tag
My tag
SNAFU'ed? Turn the Page! :D
ID: 11295 · Report as offensive     Reply Quote
Profile meckano
Avatar

Send message
Joined: 17 Sep 04
Posts: 150
Credit: 20,315
RAC: 0
Message 11296 - Posted: 9 Nov 2005, 18:41:45 UTC - in response to Message 11295.  

The errors that are seen on the 2400 were at startup and cpu was not oc'ed at time. I have no idea why they're there, but all good since then. All other projects have no errors.
-----------------------
Click to see my tag
My tag
SNAFU'ed? Turn the Page! :D
ID: 11296 · Report as offensive     Reply Quote
Profile meckano
Avatar

Send message
Joined: 17 Sep 04
Posts: 150
Credit: 20,315
RAC: 0
Message 11297 - Posted: 9 Nov 2005, 19:50:13 UTC - in response to Message 11296.  
Last modified: 9 Nov 2005, 19:56:27 UTC

Wow, some incredible variances while adjusting these settings.

I'm going to test 3 settings on 4 projects:
1) Leave in memory while pre-empted...
- I shall probably end up with this always at 'no' as pre-test thoughts are pointing towards a faster wu with vitual memory at Zero. And
a) I don't want my ram full
b) It should only affect project switch time and not wu processing time.
- some info may be lost during this, I doubt it / and hope not.

2) Write to disk every...
- with virtual memory at zero, this may cause ram to fill up, so shorter may be better; solong as virt.mem. at 0% turns out to be the 'cats meow'.

3) Virtual memory...
- will try 0%, 100%, then 25% if 0% is faster than 100%, or 75% if 100% is the faster of the two.(zero seems to be the winner so far.)


Are there other settings you have thoughts about trying?
I would really like to see project engineers put more testing into this as so many people have 'puters just for Boinc'ing, and we could get alot more work done. It may be coincidence but one project went up by 50% by changing from 0% to 100%, on the xp box with 512m ram(400mhz) bus/cpu 400mhz, but I'm doing tests from scratch now.

addit: re: a previous post:
atlest 2 projects of my 4 had startup probs.
-----------------------
Click to see my tag
My tag
SNAFU'ed? Turn the Page! :D
ID: 11297 · Report as offensive     Reply Quote
Travis DJ

Send message
Joined: 29 Sep 04
Posts: 196
Credit: 207,040
RAC: 0
Message 11299 - Posted: 9 Nov 2005, 20:15:00 UTC - in response to Message 11297.  

In most cases the default settings are best.

1) Use "no", if other apps need ram where that application is sitting there and not inuse, it will be paged out to the swapfile.
a) By the time swapping occurs your ram would be "full" anyway.
b) While True, if there isn't enough room in RAM to load the processing app and all it does is run from the swapfile, your crunch time would worsen exponentially.

2) 60 seconds is nice (if the project supports it) .. so if a crash occurred you'd only lose 60 seconds' processing time. You have the wrong idea here.

3) Messing with that percentage can adversely affect other applications on your PC. Again, stick with the defaults.

Meckano-- you're not going to gain any appreciable amount of performance by messing with virtual memory settings and are more likely to slow your system down by using a lower setting. The entire purpose of VM is to eliminate the problem of apps running out of memory by intelligent memory control from the OS, not to make your PC run faster. The startup problems you mention may be related to messing with the default settings (or with Windows' VM settings). The defaults are specified by either the developers or the "Boinc Default".

Best practices:
1) Don't leave apps in memory when preempted. They're done. Let 'em unload!!
2) Leave the VM settings alone, if the allocation is too low, BOINC applications will consume *more* RAM, won't swap out when the app needs to get out the way for other more important tasks, and slow your overall performance/responsiveness of your PC. The default is just fine.
3) ..and I stress.. Less is better
ID: 11299 · Report as offensive     Reply Quote
Profile meckano
Avatar

Send message
Joined: 17 Sep 04
Posts: 150
Credit: 20,315
RAC: 0
Message 11300 - Posted: 9 Nov 2005, 20:32:05 UTC - in response to Message 11299.  

Test in progress, just starting.

as for:
Best practices:
1) Don't leave apps in memory when preempted. They're done. Let 'em unload!!
2) Leave the VM settings alone, if the allocation is too low, BOINC applications will consume *more* RAM, won't swap out when the app needs to get out the way for other more important tasks, and slow your overall performance/responsiveness of your PC. The default is just fine.
3) ..and I stress.. Less is better


1) preempted does not mean done, it means paused.
2) I have possibly witnessed a huge difference, hence the tests.
3) you agree, less is the way. But unless ram is less than x. Hence the tests.

Thanks for input.
-----------------------
Click to see my tag
My tag
SNAFU'ed? Turn the Page! :D
ID: 11300 · Report as offensive     Reply Quote
Profile davidmcw
Avatar

Send message
Joined: 18 Sep 04
Posts: 17
Credit: 53,515
RAC: 0
Message 11302 - Posted: 10 Nov 2005, 5:33:55 UTC

I had read of someone suggest that you leave the app in memory when running the CPDN project, as there is a chance it may not checkpoint within the std one hour period.
ID: 11302 · Report as offensive     Reply Quote
Gaspode the UnDressed

Send message
Joined: 1 Sep 04
Posts: 506
Credit: 118,619
RAC: 0
Message 11303 - Posted: 10 Nov 2005, 8:15:28 UTC - in response to Message 11302.  

I had read of someone suggest that you leave the app in memory when running the CPDN project, as there is a chance it may not checkpoint within the std one hour period.


The version five client keeps an application in memory until it has written a new checkpoint regardless of preference settings precisely to avoid this problem.

Gaspode the UnDressed
http://www.littlevale.co.uk
ID: 11303 · Report as offensive     Reply Quote
Profile meckano
Avatar

Send message
Joined: 17 Sep 04
Posts: 150
Credit: 20,315
RAC: 0
Message 11306 - Posted: 10 Nov 2005, 12:52:09 UTC - in response to Message 11303.  

Now we're cookin'!
great input :)
-----------------------
Click to see my tag
My tag
SNAFU'ed? Turn the Page! :D
ID: 11306 · Report as offensive     Reply Quote
Profile meckano
Avatar

Send message
Joined: 17 Sep 04
Posts: 150
Credit: 20,315
RAC: 0
Message 11308 - Posted: 10 Nov 2005, 13:07:03 UTC - in response to Message 11297.  
Last modified: 10 Nov 2005, 13:07:58 UTC


1) Leave in memory while pre-empted...
- I shall probably end up with this always at 'no' as pre-test thoughts are pointing towards a faster wu with vitual memory at Zero. And
a) I don't want my ram full
b) It should only affect project switch time and not wu processing time.
- some info may be lost during this, I doubt it / and hope not.

2) Write to disk every...
- with virtual memory at zero, this may cause ram to fill up, so shorter may be better; solong as virt.mem. at 0% turns out to be the 'cats meow'.

3) Virtual memory...
- will try 0%, 100%, then 25% if 0% is faster than 100%, or 75% if 100% is the faster of the two.(zero seems to be the winner so far.)


ohhh, have no time right now, so:
1) no
2) 30 secs
3) 0%

addit:
just really baffled by this virtual memory setting as MS win. never gave us that option.
-----------------------
Click to see my tag
My tag
SNAFU'ed? Turn the Page! :D
ID: 11308 · Report as offensive     Reply Quote
Profile meckano
Avatar

Send message
Joined: 17 Sep 04
Posts: 150
Credit: 20,315
RAC: 0
Message 11330 - Posted: 11 Nov 2005, 21:37:53 UTC - in response to Message 11308.  


1) no
2) 30 secs
3) 0%


LHC seems to be affected by the above settings, and maybe seti too.
Switching to:
yes
30 secs
50%
-----------------------
Click to see my tag
My tag
SNAFU'ed? Turn the Page! :D
ID: 11330 · Report as offensive     Reply Quote
River~~

Send message
Joined: 13 Jul 05
Posts: 456
Credit: 75,142
RAC: 0
Message 11349 - Posted: 14 Nov 2005, 15:12:45 UTC - in response to Message 11299.  
Last modified: 14 Nov 2005, 15:15:37 UTC

...Best practices:
1) Don't leave apps in memory when preempted. They're done. Let 'em unload!!


One partial exception to this is if you use the setting to stop BOINC when the machine is in use. If so it is worth seeing what happens if you allow the app to stay in memory. If the machine seems responsive enough, fine. If the machine takes annoyingly long to react when you first hit the keyboard after a while away, then set it back to "Don't keep apps in memory".

Reason:

1. if you go away to make some coffee and come back before a checkpoint, then the work BOINC did will be wasted unless the app stays in memory. This can be a reason to keep apps in memory, especially users with a machine that is newer than the versions of the software they usually use (and so have plenty of excess memory available).

BUT

2. if the memory used is needed by your real work (Word or whatever) then it will take a time for the app to be swapped out and your real work swapped back in. This can be a reason to avoid keeping apps in memory, especially users with a machine with onlu just enough memory for the task they normally work on.

ID: 11349 · Report as offensive     Reply Quote
Profile meckano
Avatar

Send message
Joined: 17 Sep 04
Posts: 150
Credit: 20,315
RAC: 0
Message 11350 - Posted: 14 Nov 2005, 15:38:21 UTC - in response to Message 11349.  

Yes! Agreed!
Since I've made the switch, anything I try to do takes time if I leave the computer for the amount of time it takes to..... make a coffee. :)
-----------------------
Click to see my tag
My tag
SNAFU'ed? Turn the Page! :D
ID: 11350 · Report as offensive     Reply Quote
1 · 2 · Next

Message boards : Number crunching : Projects use very little virtual memory?


©2024 CERN