Message boards :
Number crunching :
New WUs writing too much to disk
Message board moderation
Author | Message |
---|---|
Send message Joined: 16 Oct 13 Posts: 59 Credit: 342,408 RAC: 0 |
Today's new WUs write to disk every 3-10 seconds, depending on the computer I use. Why so much writing? |
Send message Joined: 26 Sep 13 Posts: 3 Credit: 1,241,313 RAC: 0 |
I too am having issues with these w-b3_-26000 units where they crunch fine for around an hour, reach 100% then just continually write to the HDD. Using the Win8 task manager it shows 100% HDD write. Then the computer becomes immensely unresponsive. All of the unit's "Wingmen" have finished these fine but use Linux rather than the Win8.1 I'm using. I have suspended these units for now and likely to have to abort them just to try and get some new work. Any ideas? |
Send message Joined: 15 Oct 13 Posts: 6 Credit: 28,625 RAC: 0 |
Also seem to use a lot of disk space until it completes. This task http://lhcathomeclassic.cern.ch/sixtrack/result.php?resultid=48058882 was up to 600MB just before completing. |
Send message Joined: 26 Sep 13 Posts: 3 Credit: 1,241,313 RAC: 0 |
I have been able to leave these units running while away from the computer for a few hours and it some have now been returned and validated. Shame about the large amount of disk write that meant that I couldn't do these units without significantly affecting the computer use during the normal day. |
Send message Joined: 12 Jul 11 Posts: 857 Credit: 1,619,050 RAC: 0 |
Sorry about this. These WUs require a very precise estimation of the DA which implies writing the particle co-ordinates every turn to 30 files (one file per pair of particles), about 100 bytes each time. This data is then used by the post-processing to make the results summary. I also endfile and backspace after each write to flush the buffer. This seems to be OK on Linux but from your comments appears to be a problem on Windows. The total data is indeed about 600MB for 100,000 turns, 30 particle pairs. I am wondering if I can remove the endfile/backspace except before doing a checkpoint. Difficult to change right now as we have a lot of work to process before he upcoming HL-LHC conference. I shall see what I can do, but I shall be on vacation for two weeks and will have no remote access to my Windows box for testing and I am reluctant to modify a well proven procedure. Eric. |
Send message Joined: 26 Sep 13 Posts: 3 Credit: 1,241,313 RAC: 0 |
Thanks for the reply Eric. All of mine have now cleared through my system. During the actual crunching, the write didn't seem to be a problem it was when it had completed the unit that the writing made the computer almost unusable. Leaving the machine alone for a few hours allowed the write to complete and then upload the result, unfortunately I don't know how long this took as I was away from the computer. The computer used for this batch of units was a Win8.1 Pro 32bit. Can't say I have seen this happen on any previous units. Each of the units I had, CPU time was around 3,500 seconds but run time varied from 4,000 to 6,000 seconds. WU numbers 22544559 - 22544565 and 22544603 -22544610. |
Send message Joined: 12 Jul 11 Posts: 857 Credit: 1,619,050 RAC: 0 |
Thanks for the feedback; I am running 4 such WUs on my Windows 7 with 4 threads BUT on Cygwin.......seems OK but it then takes a VERY long time to finish. Investigation proceeds. Eric. |
Send message Joined: 12 Jul 11 Posts: 857 Credit: 1,619,050 RAC: 0 |
First look seems to eliminate the "tracking". My suspicions are now centered on the post-processing phase where 30 files will be read. Hopefully this will take at most some minutes. Investigation continues; can't make changes late on Friday! |
Send message Joined: 4 May 07 Posts: 250 Credit: 826,541 RAC: 0 |
At one time I did "real work" (business) on my Windowz system. Mainly using Photoshop, Illustrator, and Framemaker. I put these applications in the BOINC->Preferences->Exclusive applications tab. This seemed to work very well for me. Now I don't do a lot of "heavy lifting" so the slowdown is something I can live with. The only thing I've noticed is the Exclusive applications option may not suspend vLHC crunching. (But I haven't really looked closely.) |
©2024 CERN