Message boards : Number crunching : Invalid tasks
Message board moderation

To post messages, you must log in.

Previous · 1 · 2 · 3 · 4 · Next

AuthorMessage
henry

Send message
Joined: 15 Sep 13
Posts: 73
Credit: 5,763
RAC: 0
Message 26270 - Posted: 12 Mar 2014, 13:41:17 UTC - in response to Message 26268.  

Lots of people call good advice they don't like a flame. Games programmers don't write fortran so they won't be using your libs either and I fail to see how Intel can earn any additional profit by generating different code for different OS. Different code for a competing brand might be profitable for them but not for a different OS.

If a VM helps you then it helps this project. If the issue of floating point error accumulation is due to hardware issues then I guess you have a point but solving that in software is futile.

6 bangers are for wussies.
ID: 26270 · Report as offensive     Reply Quote
Eric Mcintosh
Volunteer moderator
Project administrator
Project developer
Project tester
Project scientist

Send message
Joined: 12 Jul 11
Posts: 852
Credit: 1,619,050
RAC: 0
Message 26273 - Posted: 12 Mar 2014, 15:05:37 UTC - in response to Message 26270.  

Lots of people call good advice they don't like a flame. Games programmers don't write fortran so they won't be using your libs either and I fail to see how Intel can earn any additional profit by generating different code for different OS. Different code for a competing brand might be profitable for them but not for a different OS.

Thanks Henry; I just said I don't want a flame war.
Games programmers use the elementary functions and my methodology applies to C as well
maybe even C++.
Intel could make more profit or spend less by using the SAME code generator. (In fact they deliberately don't generate their best code for competing brands.)

If a VM helps you then it helps this project. If the issue of floating point error accumulation is due to hardware issues then I guess you have a point but solving that in software is futile.
....but it needs more volunteer resources and I can't change the hardware.

...and yes I find it hard to take criticism (like most people?) but at least I am listening
and your questions on WHY are rather pertinent. Eric.

ID: 26273 · Report as offensive     Reply Quote
henry

Send message
Joined: 15 Sep 13
Posts: 73
Credit: 5,763
RAC: 0
Message 26274 - Posted: 12 Mar 2014, 17:17:04 UTC - in response to Message 26273.  
Last modified: 12 Mar 2014, 17:20:02 UTC

Lots of people call good advice they don't like a flame. Games programmers don't write fortran so they won't be using your libs either and I fail to see how Intel can earn any additional profit by generating different code for different OS. Different code for a competing brand might be profitable for them but not for a different OS.

Thanks Henry; I just said I don't want a flame war.
Games programmers use the elementary functions and my methodology applies to C as well
maybe even C++.
Intel could make more profit or spend less by using the SAME code generator. (In fact they deliberately don't generate their best code for competing brands.)

If a VM helps you then it helps this project. If the issue of floating point error accumulation is due to hardware issues then I guess you have a point but solving that in software is futile.
....but it needs more volunteer resources and I can't change the hardware.


Indeed the hardware is out of your control but it seems to me your libs work for a while then suddenly they don't. So then you tweak and then they work again but eventually they don't work again. The cycle seems to never end. I doubt it's due to a lack of talent on your end. I'm thinking some other thing that is out of your control is changing and breaking your libs. Perhaps a change in the Linux kernel or something?

You're saying using a VM takes more volunteer resources. Please elaborate. Do you mean additional volunteer resources to setup the project server or do you mean the additional overhead required of the host? If that's all that is preventing you from using a VM then perhaps those additional resources can be found somewhere/somehow. What I mean is perhaps someone can find them for you seeing as how you're busy with other stuff.

...and yes I find it hard to take criticism (like most people?) but at least I am listening
and your questions on WHY are rather pertinent. Eric.


Sorry, I don't follow. Questions on why you find it hard to take criticism? No, you probably don't mean that but unfortunately I don't know exactly which "questions on WHY" you refer to so again, please elaborate.
6 bangers are for wussies.
ID: 26274 · Report as offensive     Reply Quote
tom310

Send message
Joined: 28 Aug 12
Posts: 15
Credit: 500,336
RAC: 0
Message 26275 - Posted: 12 Mar 2014, 20:37:27 UTC
Last modified: 12 Mar 2014, 20:38:10 UTC

Eric, honestly, you need some coaching in regard to vacation.

http://www.worthamphotography.com/beachchairsunset.html

See you later

;)
ID: 26275 · Report as offensive     Reply Quote
Ano

Send message
Joined: 29 Nov 09
Posts: 42
Credit: 229,229
RAC: 0
Message 26276 - Posted: 12 Mar 2014, 21:59:33 UTC

I don't understand why you insist so much henry.
BOINC is:
Use the idle time on your computer (Windows, Mac, Linux, or Android) to cure diseases, study global warming, discover pulsars, and do many other types of scientific research. It's safe, secure, and easy

You're basically saying: don't use BOINC, since you don't care about the concept.
It's not impossible (I think seti@home got something like an optimized application not needing boinc or anything else,maybe something like that is what you want, optimized for some sort of virtual computing or something), but then you're asking the wrong way at the wrong place I think.
ID: 26276 · Report as offensive     Reply Quote
henry

Send message
Joined: 15 Sep 13
Posts: 73
Credit: 5,763
RAC: 0
Message 26277 - Posted: 12 Mar 2014, 23:15:02 UTC - in response to Message 26276.  
Last modified: 12 Mar 2014, 23:26:38 UTC

I don't understand why you insist so much henry.


I insist so much because at this point in time results from Linux hosts still don't reliably verify against results from Windows hosts and the LHC is scheduled to be back online in the not too distant future. If we're to be prepared for doing the magnet/beam tuning work required for the higher luminosity and new hadrons and if we want to do that work without a whole bunch of tasks failing on verification then something needs to be done soon. Eric's efforts so far have given what appears to be intermittent success therefore I propose that either the project drop support for Linux hosts and go with only Windows hosts or they adopt the VM approach.

BOINC is:
Use the idle time on your computer (Windows, Mac, Linux, or Android) to cure diseases, study global warming, discover pulsars, and do many other types of scientific research. It's safe, secure, and easy

You're basically saying: don't use BOINC, since you don't care about the concept.


Wrong. Not just 90% wrong, completely wrong with bonus points for a total of 110% wrong. I do care about the concept and I am not proposing that anybody abandon BOINC. I am asking Eric/Sixtrack to use the approach T4T uses which is to run its tasks in a VM that is issued as a BOINC task. I am quite certain that the entire infrastructure already exists (or very nearly exists) at CERN to allow Sixtrack tasks to be issued along with T4T tasks using their pilot & copilot mechanism. If that infrastructure does not exist or cannot be adapted to include Sixtrack then whatever is needed can be built by community volunteers if necessary.

It's not impossible (I think seti@home got something like an optimized application not needing boinc or anything else,maybe something like that is what you want, optimized for some sort of virtual computing or something), but then you're asking the wrong way at the wrong place I think.


I am proud to say I have absolutely nothing to do with that useless waste of CPU cycles called SETI and therefore have no idea what they've done but from your description I can assure you that if they have what you say then I don't want it nor do I want Sixtrack to want it.

The way I ask isn't going to change so get used to it. If you think there is a better place to ask then please tell me what and where that place is and I will consider it.
6 bangers are for wussies.
ID: 26277 · Report as offensive     Reply Quote
Profile yo2013
Avatar

Send message
Joined: 16 Oct 13
Posts: 50
Credit: 259,979
RAC: 1
Message 26279 - Posted: 13 Mar 2014, 8:33:04 UTC

Maybe it's time to switch to an open-source compiler.

I'll not participate on henry's flame war.
ID: 26279 · Report as offensive     Reply Quote
Eric Mcintosh
Volunteer moderator
Project administrator
Project developer
Project tester
Project scientist

Send message
Joined: 12 Jul 11
Posts: 852
Credit: 1,619,050
RAC: 0
Message 26280 - Posted: 13 Mar 2014, 8:49:24 UTC - in response to Message 26279.  

I'd love to switch to gfortran; apart from performance issues I
can't switch horses in mid-stream though. There are also problems
in identifying the code generation options that I don't really understand
(yet) but the default gfortran O4 does produce results identical to
ifort and nagfor so it can't be using e.g.FMA by default.
One day perhaps, because it would be great for our open and volunteer project
to use the open source compiler. Eric.
ID: 26280 · Report as offensive     Reply Quote
Eric Mcintosh
Volunteer moderator
Project administrator
Project developer
Project tester
Project scientist

Send message
Joined: 12 Jul 11
Posts: 852
Credit: 1,619,050
RAC: 0
Message 26282 - Posted: 13 Mar 2014, 9:23:20 UTC - in response to Message 26275.  

Thanks Tom I am enjoying vacation and seeing my grand-daughters
but time waits for no man. It does give me a chance to reflect and
to at last try and show some response on the Message Boards.
I think it is the least we can do to try and keep volunteers aware of
what is going on. Eric.
ID: 26282 · Report as offensive     Reply Quote
Eric Mcintosh
Volunteer moderator
Project administrator
Project developer
Project tester
Project scientist

Send message
Joined: 12 Jul 11
Posts: 852
Credit: 1,619,050
RAC: 0
Message 26283 - Posted: 13 Mar 2014, 9:43:40 UTC - in response to Message 26277.  

[quote]I don't understand why you insist so much henry.


I insist so much because at this point in time results from Linux hosts still don't reliably verify against results from Windows hosts and the LHC is scheduled to be back online in the not too distant future. If we're to be prepared for doing the magnet/beam tuning work required for the higher luminosity and new hadrons and if we want to do that work without a whole bunch of tasks failing on verification then something needs to be done soon. Eric's efforts so far have given what appears to be intermittent success therefore I propose that either the project drop support for Linux hosts and go with only Windows hosts or they adopt the VM approach.

The only way I know to identify correct results is to by repeatability across compilers
and OS. In fact right now I believe the Windows results are slightly wrong. I ran for two years or
more using the Lahey-Fujitsu compiler acrosss Windows/Linux with no problem apart from
failing hardware. My crlibm libraries are just fine and have never given a problem.
The main difficulty is organisational and manpower and compilers. The last six months
have been very very difficult, my/our own problem. If we can't resolve soon then I guess
you will be glad to hear we shall have to fall back to Linux VMs on Windows and Mac and
maybe Android one day, not to mention GPUs. Not yet though :-) I am a stubborn old so and
so and I believe in what I am trying to do even if of limited relevance to most applications.
I think the integrity of the results is vital and I hope I could repeat some runs
in 20 years time and still get the same results. Eric.


ID: 26283 · Report as offensive     Reply Quote
henry

Send message
Joined: 15 Sep 13
Posts: 73
Credit: 5,763
RAC: 0
Message 26288 - Posted: 13 Mar 2014, 15:27:04 UTC - in response to Message 26279.  

Maybe it's time to switch to an open-source compiler.

I'll not participate on henry's flame war.


That's good because it's doubtful you have anything useful to add to the conversation anyway.

6 bangers are for wussies.
ID: 26288 · Report as offensive     Reply Quote
henry

Send message
Joined: 15 Sep 13
Posts: 73
Credit: 5,763
RAC: 0
Message 26289 - Posted: 13 Mar 2014, 15:39:14 UTC - in response to Message 26283.  

Eric,

Stubborn and tenacious are not far apart, I can relate to both. Glad to hear you at least entertain the possibility of having to fall back to using VMs. That's good enough for me so I'll STFU. Thanks for your timely, detailed responses, they are a refreshing change from your usual and I hope they become the new norm.

6 bangers are for wussies.
ID: 26289 · Report as offensive     Reply Quote
Profile yo2013
Avatar

Send message
Joined: 16 Oct 13
Posts: 50
Credit: 259,979
RAC: 1
Message 26291 - Posted: 13 Mar 2014, 16:39:10 UTC - in response to Message 26288.  
Last modified: 13 Mar 2014, 16:45:37 UTC

That's good because it's doubtful you have anything useful to add to the conversation anyway.


I found a lot of trolls much more smart than you.

It's good that the MB has an Ignore option.
ID: 26291 · Report as offensive     Reply Quote
henry

Send message
Joined: 15 Sep 13
Posts: 73
Credit: 5,763
RAC: 0
Message 26293 - Posted: 13 Mar 2014, 18:21:24 UTC - in response to Message 26291.  

That's good because it's doubtful you have anything useful to add to the conversation anyway.


I found a lot of trolls much more smart than you.

It's good that the MB has an Ignore option.


Really?! Well then answer me this Mr. Smarty Pants... Anyone who falls for my trolling is:

a) much more dumber than dog poop
b) dumber than me
c) about as dumb as dog poop
d) dog poop

6 bangers are for wussies.
ID: 26293 · Report as offensive     Reply Quote
Profile Ananas

Send message
Joined: 17 Jul 05
Posts: 102
Credit: 542,016
RAC: 0
Message 26295 - Posted: 14 Mar 2014, 9:13:54 UTC - in response to Message 26264.  
Last modified: 14 Mar 2014, 9:21:52 UTC

... (By the way I cannot undrestand why Intel has different
Code generation between Linux/Windows. A different OS
interface OK....but why code generation which is surely
just hardware dependent!) ...

The CPUs sure do the same - but the math. functions are often compiler specific, sometimes in the floating point libraries that the programs rely on, sometimes in the generated code.

Microsoft for example used to change their rounding policy nearly from each x86 version to the next. With GCC, if you tell the linker to use a specific runtime library (which then has to be delivered with the project files), you can get identical results between Windows versions that would differ if you used the bundled runtime libraries.

Afaik. the windows x64 libraries (currently) produce identical rounded results, independant from the windows version, so the problem is worse in x86.

We didn't manage to get guaranteed identical results at RNA-World between Linux and Windows, so the project decided to deliver two results of the same workunit always to the same operating system class. This solved the validator problem - but it is sure not the ideal way.

Einstein *) (or Pirates or both together) did a lot of research on this issue, afaik. they have written their own libraries. I cannot give you reliable informations about the functions they use, but they might have something like wrappers for the standard libraries, but with a reproducable rounding behaviour.

*) The project, not the guy with the funny haircut :-)
ID: 26295 · Report as offensive     Reply Quote
Profile Ray Murray
Volunteer moderator
Avatar

Send message
Joined: 29 Sep 04
Posts: 252
Credit: 11,225,577
RAC: 0
Message 26314 - Posted: 29 Mar 2014, 15:48:54 UTC

Of the 8 tasks completed over the last couple of days on my Linux machine, 4 validated against other Linux but good to see the 4 others validating against Windows at the first attempt.
ID: 26314 · Report as offensive     Reply Quote
tullio

Send message
Joined: 19 Feb 08
Posts: 614
Credit: 3,834,692
RAC: 843
Message 26315 - Posted: 29 Mar 2014, 15:58:08 UTC - in response to Message 26314.  

On my Linux box 6 have validated against various Windows, 2 against Linux.
Tullio
ID: 26315 · Report as offensive     Reply Quote
Profile Ananas

Send message
Joined: 17 Jul 05
Posts: 102
Credit: 542,016
RAC: 0
Message 26316 - Posted: 30 Mar 2014, 22:59:20 UTC
Last modified: 30 Mar 2014, 22:59:49 UTC

currently 8 inconclusive, 1 invalid, 768 valid

Inconclusive/invalid had been down to 0 for a while.
ID: 26316 · Report as offensive     Reply Quote
Eric Mcintosh
Volunteer moderator
Project administrator
Project developer
Project tester
Project scientist

Send message
Joined: 12 Jul 11
Posts: 852
Credit: 1,619,050
RAC: 0
Message 26317 - Posted: 2 Apr 2014, 2:31:33 UTC

Thanks for these reports; the workload has changed and these
cases should be, and seem to be, OK.

I believe the problem is case dependent in that it arises when making
extensive studies of power supply ripple. The problem appears to be
with the Intel ifort compiler for Windows. Hope to confirm and fix in
the next couple of days, but then need to make sure that a new
Windows build does not have the "Create process" nor the "Analysis
of PCU info problem". Thanks. Eric.
ID: 26317 · Report as offensive     Reply Quote
Matthias Lehmkuhl

Send message
Joined: 15 Jul 05
Posts: 21
Credit: 1,194,006
RAC: 355
Message 26338 - Posted: 9 Apr 2014, 7:16:16 UTC

I've 2 more invalid Linux SixTrack v446.05 (pni) against 2 Windows systems SixTrack v446.03 (pni)
http://lhcathomeclassic.cern.ch/sixtrack/workunit.php?wuid=16419099
http://lhcathomeclassic.cern.ch/sixtrack/workunit.php?wuid=16425599

and 2 more Validation inconclusive Linux against Windows
http://lhcathomeclassic.cern.ch/sixtrack/workunit.php?wuid=16583683
http://lhcathomeclassic.cern.ch/sixtrack/workunit.php?wuid=16592011
on both the next result is now not send.

Matthias

ID: 26338 · Report as offensive     Reply Quote
Previous · 1 · 2 · 3 · 4 · Next

Message boards : Number crunching : Invalid tasks


©2020 CERN