Message boards : Sixtrack Application : Shiny new executables! Could you try them?
Message board moderation

To post messages, you must log in.

Previous · 1 · 2 · 3 · Next

AuthorMessage
m

Send message
Joined: 6 Sep 08
Posts: 116
Credit: 10,927,002
RAC: 2,464
Message 28887 - Posted: 15 Feb 2017, 11:28:04 UTC

OK on Ubuntu 12.04(64bit) on this host.
Took 5min 9 sec according to stderr.
Fort 6 says it took 166.625 sec to do the calculations.

Much trouble downloading the canonical file. Kept getting the HTML, then
couldn't unwrap the lines.
In the end copied & pasted the text bit, had to reformat the last line (lots of the spaces missing - don't know how that happened), fix the line endings...
Then diff tells me I've missed the newline at the end...

Please, someone tell me how to do something simple like download a file.
It was 0230 this morning.....zzzzzzz
ID: 28887 · Report as offensive     Reply Quote
gyllic

Send message
Joined: 9 Dec 14
Posts: 202
Credit: 2,533,875
RAC: 0
Message 28888 - Posted: 15 Feb 2017, 12:26:36 UTC - in response to Message 28887.  
Last modified: 15 Feb 2017, 12:34:46 UTC

Please, someone tell me how to do something simple like download a file.


1. Go to the file you want to download
2. Click it to view the contents within the GitHub UI
3. In the top right, right click the RAW button
4. Save as...

on linux: diff -s fma_sixtrack fma_sixtrack.canonical will give you something like that if the files are identical: files fma_sixtrack and fma_sixtrack.canonical are identical

You can also clone the GitHub repository, but this will download all files (also those you wont need)
ID: 28888 · Report as offensive     Reply Quote
m

Send message
Joined: 6 Sep 08
Posts: 116
Credit: 10,927,002
RAC: 2,464
Message 28889 - Posted: 15 Feb 2017, 13:00:47 UTC - in response to Message 28888.  
Last modified: 15 Feb 2017, 13:03:05 UTC


right click


Thank you... thank you. Don't know why I didn't try that... but clearly I didn't.
Brain cells dying faster than I thought...
ID: 28889 · Report as offensive     Reply Quote
gyllic

Send message
Joined: 9 Dec 14
Posts: 202
Credit: 2,533,875
RAC: 0
Message 28890 - Posted: 15 Feb 2017, 14:50:12 UTC - in response to Message 28889.  
Last modified: 15 Feb 2017, 14:52:14 UTC

no problem!

right click


you can also left click the raw button which gives you the needed link and then in your console:
wget https://raw.githubusercontent.com/SixTrack/SixTrack/master/SixTest/fma/fma_sixtrack.canonical

;:))
ID: 28890 · Report as offensive     Reply Quote
kyrsjo
Project administrator
Project developer
Project tester
Project scientist

Send message
Joined: 23 Jan 17
Posts: 29
Credit: 375,570
RAC: 0
Message 28891 - Posted: 15 Feb 2017, 15:49:23 UTC - in response to Message 28886.  

To check the contents, use "diff", as in "diff fma_sixtrack fma_sixtrack.canonical" :) If it returns nothing, then everything is great.

If one of the files have windows line endings and the other not, diff will claim they are different. To make diff ignore those, add the flag "--strip-trailing-cr".

For a graphical diff, install "meld" from your distributions repos.
ID: 28891 · Report as offensive     Reply Quote
boinc127

Send message
Joined: 5 Feb 12
Posts: 19
Credit: 700,079
RAC: 0
Message 28892 - Posted: 15 Feb 2017, 18:21:33 UTC

I must say first of all thank you for compiling a sixtrack executable for FreeBSD. Very few BOINC projects compile apps for FreeBSD. Secondly, I successfully ran the executable for FreeBSD, using FreeBSD 11 stable 64-bit (obviously) and it successfully ran and the fma_sixtrack file was the same as the fma_sixtrack.canonical file from the github site. Also, the app ran for about 5 minutes.
ID: 28892 · Report as offensive     Reply Quote
kyrsjo
Project administrator
Project developer
Project tester
Project scientist

Send message
Joined: 23 Jan 17
Posts: 29
Credit: 375,570
RAC: 0
Message 28894 - Posted: 16 Feb 2017, 12:56:04 UTC - in response to Message 28892.  

Great, but don't thank me, thank James:
https://lhcathome.cern.ch/lhcathome/show_user.php?userid=454585

FYI, I'm a bit lost for what to do with the WinXP problem; it seems to be some kind of bug in libarchive's cmake build system. I did try to fix it in this pull request:
https://github.com/SixTrack/SixTrack/pull/222
which is where the second set of Windows binaries came from. It didn't help tough :(

If someone have suggestions for how to fix this (or patches to LibArchive...), I'm all ears!

Otherwise, I'm very happy to hear that the binaries seem to work on everything but Windows XP! If we can fix that, we will; if not I fear we will have to do without it...

On our end, we're waiting for the guy who runs the BOINC server to get back from holidays (which will happen next week), so that we can push out the new binaries. So soon.
ID: 28894 · Report as offensive     Reply Quote
m

Send message
Joined: 6 Sep 08
Posts: 116
Credit: 10,927,002
RAC: 2,464
Message 28895 - Posted: 16 Feb 2017, 20:14:43 UTC
Last modified: 16 Feb 2017, 21:12:42 UTC

First off, I'm no writer of software but could an older version provide some clues? This implies that v2.4.12 of libarchive runs (ran) on XP. Looks like it should work on any old Windows.

Edit. In any case, is it not possible to retain the dull old exe for XP hosts? It's certainly stood the test of time and you didn't say that the new ones provided any new functions...
ID: 28895 · Report as offensive     Reply Quote
boinc127

Send message
Joined: 5 Feb 12
Posts: 19
Credit: 700,079
RAC: 0
Message 28896 - Posted: 17 Feb 2017, 5:30:30 UTC
Last modified: 17 Feb 2017, 5:39:02 UTC

Concerning the issue with Windows XP, I installed Windows XP through Virtualbox and attempted to run the sixtrack binaries for win32 on Windows XP and also received the message that the program failed because of the missing bcrypt.dll. Unfortunate, since bcrypt.dll is not a Windows XP library.

I'm not a programmer by any definition of the word, however, I downloaded the source code from the github site where the executables are located. I searched the source code for the term "bcrypt" and discovered in /SixTrack-4.6.5/CMakeLists.txt the term bcrypt. I believe I have found where bcrypt.dll is being pulled into the compilation.

Starting at line 938 from the CMakeLists.txt file in the SixTrack-4.6.5 directory:

if(WIN32)
#Windows needs a trampoline between __imp__FUNNAME -> FUNNAME, which is provided by the .dll.a
# Therefore, must link as -L${LIBARCHIVE_BUILD_DIR}/libarchive/ -larchive
# Not just include the .a
find_library(archive STATIC PATH ${LIBARCHIVE_BUILD_DIR}/libarchive/ archive)

find_library(zlib STATIC z)
find_library(pthread STATIC pthread)
find_library(bz2 STATIC bz2)
find_library(Bcrypt STATIC Bcrypt)
find_library(iconv STATIC iconv)
target_link_libraries(SixTrack archive z bz2 Bcrypt pthread iconv)
target_link_libraries(libArchiveTester archive z bz2 Bcrypt pthread iconv)

else()
add_library(libarchive STATIC IMPORTED)
set_target_properties(libarchive PROPERTIES IMPORTED_LOCATION ${LIBARCHIVE_BUILD_DIR}/libarchive/libarchive.a)

find_library(pthread STATIC pthread)
find_library(bz2 STATIC bz2)
find_library(z STATIC z)

target_link_libraries(SixTrack libarchive z bz2 pthread)
target_link_libraries(libArchiveTester libarchive z bz2 pthread)
endif()
add_definitions( -DLIBARCHIVE )
endif()


It is probably easier to read from the original file. I'm not a programmer, but my interpretation of the code is, from the conditional, if its defined as win32 (for libarchive, line 924) include these libraries: zlib, pthread, bz2, Bcrypt, and iconv as well; and then link them. Else, if its any other OS (?) find the libraries pthread, bz2, and z. So would winxp need to be separately defined without finding and linking Bcrypt? Thats my two cents, and you guys have probably already spotted that. I just thought I would point it out, just in case.
ID: 28896 · Report as offensive     Reply Quote
kyrsjo
Project administrator
Project developer
Project tester
Project scientist

Send message
Joined: 23 Jan 17
Posts: 29
Credit: 375,570
RAC: 0
Message 28902 - Posted: 17 Feb 2017, 9:07:57 UTC - in response to Message 28896.  

Hmm, I wrote that code some weeks ago :)

So.. LibArchive is (no longer) linking to BCrypt, we are manually forcing SixTrack to link to BCrypt. The reason is that when I wrote this, LibArchive on Windows required BCrypt, so to link to LibArchive, I needed to link to BCrypt.

Removing that dependency should fix it. I'll try to push a new executable around 5-ish today!
ID: 28902 · Report as offensive     Reply Quote
kyrsjo
Project administrator
Project developer
Project tester
Project scientist

Send message
Joined: 23 Jan 17
Posts: 29
Credit: 375,570
RAC: 0
Message 28907 - Posted: 17 Feb 2017, 16:51:16 UTC

Hi all, sorry for yet another delay - boinc127 identified one problem, problem number 2 was that on Windows, I was linking the *system* libarchive, which on MSYS/CYGWIN32 was again linking BCrypt (dynamically); not my carefully stripped-down version.

I fixed that too, but now there is some unnamed library it can't find... Will look more on monday. Worklog is in the issue tracker:
https://github.com/SixTrack/SixTrack/pull/222

Regarding keeping an old WinXP build machine around, we tried, but IT would not let us... :( I can understand why though :)
ID: 28907 · Report as offensive     Reply Quote
m

Send message
Joined: 6 Sep 08
Posts: 116
Credit: 10,927,002
RAC: 2,464
Message 28910 - Posted: 18 Feb 2017, 20:00:50 UTC
Last modified: 18 Feb 2017, 20:04:12 UTC

Just for fun, tested OK (output files identical, got the endings right this time) on native 32bit Linux (Mint 17.3) . It ran for about 20 minutes but I forgot to shut down BOINC which went off and downloaded a couple of "proper" tasks which were running as well.
ID: 28910 · Report as offensive     Reply Quote
kyrsjo
Project administrator
Project developer
Project tester
Project scientist

Send message
Joined: 23 Jan 17
Posts: 29
Credit: 375,570
RAC: 0
Message 28915 - Posted: 18 Feb 2017, 23:31:36 UTC - in response to Message 28910.  

Good to hear! The Linux execs were compiled on a 32-bit VM with fedora 25, so pretty much native.

In general we've had much less problems with building up the *NIX execs. I really need to figure out whats going on with libarchive on Windows on Monday :/
ID: 28915 · Report as offensive     Reply Quote
m

Send message
Joined: 6 Sep 08
Posts: 116
Credit: 10,927,002
RAC: 2,464
Message 28923 - Posted: 19 Feb 2017, 16:35:43 UTC

Just an idle thought...
Running Dependency Walker (depends) on the v2 exe shows the missing bcrypt (now, presumably, fixed) also potentially problems with mpr.dll. This is an XP library and is present on the system I used.
It seems to be a delay-load dependency of setupapi.dll - which was OK and of shlwapi.dll, which wasn't.
This is from the FAQ for depends:-

Q Why am I seeing a lot of applications where MPR.DLL shows up in red under SHLWAPI.DLL because it is missing a function named WNetRestoreConnectionA? I also get a "Warning: At least one module has an unresolved import due to a missing export function in a delay-load dependent module" message.

A Some versions of SHLWAPI.DLL (like the one on Windows XP) have a delay-load dependency on the function WNetRestoreConnectionA in MPR.DLL. Missing delay-load functions are not a problem as long as the calling DLL is prepared to handle the situation. Dependency Walker flags all potential problems as it cannot detect if an application intends to handle the issue. In the case of SHLWAPI.DLL, this is not an problem as it does not require WNetRestoreConnectionA to exist and handles the missing function at runtime. This warning can be ignored. See the "How to Interpret Warnings and Errors in Dependency Walker" section in help for more details.


So perhaps you could just try it and see what happens.
ID: 28923 · Report as offensive     Reply Quote
kyrsjo
Project administrator
Project developer
Project tester
Project scientist

Send message
Joined: 23 Jan 17
Posts: 29
Credit: 375,570
RAC: 0
Message 28924 - Posted: 19 Feb 2017, 17:01:00 UTC - in response to Message 28923.  

I'll have a look at that; I normally just use the standard "ldd" tool that I know from Linux; but it seems that library loading on Windows is a bit more convoluted than on Linux.
ID: 28924 · Report as offensive     Reply Quote
kyrsjo
Project administrator
Project developer
Project tester
Project scientist

Send message
Joined: 23 Jan 17
Posts: 29
Credit: 375,570
RAC: 0
Message 28932 - Posted: 20 Feb 2017, 15:45:32 UTC
Last modified: 20 Feb 2017, 15:45:48 UTC

And we try again: https://github.com/SixTrack/SixTrack/releases/tag/v4.6.6
Note that there is a new version number - I cleaned up a few things in the build system (for UNIX too!) while I was at it.

It turned out that (a) I was linking the system libarchive, which WAS static except for linking the Windows system libraries, and (2) when I built my own libarchive, it picked up the dll for zlib istead of the .a, making it not fully static. The big problem is that when linking to a dll on windows, the function (as an example) "deflate" gets the symbol "__imp_deflate" instead of just "deflate" as when it is statically linked. Therefore I was not able to link my own self-built libarchive (at least not without needing a system zlib, which isn't generally present on a Windows system).

However, I managed to convince libarchive to link *statically* to zlib, so that I could do the same for SixTrack -> libarchive.

So unless there are other problems not yet seen, this should work.

Regarding mpr.dll, this is a standard Windows system library, present since XP:
https://msdn.microsoft.com/en-us/library/windows/desktop/aa378771(v=vs.85).aspx
I'm not sure why this gets linked - it could be from BOINC? However since this is a standard system library, present since the stone age, it shouldn't make any problems (famous last words).

The binaries are currently being tested on my workstation / under-desk electrical heater, and once they have passed + you confirm that XP works, we can push it to our test queue, then to production. We're getting there!
ID: 28932 · Report as offensive     Reply Quote
m

Send message
Joined: 6 Sep 08
Posts: 116
Credit: 10,927,002
RAC: 2,464
Message 28936 - Posted: 20 Feb 2017, 17:49:24 UTC

You're there. Tested OK on native 32bitXP. Output files identical (changed endings in Notepad++, then used fc) Took a bit over 20mins but there were two Rosetta BOINC tasks (forgot again.. it's the brain cells) running alongside.
Thanks for all the effort.
ID: 28936 · Report as offensive     Reply Quote
maeax

Send message
Joined: 2 May 07
Posts: 2071
Credit: 156,173,543
RAC: 105,669
Message 28939 - Posted: 20 Feb 2017, 19:21:33 UTC - in response to Message 28886.  

OpenSuse-Leap_42.2(x64) - Linux: 4.4.46-11-default
Finished successful after 9 min.

Windows XP(x86)pro
Finished successful after 16 min.

Thank you Sixtrack-Team.
ID: 28939 · Report as offensive     Reply Quote
Crystal Pellet
Volunteer moderator
Volunteer tester

Send message
Joined: 14 Jan 10
Posts: 1268
Credit: 8,421,616
RAC: 2,139
Message 28940 - Posted: 20 Feb 2017, 19:44:11 UTC

I tested the newest SixTrack_466_libarchive_cr_boinc_api_crlibm_fast_tilt_cmake_Windows_gfortran.exe_static_AMD64_32bit.exe on my Intel Atom CPU Z3735F with Windows 10 32bit-OS.
It finished after 27 minutes run time. Other cores busy.
The contents of fma_sixtrack is corresponding to the 4.6.5 version.
ID: 28940 · Report as offensive     Reply Quote
gyllic

Send message
Joined: 9 Dec 14
Posts: 202
Credit: 2,533,875
RAC: 0
Message 28942 - Posted: 20 Feb 2017, 23:21:30 UTC

like the previous version, everything worked for me:

- tested the 4.6.6 version on Debian 9.0 Stretch (testing) 32 bit with linux 4.9 (in QEMU/KVM), finished after 4min 45sec and fma_sixtrack and fma_sictrack.canonical are identic.
ID: 28942 · Report as offensive     Reply Quote
Previous · 1 · 2 · 3 · Next

Message boards : Sixtrack Application : Shiny new executables! Could you try them?


©2024 CERN