Posts by [AF] Hydrosaure

1) Message boards : Number crunching : Setting up a local Squid to work with LHC@home - Comments and Questions (Message 47000) Posted 9 Jul 2022 by [AF] Hydrosaure Post: Just tested with Alpine Linux 3.16 bundled Squid version 5.5 and this upload issue is still present. So the recommendation still stands true to this day: The preferred version should be the most recent squid package from your Linux distribution repository. Version >=3.5.27 and <5.x
2) Message boards : Number crunching : Setting up a local Squid to work with LHC@home - Comments and Questions (Message 45610) Posted 4 Nov 2021 by [AF] Hydrosaure Post: With Squid version 5.0.6 I'm having difficulties uploading results to LHC. In the logs I seem to be getting a lot of messages like these: 1635870641.453 135 192.168.3.69 TCP_MISS/200 348 POST http://lhcathome-upload.cern.ch/lhcathome_cgi/file_upload_handler - HIER_DIRECT/188.185.101.107 text/plain 1635870643.445 120 192.168.3.69 TCP_MISS_ABORTED/100 0 POST http://lhcathome-upload.cern.ch/lhcathome_cgi/file_upload_handler - HIER_DIRECT/188.185.101.107 - 1635871151.414 332 192.168.3.69 TCP_MISS/200 329 POST http://lhcathome-upload.cern.ch/lhcathome_cgi/file_upload_handler - HIER_DIRECT/188.185.125.226 text/plain 1635871151.414 332 192.168.3.69 TCP_MISS/200 348 POST http://lhcathome-upload.cern.ch/lhcathome_cgi/file_upload_handler - HIER_DIRECT/188.185.125.226 text/plain 1635871153.135 129 192.168.3.69 TCP_MISS_ABORTED/100 0 POST http://lhcathome-upload.cern.ch/lhcathome_cgi/file_upload_handler - HIER_DIRECT/188.185.125.226 - 1635871153.137 132 192.168.3.69 TCP_MISS_ABORTED/100 0 POST http://lhcathome-upload.cern.ch/lhcathome_cgi/file_upload_handler - HIER_DIRECT/188.185.125.226 - 1635874194.290 155 192.168.3.69 TCP_MISS/200 348 POST http://lhcathome-upload.cern.ch/lhcathome_cgi/file_upload_handler - HIER_DIRECT/188.185.75.236 text/plain 1635874194.301 165 192.168.3.69 TCP_MISS/200 353 POST http://lhcathome-upload.cern.ch/lhcathome_cgi/file_upload_handler - HIER_DIRECT/188.185.75.236 text/plain 1635874196.274 134 192.168.3.69 TCP_MISS_ABORTED/100 0 POST http://lhcathome-upload.cern.ch/lhcathome_cgi/file_upload_handler - HIER_DIRECT/188.185.75.236 - 1635874196.284 144 192.168.3.69 TCP_MISS_ABORTED/100 0 POST http://lhcathome-upload.cern.ch/lhcathome_cgi/file_upload_handler - HIER_DIRECT/188.185.75.236 - 1635877114.137 90 192.168.3.69 TCP_MISS/200 348 POST http://lhcathome-upload.cern.ch/lhcathome_cgi/file_upload_handler - HIER_DIRECT/188.185.75.236 text/plain 1635877114.137 90 192.168.3.69 TCP_MISS/200 348 POST http://lhcathome-upload.cern.ch/lhcathome_cgi/file_upload_handler - HIER_DIRECT/188.185.75.236 text/plain 1635877115.554 101 192.168.3.69 TCP_MISS_ABORTED/100 0 POST http://lhcathome-upload.cern.ch/lhcathome_cgi/file_upload_handler - HIER_DIRECT/188.185.75.236 - 1635877115.561 108 192.168.3.69 TCP_MISS_ABORTED/100 0 POST http://lhcathome-upload.cern.ch/lhcathome_cgi/file_upload_handler - HIER_DIRECT/188.185.75.236 - 1635883410.768 157 192.168.3.69 TCP_MISS/200 329 POST http://lhcathome-upload.cern.ch/lhcathome_cgi/file_upload_handler - HIER_DIRECT/188.185.86.60 text/plain 1635883410.769 158 192.168.3.69 TCP_MISS/200 348 POST http://lhcathome-upload.cern.ch/lhcathome_cgi/file_upload_handler - HIER_DIRECT/188.185.86.60 text/plain 1635883412.859 104 192.168.3.69 TCP_MISS_ABORTED/100 0 POST http://lhcathome-upload.cern.ch/lhcathome_cgi/file_upload_handler - HIER_DIRECT/188.185.86.60 - 1635883412.884 129 192.168.3.69 TCP_MISS_ABORTED/100 0 POST http://lhcathome-upload.cern.ch/lhcathome_cgi/file_upload_handler - HIER_DIRECT/188.185.86.60 - Downgrading to version 4.17 solved the blocked upload. Any idea what has evolved between these versions to explain this ?
3) Message boards : ATLAS application : Debian 11 (Message 45219) Posted 17 Aug 2021 by [AF] Hydrosaure Post: Happy to report a working and completed ATLAS job on a fresh Debian 11 install. Did you the same .deb as from a Debian 10. https://lhcathome.cern.ch/lhcathome/result.php?resultid=323787719
4) Message boards : Number crunching : Recommended CVMFS Configuration for Native Apps - Comments and Questions (Message 44240) Posted 31 Jan 2021 by [AF] Hydrosaure Post: Hello, My systems are all configured to use at least 4GB of local CVMFS cache and according to stats, they seem to reach pretty high hit ratios. Here is just a look at a couple of them: Would a proxy still help here ? Also are there specific Squid options one should set in regards to caching CVMFS data (ie. max object size, retention or refresh policy) ?
5) Message boards : ATLAS application : error on Atlas native: 195 (0x000000C3) EXIT_CHILD_FAILED (Message 44236) Posted 30 Jan 2021 by [AF] Hydrosaure Post: Thank you, will read and adjust my configs accordingly.
6) Message boards : ATLAS application : error on Atlas native: 195 (0x000000C3) EXIT_CHILD_FAILED (Message 44231) Posted 30 Jan 2021 by [AF] Hydrosaure Post: ...and yes installing a proxy is on my todo list :-) +1 Getting off topic but while we're on the subject of configuring a proxy, is CVMFS cache (ie CVMFS_QUOTA_LIMIT=4096) still required in the case of going through a caching proxy ?
7) Message boards : ATLAS application : error on Atlas native: 195 (0x000000C3) EXIT_CHILD_FAILED (Message 44225) Posted 29 Jan 2021 by [AF] Hydrosaure Post: On this test machine, configuration is now as follows: $ cat /etc/cvmfs/default.local CVMFS_USE_CDN=yes # BEGIN ANSIBLE MANAGED BLOCK CVMFS_REPOSITORIES=atlas.cern.ch,atlas-condb.cern.ch,grid.cern.ch,cernvm-prod.cern.ch,sft.cern.ch,alice.cern.ch CVMFS_QUOTA_LIMIT=4096 CVMFS_CACHE_BASE=/var/lib/cvmfs CVMFS_HTTP_PROXY=DIRECT CVMFS_SEND_INFO_HEADER=yes # END ANSIBLE MANAGED BLOCK After a while I ran $ sudo cvmfs_config stat Running /usr/bin/cvmfs_config stat cvmfs-config.cern.ch: VERSION PID UPTIME(M) MEM(K) REVISION EXPIRES(M) NOCATALOGS CACHEUSE(K) CACHEMAX(K) NOFDUSE NOFDMAX NOIOERR NOOPEN HITRATE(%) RX(K) SPEED(K/S) HOST PROXY ONLINE 2.7.5.0 1596 22 23752 14 2 1 3189293 4194304 0 65024 0 194 100 3 1 http://s1cern-cvmfs.openhtc.io/cvmfs/cvmfs-config.cern.ch DIRECT 1 Running /usr/bin/cvmfs_config stat atlas.cern.ch: VERSION PID UPTIME(M) MEM(K) REVISION EXPIRES(M) NOCATALOGS CACHEUSE(K) CACHEMAX(K) NOFDUSE NOFDMAX NOIOERR NOOPEN HITRATE(%) RX(K) SPEED(K/S) HOST PROXY ONLINE 2.7.5.0 1687 22 36640 78110 2 75 3189293 4194304 559 65024 0 10889 99.8898 278 330 http://s1cern-cvmfs.openhtc.io/cvmfs/atlas.cern.ch DIRECT 1 Running /usr/bin/cvmfs_config stat atlas-condb.cern.ch: VERSION PID UPTIME(M) MEM(K) REVISION EXPIRES(M) NOCATALOGS CACHEUSE(K) CACHEMAX(K) NOFDUSE NOFDMAX NOIOERR NOOPEN HITRATE(%) RX(K) SPEED(K/S) HOST PROXY ONLINE 2.7.5.0 1811 22 24372 8534 2 1 3189293 4194304 0 65024 0 13 92.3077 66 102 http://s1cern-cvmfs.openhtc.io/cvmfs/atlas-condb.cern.ch DIRECT 1 Running /usr/bin/cvmfs_config stat sft.cern.ch: VERSION PID UPTIME(M) MEM(K) REVISION EXPIRES(M) NOCATALOGS CACHEUSE(K) CACHEMAX(K) NOFDUSE NOFDMAX NOIOERR NOOPEN HITRATE(%) RX(K) SPEED(K/S) HOST PROXY ONLINE 2.7.5.0 2186 21 24976 19547 2 5 3189293 4194304 7 65024 0 209 99.5215 3 5 http://s1cern-cvmfs.openhtc.io/cvmfs/sft.cern.ch DIRECT 1 Seems to be using openhtc.io if I read this right. and yes installing a proxy is on my todo list :-)
8) Message boards : ATLAS application : error on Atlas native: 195 (0x000000C3) EXIT_CHILD_FAILED (Message 44218) Posted 28 Jan 2021 by [AF] Hydrosaure Post: Can confirm that with a self built version of Singularity (version 3.7.1) ATLAS works correctly on a fresh Ubuntu 20.04. A result as example: https://lhcathome.cern.ch/lhcathome/result.php?resultid=297199728
9) Questions and Answers : Unix/Linux : Getting CVMFS to work under Linux Containers (LXC) (Message 40718) Posted 29 Nov 2019 by [AF] Hydrosaure Post: I have setup a Boinc VM running as a container under LXC on Proxmox. All projects seem to work fine except for CERN and the CVMFS based apps. I've changed the container to run with an "unconfined" profile still all tasks requiring CVMFS fail at even detecting cvmfs program Example log: 08:11:11 (12658): wrapper (7.15.26016): starting 08:11:11 (12658): wrapper (7.15.26016): starting 08:11:11 (12658): wrapper: running ../../projects/lhcathome.cern.ch_lhcathome/cranky-0.0.29 () 08:11:12 UTC +00:00 2019-11-29: cranky-0.0.29: [INFO] Detected Theory App 08:11:12 UTC +00:00 2019-11-29: cranky-0.0.29: [INFO] Checking CVMFS. 08:11:13 UTC +00:00 2019-11-29: cranky-0.0.29: [ERROR] 'cvmfs_config probe sft.cern.ch' failed. 08:11:13 (12658): cranky exited; CPU time 0.083059 08:11:13 (12658): app exit status: 0xce 08:11:13 (12658): called boinc_finish(195) A "cvmfs_config probe" returns OK for all targets when ran manually. Any help appreciated. Host in question => https://lhcathome.cern.ch/lhcathome/show_host_detail.php?hostid=10624114
10) Questions and Answers : Unix/Linux : ATLAS (native_mt) Python incompatibility on Linux (Message 40607) Posted 23 Nov 2019 by [AF] Hydrosaure Post: Latest version of native ATALS is still using Python2; it is starting to become a pressing matter seeing the EOL of that version.
11) Message boards : ATLAS application : ATLAS native version 2.73 (Message 40600) Posted 22 Nov 2019 by [AF] Hydrosaure Post: Any idea what could be the issue here ? https://lhcathome.cern.ch/lhcathome/result.php?resultid=252658162 2019-11-22 15:36:27,522: Checking singularity with cmd:/cvmfs/atlas.cern.ch/repo/containers/sw/singularity/x86_64-el7/current/bin/singularity exec -B /cvmfs /cvmfs/atlas.cern.ch/repo/containers/images/singularity/x86_64-centos7.img hostname 2019-11-22 15:37:43,370: Singularity isnt working: [34mINFO: [0m Convert SIF file to sandbox... [34mINFO: [0m Cleaning up image... [31mFATAL: [0m container creation failed: mount ->/cvmfs/atlas.cern.ch/repo/containers/sw/singularity/x86_64-el7/3.2.1/var/singularity/mnt/session/home/boinc error: can't remount /cvmfs/atlas.cern.ch/repo/containers/sw/singularity/x86_64-el7/3.2.1/var/singularity/mnt/session/home/boinc: invalid argument 2019-11-22 15:37:45,377: running start_atlas return value is 3 2019-11-22 15:47:45,448: parent process exit 3 2019-11-22 15:47:45,449: child process exit 3 Yet if I manually try to execute this command line boinc@boinc-z87:~$ /cvmfs/atlas.cern.ch/repo/containers/sw/singularity/x86_64-el7/current/bin/singularity --debug exec -B /cvmfs /cvmfs/atlas.cern.ch/repo/containers/images/singularity/x86_64-centos7.img hostname VERBOSE [U=1000,P=5653] execStarter() starter-suid not found, using user namespace VERBOSE [U=1000,P=5653] execStarter() User namespace requested, convert image /cvmfs/atlas.cern.ch/repo/containers/images/singularity/x86_64-centos7.img to sandbox INFO [U=1000,P=5653] execStarter() Convert SIF file to sandbox... DEBUG [U=1000,P=5653] Init() Entering image format intializer DEBUG [U=1000,P=5653] Init() Check for image format sif VERBOSE [U=1000,P=5653] print() Set messagelevel to: 5 DEBUG [U=1000,P=5653] init() PIPE_EXEC_FD value: 7 VERBOSE [U=1000,P=5653] init() Container runtime VERBOSE [U=1000,P=5653] is_suid() Check if we are running as setuid DEBUG [U=1000,P=5653] init() Read json configuration from pipe DEBUG [U=1000,P=5653] init() Set child signal mask DEBUG [U=1000,P=5653] init() Wait completion of stage1 DEBUG [U=1000,P=15975] set_parent_death_signal() Set parent death signal to 9 VERBOSE [U=1000,P=15975] init() Spawn stage 1 VERBOSE [U=1000,P=15975] startup() Execute stage 1 DEBUG [U=1000,P=15975] Stage() Entering stage 1 DEBUG [U=1000,P=15975] prepareFd() Open file descriptor for /cvmfs DEBUG [U=1000,P=15975] Init() Entering image format intializer DEBUG [U=1000,P=15975] Init() Check for image format sif DEBUG [U=1000,P=15975] Init() sif format initializer returns: not a SIF file image DEBUG [U=1000,P=15975] Init() Check for image format sandbox DEBUG [U=1000,P=5653] init() Create socketpair for master communication channel DEBUG [U=1000,P=5653] cleanup_fd() Check file descriptor /proc/self/fd/3 pointing to /cvmfs DEBUG [U=1000,P=5653] cleanup_fd() Check file descriptor /proc/self/fd/4 pointing to anon_inode:[eventpoll] DEBUG [U=1000,P=5653] cleanup_fd() Closing /proc/self/fd/4 DEBUG [U=1000,P=5653] cleanup_fd() Check file descriptor /proc/self/fd/5 pointing to /tmp/rootfs-490992350 VERBOSE [U=1000,P=5653] user_namespace_init() Create user namespace DEBUG [U=65534,P=5653] setup_userns_mappings() Write deny to set group file DEBUG [U=65534,P=5653] setup_userns_mappings() Write to GID map DEBUG [U=65534,P=5653] setup_userns_mappings() Write to UID map VERBOSE [U=1000,P=5653] create_namespace() Create mount namespace DEBUG [U=1000,P=5653] init() Create RPC socketpair for communication between stage 2 and RPC server VERBOSE [U=1000,P=5653] init() Spawn master process DEBUG [U=1000,P=16013] set_parent_death_signal() Set parent death signal to 9 VERBOSE [U=1000,P=16013] create_namespace() Create mount namespace DEBUG [U=1000,P=16013] set_parent_death_signal() Set parent death signal to 9 DEBUG [U=1000,P=16013] prepare_stage() Entering in stage 2 VERBOSE [U=1000,P=16014] init() Spawn RPC server VERBOSE [U=1000,P=16014] startup() Serve RPC requests VERBOSE [U=1000,P=5653] startup() Execute master process DEBUG [U=1000,P=5653] setupSessionLayout() Attempting to use underlay (enable underlay = yes) DEBUG [U=1000,P=5653] setupUnderlayLayout() Creating underlay SESSIONDIR layout DEBUG [U=1000,P=5653] addRootfsMount() Mount rootfs in read-only mode DEBUG [U=1000,P=5653] addRootfsMount() Mounting directory rootfs: /tmp/rootfs-490992350 DEBUG [U=1000,P=5653] addKernelMount() Checking configuration file for 'mount proc' DEBUG [U=1000,P=5653] addKernelMount() Adding proc to mount list VERBOSE [U=1000,P=5653] addKernelMount() Default mount: /proc:/proc DEBUG [U=1000,P=5653] addKernelMount() Checking configuration file for 'mount sys' DEBUG [U=1000,P=5653] addKernelMount() Adding sysfs to mount list VERBOSE [U=1000,P=5653] addKernelMount() Default mount: /sys:/sys DEBUG [U=1000,P=5653] addDevMount() Checking configuration file for 'mount dev' DEBUG [U=1000,P=5653] addDevMount() Adding dev to mount list VERBOSE [U=1000,P=5653] addDevMount() Default mount: /dev:/dev DEBUG [U=1000,P=5653] addHostMount() Not mounting host file systems per configuration VERBOSE [U=1000,P=5653] addBindsMount() Found 'bind path' = /etc/localtime, /etc/localtime VERBOSE [U=1000,P=5653] addBindsMount() Found 'bind path' = /etc/hosts, /etc/hosts DEBUG [U=1000,P=5653] addHomeStagingDir() Staging home directory (/home/boinc) at /cvmfs/atlas.cern.ch/repo/containers/sw/singularity/x86_64-el7/3.2.1/var/singularity/mnt/session/home/boinc DEBUG [U=1000,P=5653] addHomeMount() Adding home directory mount [/cvmfs/atlas.cern.ch/repo/containers/sw/singularity/x86_64-el7/3.2.1/var/singularity/mnt/session/home/boinc:/home/boinc] to list using layer: underlay DEBUG [U=1000,P=5653] isLayerEnabled() Using Layer system: underlay DEBUG [U=1000,P=5653] addUserbindsMount() Adding /cvmfs to mount list DEBUG [U=1000,P=5653] addUserbindsMount() Checking for 'user bind control' in configuration file DEBUG [U=1000,P=5653] addTmpMount() Checking for 'mount tmp' in configuration file VERBOSE [U=1000,P=5653] addTmpMount() Default mount: /tmp:/tmp VERBOSE [U=1000,P=5653] addTmpMount() Default mount: /var/tmp:/var/tmp DEBUG [U=1000,P=5653] addScratchMount() Not mounting scratch directory: Not requested VERBOSE [U=1000,P=5653] addCwdMount() Default mount: /home/boinc: to the container DEBUG [U=1000,P=5653] addLibsMount() Checking for 'user bind control' in configuration file DEBUG [U=1000,P=5653] addResolvConfMount() Adding /etc/resolv.conf to mount list VERBOSE [U=1000,P=5653] addResolvConfMount() Default mount: /etc/resolv.conf:/etc/resolv.conf DEBUG [U=1000,P=5653] addHostnameMount() Skipping hostname mount, not virtualizing UTS namespace on user request DEBUG [U=1000,P=5653] create() Mount all DEBUG [U=1000,P=5653] mountGeneric() Mounting tmpfs to /cvmfs/atlas.cern.ch/repo/containers/sw/singularity/x86_64-el7/3.2.1/var/singularity/mnt/session DEBUG [U=1000,P=5653] mountGeneric() Mounting /tmp/rootfs-490992350 to /cvmfs/atlas.cern.ch/repo/containers/sw/singularity/x86_64-el7/3.2.1/var/singularity/mnt/session/rootfs DEBUG [U=1000,P=5653] mountGeneric() Mounting /cvmfs/atlas.cern.ch/repo/containers/sw/singularity/x86_64-el7/3.2.1/var/singularity/mnt/session/rootfs/.singularity.d to /cvmfs/atlas.cern.ch/repo/containers/sw/singularity/x86_64-el7/3.2.1/var/singularity/mnt/session/underlay/.singularity.d DEBUG [U=1000,P=5653] mountGeneric() Mounting /cvmfs/atlas.cern.ch/repo/containers/sw/singularity/x86_64-el7/3.2.1/var/singularity/mnt/session/rootfs/alrb to /cvmfs/atlas.cern.ch/repo/containers/sw/singularity/x86_64-el7/3.2.1/var/singularity/mnt/session/underlay/alrb DEBUG [U=1000,P=5653] mountGeneric() Mounting /cvmfs/atlas.cern.ch/repo/containers/sw/singularity/x86_64-el7/3.2.1/var/singularity/mnt/session/rootfs/anaconda-post.log to /cvmfs/atlas.cern.ch/repo/containers/sw/singularity/x86_64-el7/3.2.1/var/singularity/mnt/session/underlay/anaconda-post.log DEBUG [U=1000,P=5653] mountGeneric() Mounting /cvmfs/atlas.cern.ch/repo/containers/sw/singularity/x86_64-el7/3.2.1/var/singularity/mnt/session/rootfs/cvmfs to /cvmfs/atlas.cern.ch/repo/containers/sw/singularity/x86_64-el7/3.2.1/var/singularity/mnt/session/underlay/cvmfs DEBUG [U=1000,P=5653] mountGeneric() Mounting /cvmfs/atlas.cern.ch/repo/containers/sw/singularity/x86_64-el7/3.2.1/var/singularity/mnt/session/rootfs/dev to /cvmfs/atlas.cern.ch/repo/containers/sw/singularity/x86_64-el7/3.2.1/var/singularity/mnt/session/underlay/dev DEBUG [U=1000,P=5653] mountGeneric() Mounting /cvmfs/atlas.cern.ch/repo/containers/sw/singularity/x86_64-el7/3.2.1/var/singularity/mnt/session/rootfs/docker-creation-date to /cvmfs/atlas.cern.ch/repo/containers/sw/singularity/x86_64-el7/3.2.1/var/singularity/mnt/session/underlay/docker-creation-date DEBUG [U=1000,P=5653] mountGeneric() Mounting /cvmfs/atlas.cern.ch/repo/containers/sw/singularity/x86_64-el7/3.2.1/var/singularity/mnt/session/rootfs/etc to /cvmfs/atlas.cern.ch/repo/containers/sw/singularity/x86_64-el7/3.2.1/var/singularity/mnt/session/underlay/etc DEBUG [U=1000,P=5653] mountGeneric() Mounting /cvmfs/atlas.cern.ch/repo/containers/sw/singularity/x86_64-el7/3.2.1/var/singularity/mnt/session/rootfs/gpfs to /cvmfs/atlas.cern.ch/repo/containers/sw/singularity/x86_64-el7/3.2.1/var/singularity/mnt/session/underlay/gpfs DEBUG [U=1000,P=5653] mountGeneric() Mounting /cvmfs/atlas.cern.ch/repo/containers/sw/singularity/x86_64-el7/3.2.1/var/singularity/mnt/session/rootfs/media to /cvmfs/atlas.cern.ch/repo/containers/sw/singularity/x86_64-el7/3.2.1/var/singularity/mnt/session/underlay/media DEBUG [U=1000,P=5653] mountGeneric() Mounting /cvmfs/atlas.cern.ch/repo/containers/sw/singularity/x86_64-el7/3.2.1/var/singularity/mnt/session/rootfs/mnt to /cvmfs/atlas.cern.ch/repo/containers/sw/singularity/x86_64-el7/3.2.1/var/singularity/mnt/session/underlay/mnt DEBUG [U=1000,P=5653] mountGeneric() Mounting /cvmfs/atlas.cern.ch/repo/containers/sw/singularity/x86_64-el7/3.2.1/var/singularity/mnt/session/rootfs/opt to /cvmfs/atlas.cern.ch/repo/containers/sw/singularity/x86_64-el7/3.2.1/var/singularity/mnt/session/underlay/opt DEBUG [U=1000,P=5653] mountGeneric() Mounting /cvmfs/atlas.cern.ch/repo/containers/sw/singularity/x86_64-el7/3.2.1/var/singularity/mnt/session/rootfs/proc to /cvmfs/atlas.cern.ch/repo/containers/sw/singularity/x86_64-el7/3.2.1/var/singularity/mnt/session/underlay/proc DEBUG [U=1000,P=5653] mountGeneric() Mounting /cvmfs/atlas.cern.ch/repo/containers/sw/singularity/x86_64-el7/3.2.1/var/singularity/mnt/session/rootfs/root to /cvmfs/atlas.cern.ch/repo/containers/sw/singularity/x86_64-el7/3.2.1/var/singularity/mnt/session/underlay/root DEBUG [U=1000,P=5653] mountGeneric() Mounting /cvmfs/atlas.cern.ch/repo/containers/sw/singularity/x86_64-el7/3.2.1/var/singularity/mnt/session/rootfs/run to /cvmfs/atlas.cern.ch/repo/containers/sw/singularity/x86_64-el7/3.2.1/var/singularity/mnt/session/underlay/run DEBUG [U=1000,P=5653] mountGeneric() Mounting /cvmfs/atlas.cern.ch/repo/containers/sw/singularity/x86_64-el7/3.2.1/var/singularity/mnt/session/rootfs/scratch to /cvmfs/atlas.cern.ch/repo/containers/sw/singularity/x86_64-el7/3.2.1/var/singularity/mnt/session/underlay/scratch DEBUG [U=1000,P=5653] mountGeneric() Mounting /cvmfs/atlas.cern.ch/repo/containers/sw/singularity/x86_64-el7/3.2.1/var/singularity/mnt/session/rootfs/singularity-creation-date to /cvmfs/atlas.cern.ch/repo/containers/sw/singularity/x86_64-el7/3.2.1/var/singularity/mnt/session/underlay/singularity-creation-date DEBUG [U=1000,P=5653] mountGeneric() Mounting /cvmfs/atlas.cern.ch/repo/containers/sw/singularity/x86_64-el7/3.2.1/var/singularity/mnt/session/rootfs/srv to /cvmfs/atlas.cern.ch/repo/containers/sw/singularity/x86_64-el7/3.2.1/var/singularity/mnt/session/underlay/srv DEBUG [U=1000,P=5653] mountGeneric() Mounting /cvmfs/atlas.cern.ch/repo/containers/sw/singularity/x86_64-el7/3.2.1/var/singularity/mnt/session/rootfs/sys to /cvmfs/atlas.cern.ch/repo/containers/sw/singularity/x86_64-el7/3.2.1/var/singularity/mnt/session/underlay/sys DEBUG [U=1000,P=5653] mountGeneric() Mounting /cvmfs/atlas.cern.ch/repo/containers/sw/singularity/x86_64-el7/3.2.1/var/singularity/mnt/session/rootfs/tmp to /cvmfs/atlas.cern.ch/repo/containers/sw/singularity/x86_64-el7/3.2.1/var/singularity/mnt/session/underlay/tmp DEBUG [U=1000,P=5653] mountGeneric() Mounting /cvmfs/atlas.cern.ch/repo/containers/sw/singularity/x86_64-el7/3.2.1/var/singularity/mnt/session/rootfs/usr to /cvmfs/atlas.cern.ch/repo/containers/sw/singularity/x86_64-el7/3.2.1/var/singularity/mnt/session/underlay/usr DEBUG [U=1000,P=5653] mountGeneric() Mounting /cvmfs/atlas.cern.ch/repo/containers/sw/singularity/x86_64-el7/3.2.1/var/singularity/mnt/session/rootfs/var to /cvmfs/atlas.cern.ch/repo/containers/sw/singularity/x86_64-el7/3.2.1/var/singularity/mnt/session/underlay/var DEBUG [U=1000,P=5653] mountGeneric() Mounting /cvmfs/atlas.cern.ch/repo/containers/sw/singularity/x86_64-el7/3.2.1/var/singularity/mnt/session/underlay to /cvmfs/atlas.cern.ch/repo/containers/sw/singularity/x86_64-el7/3.2.1/var/singularity/mnt/session/final DEBUG [U=1000,P=5653] mountGeneric() Remounting /cvmfs/atlas.cern.ch/repo/containers/sw/singularity/x86_64-el7/3.2.1/var/singularity/mnt/session/final DEBUG [U=1000,P=5653] setPropagationMount() Set RPC mount propagation flag to SLAVE VERBOSE [U=1000,P=5653] Passwd() Checking for template passwd file: /cvmfs/atlas.cern.ch/repo/containers/sw/singularity/x86_64-el7/3.2.1/var/singularity/mnt/session/rootfs/etc/passwd VERBOSE [U=1000,P=5653] Passwd() Creating passwd content VERBOSE [U=1000,P=5653] Passwd() Creating template passwd file and appending user data: /cvmfs/atlas.cern.ch/repo/containers/sw/singularity/x86_64-el7/3.2.1/var/singularity/mnt/session/rootfs/etc/passwd DEBUG [U=1000,P=5653] addIdentityMount() Adding /etc/passwd to mount list VERBOSE [U=1000,P=5653] addIdentityMount() Default mount: /etc/passwd:/etc/passwd VERBOSE [U=1000,P=5653] Group() Checking for template group file: /cvmfs/atlas.cern.ch/repo/containers/sw/singularity/x86_64-el7/3.2.1/var/singularity/mnt/session/rootfs/etc/group VERBOSE [U=1000,P=5653] Group() Creating group content DEBUG [U=1000,P=5653] addIdentityMount() Adding /etc/group to mount list VERBOSE [U=1000,P=5653] addIdentityMount() Default mount: /etc/group:/etc/group DEBUG [U=1000,P=5653] mountGeneric() Mounting /dev to /cvmfs/atlas.cern.ch/repo/containers/sw/singularity/x86_64-el7/3.2.1/var/singularity/mnt/session/final/dev DEBUG [U=1000,P=5653] mountGeneric() Mounting /etc/localtime to /cvmfs/atlas.cern.ch/repo/containers/sw/singularity/x86_64-el7/3.2.1/var/singularity/mnt/session/final/usr/share/zoneinfo/UTC DEBUG [U=1000,P=5653] mountGeneric() Mounting /etc/hosts to /cvmfs/atlas.cern.ch/repo/containers/sw/singularity/x86_64-el7/3.2.1/var/singularity/mnt/session/final/etc/hosts DEBUG [U=1000,P=5653] mountGeneric() Mounting /cvmfs/atlas.cern.ch/repo/containers/sw/singularity/x86_64-el7/3.2.1/etc/singularity/actions to /cvmfs/atlas.cern.ch/repo/containers/sw/singularity/x86_64-el7/3.2.1/var/singularity/mnt/session/final/.singularity.d/actions DEBUG [U=1000,P=5653] mountGeneric() Remounting /cvmfs/atlas.cern.ch/repo/containers/sw/singularity/x86_64-el7/3.2.1/var/singularity/mnt/session/final/.singularity.d/actions DEBUG [U=1000,P=5653] mountGeneric() Mounting /proc to /cvmfs/atlas.cern.ch/repo/containers/sw/singularity/x86_64-el7/3.2.1/var/singularity/mnt/session/final/proc DEBUG [U=1000,P=5653] mountGeneric() Mounting /sys to /cvmfs/atlas.cern.ch/repo/containers/sw/singularity/x86_64-el7/3.2.1/var/singularity/mnt/session/final/sys DEBUG [U=1000,P=5653] mountGeneric() Mounting /home/boinc to /cvmfs/atlas.cern.ch/repo/containers/sw/singularity/x86_64-el7/3.2.1/var/singularity/mnt/session/home/boinc DEBUG [U=1000,P=5653] mountGeneric() Remounting /cvmfs/atlas.cern.ch/repo/containers/sw/singularity/x86_64-el7/3.2.1/var/singularity/mnt/session/home/boinc DEBUG [U=1000,P=5653] mountGeneric() Mounting /cvmfs/atlas.cern.ch/repo/containers/sw/singularity/x86_64-el7/3.2.1/var/singularity/mnt/session/home/boinc to /cvmfs/atlas.cern.ch/repo/containers/sw/singularity/x86_64-el7/3.2.1/var/singularity/mnt/session/final/home/boinc DEBUG [U=1000,P=5653] mountGeneric() Remounting /cvmfs/atlas.cern.ch/repo/containers/sw/singularity/x86_64-el7/3.2.1/var/singularity/mnt/session/final/home/boinc DEBUG [U=1000,P=5653] mountGeneric() Mounting /tmp to /cvmfs/atlas.cern.ch/repo/containers/sw/singularity/x86_64-el7/3.2.1/var/singularity/mnt/session/final/tmp DEBUG [U=1000,P=5653] mountGeneric() Remounting /cvmfs/atlas.cern.ch/repo/containers/sw/singularity/x86_64-el7/3.2.1/var/singularity/mnt/session/final/tmp DEBUG [U=1000,P=5653] mountGeneric() Mounting /var/tmp to /cvmfs/atlas.cern.ch/repo/containers/sw/singularity/x86_64-el7/3.2.1/var/singularity/mnt/session/final/var/tmp DEBUG [U=1000,P=5653] mountGeneric() Remounting /cvmfs/atlas.cern.ch/repo/containers/sw/singularity/x86_64-el7/3.2.1/var/singularity/mnt/session/final/var/tmp DEBUG [U=1000,P=5653] mountGeneric() Skipping mount /cvmfs/atlas.cern.ch/repo/containers/sw/singularity/x86_64-el7/3.2.1/var/singularity/mnt/session/final/home/boinc, /cvmfs/atlas.cern.ch/repo/containers/sw/singularity/x86_64-el7/3.2.1/var/singularity/mnt/session/final/home/boinc already mounted DEBUG [U=1000,P=5653] mountGeneric() Mounting /cvmfs/atlas.cern.ch/repo/containers/sw/singularity/x86_64-el7/3.2.1/var/singularity/mnt/session/etc/resolv.conf to /cvmfs/atlas.cern.ch/repo/containers/sw/singularity/x86_64-el7/3.2.1/var/singularity/mnt/session/final/etc/resolv.conf DEBUG [U=1000,P=5653] mountGeneric() Mounting /cvmfs/atlas.cern.ch/repo/containers/sw/singularity/x86_64-el7/3.2.1/var/singularity/mnt/session/etc/passwd to /cvmfs/atlas.cern.ch/repo/containers/sw/singularity/x86_64-el7/3.2.1/var/singularity/mnt/session/final/etc/passwd DEBUG [U=1000,P=5653] mountGeneric() Mounting /cvmfs/atlas.cern.ch/repo/containers/sw/singularity/x86_64-el7/3.2.1/var/singularity/mnt/session/etc/group to /cvmfs/atlas.cern.ch/repo/containers/sw/singularity/x86_64-el7/3.2.1/var/singularity/mnt/session/final/etc/group DEBUG [U=1000,P=5653] mountGeneric() Mounting /cvmfs to /cvmfs/atlas.cern.ch/repo/containers/sw/singularity/x86_64-el7/3.2.1/var/singularity/mnt/session/final/cvmfs DEBUG [U=1000,P=5653] mountGeneric() Remounting /cvmfs/atlas.cern.ch/repo/containers/sw/singularity/x86_64-el7/3.2.1/var/singularity/mnt/session/final/cvmfs DEBUG [U=1000,P=5653] create() Chroot into /cvmfs/atlas.cern.ch/repo/containers/sw/singularity/x86_64-el7/3.2.1/var/singularity/mnt/session/final DEBUG [U=1000,P=16014] Chroot() Change current directory to /cvmfs/atlas.cern.ch/repo/containers/sw/singularity/x86_64-el7/3.2.1/var/singularity/mnt/session/final DEBUG [U=1000,P=16014] Chroot() Hold reference to host / directory DEBUG [U=1000,P=16014] Chroot() Called pivot_root on /cvmfs/atlas.cern.ch/repo/containers/sw/singularity/x86_64-el7/3.2.1/var/singularity/mnt/session/final DEBUG [U=1000,P=16014] Chroot() Change current directory to host / directory DEBUG [U=1000,P=16014] Chroot() Apply slave mount propagation for host / directory DEBUG [U=1000,P=16014] Chroot() Called unmount(/, syscall.MNT_DETACH) DEBUG [U=1000,P=16014] Chroot() Changing directory to / to avoid getpwd issues DEBUG [U=1000,P=5653] create() Chdir into / to avoid errors VERBOSE [U=1000,P=16013] startup() Execute stage 2 DEBUG [U=1000,P=16013] Stage() Entering stage 2 DEBUG [U=1000,P=5653] PostStartProcess() Post start process boinc-z87 DEBUG [U=1000,P=5653] CleanupContainer() Cleanup container VERBOSE [U=1000,P=5653] CleanupContainer() Removing image /tmp/rootfs-490992350 INFO [U=1000,P=5653] CleanupContainer() Cleaning up image... DEBUG [U=1000,P=5653] Master() Child exited with exit status 0 boinc@boinc-z87:~$
12) Message boards : ATLAS application : native_mt app on Ubuntu 18.08 (Message 36580) Posted 28 Aug 2018 by [AF] Hydrosaure Post: I ran both commands; they complete correctly. Good catch with the thread number, I've lowered it to 4 for now. Digging some more into system logs, I've found these errors Aug 28 18:22:12 boinc-ab350m Singularity: action-suid (U=1000,P=19539)> Home directory is not owned by calling user: /home/boinc Aug 28 18:22:12 boinc-ab350m Singularity: action-suid (U=1000,P=19539)> Retval = 255 yet boinc@boinc-ab350m:/var/lib/boinc/slots$ ls -ld /home/boinc/ drwxr-xr-x 8 boinc boinc 23 Aug 28 19:25 /home/boinc/ I've also tried what is listed under http://opensciencegrid.org/docs/worker-node/install-singularity/#validating-singularity and it looks ok boinc@boinc-ab350m:/var/lib/boinc/slots$ singularity exec --contain --ipc --pid \ > --home $PWD:/srv \ > --bind /cvmfs \ > /cvmfs/singularity.opensciencegrid.org/opensciencegrid/osgvo:el6 \ > ps -ef WARNING: Container does not have an exec helper script, calling 'ps' directly UID PID PPID C STIME TTY TIME CMD boinc 1 0 0 17:32 ? 00:00:00 ps -ef
13) Message boards : ATLAS application : native_mt app on Ubuntu 18.08 (Message 36571) Posted 27 Aug 2018 by [AF] Hydrosaure Post: Anyone got native ATLAS app to run on Ubuntu 18.04 LTS ? I have a Ryzen machine running this OS and am trying to get it to crunch. cvmfs is installed and singularity 2.5.2 as well. Still tasks fail after about 10 minutes... Looking at the logs, it fails at this step: check singularity with cmd:singularity exec -B /cvmfs /cvmfs/atlas.cern.ch/repo/images/singularity/x86_64-slc6.img hostname Singularity isnt working... running start_atlas return value is 3 Yet manually running that command it seems fine: boinc@boinc-ab350m:/var/lib/boinc/slots$ singularity exec -B /cvmfs /cvmfs/atlas.cern.ch/repo/images/singularity/x86_64-slc6.img host name boinc-ab350m boinc@boinc-ab350m:/var/lib/boinc/slots$ echo $? 0 boinc@boinc-ab350m:/var/lib/boinc/slots$ Any idea? Host link: https://lhcathome.cern.ch/lhcathome/show_host_detail.php?hostid=10510464 Example of failed task: https://lhcathome.cern.ch/lhcathome/result.php?resultid=205978265

LHC@home