Message boards : ATLAS application : A few errors, tried everything I can
Message board moderation

To post messages, you must log in.

AuthorMessage
wolfman1360

Send message
Joined: 17 Feb 17
Posts: 42
Credit: 1,564,296
RAC: 4
Message 44432 - Posted: 4 Mar 2021, 17:40:53 UTC

Hi,
A few errors on multiple machines, Ubuntu 20.04.
[2021-03-04 08:18:44] FATAL:  while extracting /cvmfs/atlas.cern.ch/repo/containers/images/singularity/x86_64-centos7.img: root filesystem extraction failed: extract command failed: exit status 1
https://lhcathome.cern.ch/lhcathome/result.php?resultid=304481909
And this one.
[2021-03-04 13:04:23] FATAL:  container creation failed: mount ->/var error: can't remount /var: operation not permitted
https://lhcathome.cern.ch/lhcathome/result.php?resultid=304481865
No idea how to go about fixing these.
ID: 44432 · Report as offensive     Reply Quote
maeax

Send message
Joined: 2 May 07
Posts: 1301
Credit: 39,561,289
RAC: 11,747
Message 44434 - Posted: 5 Mar 2021, 6:07:24 UTC - in response to Message 44432.  
Last modified: 5 Mar 2021, 6:08:39 UTC

have the same info from CentOS8.
After installing singularity from OS - Tasks are running well.
https://lhcathome.cern.ch/lhcathome/forum_thread.php?id=5613
ID: 44434 · Report as offensive     Reply Quote
wolfman1360

Send message
Joined: 17 Feb 17
Posts: 42
Credit: 1,564,296
RAC: 4
Message 44437 - Posted: 5 Mar 2021, 7:46:13 UTC - in response to Message 44434.  

have the same info from CentOS8.
After installing singularity from OS - Tasks are running well.
https://lhcathome.cern.ch/lhcathome/forum_thread.php?id=5613

Thank you. I did sudo apt-get install singularity, so we'll see what happens.
thanks
ID: 44437 · Report as offensive     Reply Quote
Jim1348

Send message
Joined: 15 Nov 14
Posts: 568
Credit: 17,795,575
RAC: 25,114
Message 44442 - Posted: 5 Mar 2021, 14:27:19 UTC - in response to Message 44437.  

Thank you. I did sudo apt-get install singularity, so we'll see what happens.

I think you get an old version that way, though I have not tried it recently. It might work.

You can get the latetest version by following these instructions:
https://sylabs.io/guides/3.0/user-guide/installation.html
Note that you have to install GO first.

The following works for me on Ubuntu 18.04/20.04.
Not all versions of GO and Singularity work for me, but these do.
You can find the latest versions as indicated.

Install GO (check latest version at https://pkg.go.dev/golang.org/dl#pkg-subdirectories):

export VERSION=1.15.2 OS=linux ARCH=amd64 && \
    wget https://dl.google.com/go/go$VERSION.$OS-$ARCH.tar.gz && \
    sudo tar -C /usr/local -xzvf go$VERSION.$OS-$ARCH.tar.gz && \
    rm go$VERSION.$OS-$ARCH.tar.gz

echo 'export GOPATH=${HOME}/go' >> ~/.bashrc && \
    echo 'export PATH=/usr/local/go/bin:${PATH}:${GOPATH}/bin' >> ~/.bashrc && \
    source ~/.bashrc

go get -u github.com/golang/dep/cmd/dep


-----------------------------------------------------------------------------------

Install Singularity (check latest version at https://github.com/hpcng/singularity/releases):

export VERSION=3.6.4 && # adjust this as necessary \
    mkdir -p $GOPATH/src/github.com/sylabs && \
    cd $GOPATH/src/github.com/sylabs && \
    wget https://github.com/sylabs/singularity/releases/download/v${VERSION}/singularity-${VERSION}.tar.gz && \
    tar -xzf singularity-${VERSION}.tar.gz && \
    cd ./singularity && \
    ./mconfig

./mconfig && \
    make -C ./builddir && \
    sudo make -C ./builddir install


Check Version: singularity --version
ID: 44442 · Report as offensive     Reply Quote
wolfman1360

Send message
Joined: 17 Feb 17
Posts: 42
Credit: 1,564,296
RAC: 4
Message 44449 - Posted: 5 Mar 2021, 20:43:23 UTC - in response to Message 44442.  

Thank you. I did sudo apt-get install singularity, so we'll see what happens.

I think you get an old version that way, though I have not tried it recently. It might work.

You can get the latetest version by following these instructions:
https://sylabs.io/guides/3.0/user-guide/installation.html
Note that you have to install GO first.

The following works for me on Ubuntu 18.04/20.04.
Not all versions of GO and Singularity work for me, but these do.
You can find the latest versions as indicated.

Install GO (check latest version at https://pkg.go.dev/golang.org/dl#pkg-subdirectories):

export VERSION=1.15.2 OS=linux ARCH=amd64 && \
    wget https://dl.google.com/go/go$VERSION.$OS-$ARCH.tar.gz && \
    sudo tar -C /usr/local -xzvf go$VERSION.$OS-$ARCH.tar.gz && \
    rm go$VERSION.$OS-$ARCH.tar.gz

echo 'export GOPATH=${HOME}/go' >> ~/.bashrc && \
    echo 'export PATH=/usr/local/go/bin:${PATH}:${GOPATH}/bin' >> ~/.bashrc && \
    source ~/.bashrc

go get -u github.com/golang/dep/cmd/dep


-----------------------------------------------------------------------------------

Install Singularity (check latest version at https://github.com/hpcng/singularity/releases):

export VERSION=3.6.4 && # adjust this as necessary \
    mkdir -p $GOPATH/src/github.com/sylabs && \
    cd $GOPATH/src/github.com/sylabs && \
    wget https://github.com/sylabs/singularity/releases/download/v${VERSION}/singularity-${VERSION}.tar.gz && \
    tar -xzf singularity-${VERSION}.tar.gz && \
    cd ./singularity && \
    ./mconfig

./mconfig && \
    make -C ./builddir && \
    sudo make -C ./builddir install


Check Version: singularity --version

Thanks so much. I'll give this a try if the above method fails.
Bought two year old but new Ryzen 5 3600's. They'll be a marked improvement over what I have currently, especially in the summer.
ID: 44449 · Report as offensive     Reply Quote
wolfman1360

Send message
Joined: 17 Feb 17
Posts: 42
Credit: 1,564,296
RAC: 4
Message 44457 - Posted: 7 Mar 2021, 19:54:05 UTC - in response to Message 44442.  

Thank you. I did sudo apt-get install singularity, so we'll see what happens.

I think you get an old version that way, though I have not tried it recently. It might work.

You can get the latetest version by following these instructions:
https://sylabs.io/guides/3.0/user-guide/installation.html
Note that you have to install GO first.

The following works for me on Ubuntu 18.04/20.04.
Not all versions of GO and Singularity work for me, but these do.
You can find the latest versions as indicated.

Install GO (check latest version at https://pkg.go.dev/golang.org/dl#pkg-subdirectories):

export VERSION=1.15.2 OS=linux ARCH=amd64 && \
    wget https://dl.google.com/go/go$VERSION.$OS-$ARCH.tar.gz && \
    sudo tar -C /usr/local -xzvf go$VERSION.$OS-$ARCH.tar.gz && \
    rm go$VERSION.$OS-$ARCH.tar.gz

echo 'export GOPATH=${HOME}/go' >> ~/.bashrc && \
    echo 'export PATH=/usr/local/go/bin:${PATH}:${GOPATH}/bin' >> ~/.bashrc && \
    source ~/.bashrc

go get -u github.com/golang/dep/cmd/dep


-----------------------------------------------------------------------------------

Install Singularity (check latest version at https://github.com/hpcng/singularity/releases):

export VERSION=3.6.4 && # adjust this as necessary \
    mkdir -p $GOPATH/src/github.com/sylabs && \
    cd $GOPATH/src/github.com/sylabs && \
    wget https://github.com/sylabs/singularity/releases/download/v${VERSION}/singularity-${VERSION}.tar.gz && \
    tar -xzf singularity-${VERSION}.tar.gz && \
    cd ./singularity && \
    ./mconfig

./mconfig && \
    make -C ./builddir && \
    sudo make -C ./builddir install


Check Version: singularity --version

It seems as though most of this is well above my head and knowledge. Or maybe it's simply paid work followed by 'free work and troubleshooting' as it were.
Things still don't appear to be functioning correctly. I do not appear to be out of memory.
Have switched over to Rosetta, for now, until I can troubleshoot this and get it functional. Will start with one machine at a time.
I may reinstall Ubuntu 20.04 from scratch and go through everything again. At this point I'm not sure what else to try. The most frustrating part - I have not done anything different on the machines that are able to complete these tasks versus the ones that are not, to my knowledge, so I'm at a loss as to where to begin.
ID: 44457 · Report as offensive     Reply Quote
Jim1348

Send message
Joined: 15 Nov 14
Posts: 568
Credit: 17,795,575
RAC: 25,114
Message 44485 - Posted: 14 Mar 2021, 17:01:21 UTC - in response to Message 44457.  

It seems as though most of this is well above my head and knowledge. Or maybe it's simply paid work followed by 'free work and troubleshooting' as it were.
Things still don't appear to be functioning correctly. I do not appear to be out of memory.

Well I just tried it on a new Ubuntu 20.04.2 machine, and it didn't work for me either, if that makes you feel any better.

After considerable trial-and-error and searching I found that this worked. I will reserve commenting on the Linux "developers" for another occasion.

First: install Dependencies:
sudo apt-get update && sudo apt-get install -y \
build-essential \
libssl-dev \
uuid-dev \
libgpgme11-dev \
squashfs-tools \
libseccomp-dev \
wget \
pkg-config \
git \
cryptsetup

Second: install GO
sudo apt install git
sudo snap install go --classic

sudo echo 'export GOPATH=${HOME}/go' >> ~/.bashrc && \
echo 'export PATH=/usr/local/go/bin:${PATH}:${GOPATH}/bin' >> ~/.bashrc && \
source ~/.bashrc

go get -u github.com/golang/dep/cmd/dep
(wait for the download!)

Check version: go version


Third install Singularity (latest version at https://github.com/hpcng/singularity/releases, though this works for me):
For reference, see: https://sylabs.io/guides/3.7/user-guide/quick_start.html

go get -d github.com/sylabs/singularity

export VERSION=3.7.2 && \
wget https://github.com/hpcng/singularity/releases/download/v${VERSION}/singularity-${VERSION}.tar.gz && \
tar -xzf singularity-${VERSION}.tar.gz && \
cd singularity

./mconfig && \
make -C ./builddir && \
sudo make -C ./builddir install

Check Version: singularity --version
ID: 44485 · Report as offensive     Reply Quote
Profile F6FGZ looking for DX !

Send message
Joined: 7 Jan 07
Posts: 39
Credit: 15,191,338
RAC: 363
Message 44994 - Posted: 21 May 2021, 13:53:23 UTC - in response to Message 44485.  
Last modified: 21 May 2021, 14:22:08 UTC

@Jim1348 : Thanks for the tip
As I always prioritize packages over sources, I modified your process as follow :

Needed dependencies for both cases :
build-essential uuid-dev squashfs-tools wget pkg-config

Modified process validated on Ubuntu 20.04.2.
    (1) Install Go language from the package repository :
    $ sudo apt install golang

    (2) Modified PATH without /usr/local/go/bin :
    $ sudo echo 'export GOPATH=${HOME}/go' >> ~/.bashrc && \
    $ echo 'export PATH=${PATH}:${GOPATH}/bin' >> ~/.bashrc && \
    $ source ~/.bashrc
    $ go get -u github.com/golang/dep/cmd/dep
    $ go version
    go version go1.13.8 linux/amd64
    


Modified process validated on Debian 10 (Buster).

    (1) Install Go language using temporarily backports package repository :
    $ echo 'deb http://deb.debian.org/debian buster-backports main contrib non-free'|sudo tee -a /etc/apt/sources.list
    $ sudo apt update
    $ sudo apt install -t buster-backports golang
    $ sudo sed -i '$d' /etc/apt/sources.list
    $ sudo apt update

    (2) Modified PATH without /usr/local/go/bin :
    $ sudo echo 'export GOPATH=${HOME}/go' >> ~/.bashrc && \
    $ echo 'export PATH=${PATH}:${GOPATH}/bin' >> ~/.bashrc && \
    $ source ~/.bashrc
    $ go get -u github.com/golang/dep/cmd/dep
    $ go version
    go version go1.14 linux/amd64
    


Then for both cases, install Singularity the way you describe, using 3.7.3 for VERSION :

$  singularity --version
singularity version 3.7.3
ID: 44994 · Report as offensive     Reply Quote
Profile F6FGZ looking for DX !

Send message
Joined: 7 Jan 07
Posts: 39
Credit: 15,191,338
RAC: 363
Message 44995 - Posted: 21 May 2021, 19:26:47 UTC - in response to Message 44994.  
Last modified: 21 May 2021, 19:29:16 UTC

Latest improvements :

PATH does not need to be modified since we use package for Go :
$ sudo echo 'export GOPATH=${HOME}/go' >> ~/.bashrc && \
$ source ~/.bashrc

This command is useless since we install Singularity from the tarball :
go get -d github.com/sylabs/singularity
ID: 44995 · Report as offensive     Reply Quote

Message boards : ATLAS application : A few errors, tried everything I can


©2021 CERN