Message boards : ATLAS application : Native Atlas: "Output file missing"
Message board moderation

To post messages, you must log in.

AuthorMessage
OldChap

Send message
Joined: 7 May 17
Posts: 16
Credit: 1,456,154
RAC: 0
Message 38410 - Posted: 24 Mar 2019, 19:08:55 UTC

Running Mint 19.1
Somehow I seem to have cvmfs reporting it is running OK.
Having had issues with Singularity when trying to use the Debian guide I thought I would try the version in the Mint repository.

End result is that with an app_config running 8 * 4t each with 3300MB allocated on 32t 32GB rig, all seems well for 15 Mins or so then computation error and log showing "Output file missing"

I am guessing that this is a permissions issue so I am hoping one of you knows a quick fix

Meanwhile I found https://lhcathome.cern.ch/lhcathome/forum_thread.php?id=4396&postid=34494 which seems like my next try if you think I should start over.

Thoughts anyone?
ID: 38410 · Report as offensive     Reply Quote
Gunde

Send message
Joined: 9 Jan 15
Posts: 33
Credit: 256,197,294
RAC: 117,795
Message 38412 - Posted: 24 Mar 2019, 20:00:28 UTC - in response to Message 38410.  
Last modified: 24 Mar 2019, 20:03:15 UTC

This is not SLC6, need to run with Singularity....
Checking Singularity...
Singularity is installed
copy /var/lib/boinc-client/slots/5/shared/input.tar.gz
copy /var/lib/boinc-client/slots/5/shared/copy_2LbbWw
copy /var/lib/boinc-client/slots/5/shared/ATLAS.root_0
copy /var/lib/boinc-client/slots/5/shared/RTE.tar.gz
copy /var/lib/boinc-client/slots/5/shared/start_atlas.sh
export ATHENA_PROC_NUMBER=2;start atlas job with 
Testing the function of Singularity...
check singularity with cmd:singularity exec -B /cvmfs /cvmfs/atlas.cern.ch/repo/images/singularity/x86_64-slc6.img hostname

Singularity isnt working...


Looks like Singularity is broken. Did you use version 2.4.2? That guide should work and 2.4.2 been great for my host.
ID: 38412 · Report as offensive     Reply Quote
Crystal Pellet
Volunteer moderator
Volunteer tester

Send message
Joined: 14 Jan 10
Posts: 717
Credit: 5,978,236
RAC: 2,178
Message 38413 - Posted: 24 Mar 2019, 20:05:31 UTC - in response to Message 38410.  

...running 8 * 4t each with 3300MB allocated......
Thoughts anyone?
I think your system is overloaded even when you have 32GB RAM.

For ATLAS-native you need 1900MB of RAM for each worker=thread. 8 * 4 * 1900 > 32GB.
ID: 38413 · Report as offensive     Reply Quote
OldChap

Send message
Joined: 7 May 17
Posts: 16
Credit: 1,456,154
RAC: 0
Message 38414 - Posted: 24 Mar 2019, 20:36:23 UTC - in response to Message 38412.  

still playing with singularity

The thread at https://lhcathome.cern.ch/lhcathome/forum_thread.php?id=4875&postid=37372#37372 seems to suggest that native Atlas requires 2100MB + 300MB*nCPUs. Hence the 3300 chosen
ID: 38414 · Report as offensive     Reply Quote
computezrmle

Send message
Joined: 15 Jun 08
Posts: 1102
Credit: 52,329,604
RAC: 147,148
Message 38415 - Posted: 24 Mar 2019, 20:43:07 UTC - in response to Message 38414.  

Sure you did not only install CVMFS but it is also configured correctly?
Do you get this output when you run "cvmfs_config probe"?
Probing /cvmfs/atlas.cern.ch... OK
Probing /cvmfs/atlas-condb.cern.ch... OK
Probing /cvmfs/grid.cern.ch... OK
Probing /cvmfs/cernvm-prod.cern.ch... OK

It's just that it makes no sense to check for a singularity error if CVMFS is not working since singularity needs files from the CVMFS repository.
ID: 38415 · Report as offensive     Reply Quote
OldChap

Send message
Joined: 7 May 17
Posts: 16
Credit: 1,456,154
RAC: 0
Message 38416 - Posted: 24 Mar 2019, 21:21:10 UTC - in response to Message 38415.  
Last modified: 24 Mar 2019, 21:23:28 UTC

$ cvmfs_config probe
Probing /cvmfs/atlas.cern.ch... OK
Probing /cvmfs/atlas-condb.cern.ch... OK
Probing /cvmfs/grid.cern.ch... OK
~$

One less than you ?
This is after reinstall ...not attached to LHC yet if that matterrs
ID: 38416 · Report as offensive     Reply Quote
computezrmle

Send message
Joined: 15 Jun 08
Posts: 1102
Credit: 52,329,604
RAC: 147,148
Message 38418 - Posted: 24 Mar 2019, 21:56:25 UTC - in response to Message 38416.  

Your's is OK if you only run ATLAS.
cernvm-prod is only necessary for Theory native.
Accidentally left it in the list.

Could you run the suggested test command:
singularity exec -B /cvmfs /cvmfs/atlas.cern.ch/repo/images/singularity/x86_64-slc6.img hostname

This should print your hostname.
If you don't want to post it here, just confirm that it is the command's output.
ID: 38418 · Report as offensive     Reply Quote
OldChap

Send message
Joined: 7 May 17
Posts: 16
Credit: 1,456,154
RAC: 0
Message 38419 - Posted: 24 Mar 2019, 22:21:05 UTC - in response to Message 38418.  
Last modified: 24 Mar 2019, 22:24:45 UTC

Hmmm not right now.

When following the guide I mentioned in my first post re: Singularity, it all falls apart after that slightly modified download is unzipped into the /home/username folder. I have been looking around for what and where to cd to in order to "./configure --prefix=/usr/local"
ID: 38419 · Report as offensive     Reply Quote
mmonnin

Send message
Joined: 22 Mar 17
Posts: 44
Credit: 3,801,950
RAC: 166
Message 38422 - Posted: 24 Mar 2019, 23:55:52 UTC - in response to Message 38419.  

Hmmm not right now.

When following the guide I mentioned in my first post re: Singularity, it all falls apart after that slightly modified download is unzipped into the /home/username folder. I have been looking around for what and where to cd to in order to "./configure --prefix=/usr/local"


Singularity install:

sudo wget -O- http://neuro.debian.net/lists/xenial.us-ca.full | sudo tee /etc/apt/sources.list.d/neurodebian.sources.list
sudo apt-get install -y singularity-container


And a
singularity --version

will return the version if installed correctly.
ID: 38422 · Report as offensive     Reply Quote
OldChap

Send message
Joined: 7 May 17
Posts: 16
Credit: 1,456,154
RAC: 0
Message 38423 - Posted: 25 Mar 2019, 0:06:24 UTC - in response to Message 38419.  
Last modified: 25 Mar 2019, 0:09:03 UTC

Now having installed singularity using :

sudo wget -O- http://neuro.debian.net/lists/bionic.us-ca.full | sudo tee /etc/apt/sources.list.d/neurodebian.sources.list (I am on Mint 19.1 so Bionic on Ubuntu)

Version shows fine

The answer to your question is yes

It is an older version but.......

Am currently running another 8 and progress so far even after a couple of re-starts is beyond where I was before having reached 25% in around 45 mins on a pair of E5-2670's

BoincTasks is reporting cpu just now @ 237% but it is climbing so maybe when I watch an undisturbed run it will improve
ID: 38423 · Report as offensive     Reply Quote

Message boards : ATLAS application : Native Atlas: "Output file missing"


©2019 CERN