Message boards : Theory Application : Simple Bash script which sets everything up automatically to run native apps
Message board moderation

To post messages, you must log in.

AuthorMessage
gyllic

Send message
Joined: 9 Dec 14
Posts: 201
Credit: 2,500,279
RAC: 354
Message 39899 - Posted: 10 Sep 2019, 20:59:25 UTC

Hi guys,

I am working on a simple bash script which automatically sets everything up on various distributions (only ubuntu is implemented at the moment) to run the native applications. The script can be found at https://github.com/g84ycm/LHCHome_Native/. It is basically a collection of some forum posts, scripts and info taken from documentation. You need to have root privileges on your system in order to successfully run the script. The goal of this script is to simplify the setup process for new users in order to run the native apps.

To run the script, download it with
 wget https://raw.githubusercontent.com/g84ycm/LHCHome_Native/master/setup_native_LHC.sh 
and make it executable with
 chmod +x setup_native_LHC.sh 

The syntax to use the script is:
./setup_native_LHC.sh [distribution] [install boinc yes|no] 
More parameters will be added in later versions.
Type
./setup_native_LHC.sh help 
to get more info.

What the script does:

    - optionally installs boinc and boinc manager from repositories
    - installs CVMFS from cern repositories
    - sets up CVMFS to use a local cache and no local squid proxy
    - sets up CVMFS to use openhtc.io
    - checks if CVMFS is working correctly
    - sets up user namespaces (on ubuntu nothing needs to be changed)
    - checks if user namespaces are working
    - sets up cgroup stuff for suspend/resume to work (for native theory)
    - creates systemd script for suspend/resume to work


To Do:

    - implement everything from above for different distributions (openSUSE, CentOS, ...). Some needed distinctions are already within the code.
    - add option to build everything from sources



For the moment, only native theory will work. However, at the -dev project native ATLAS is running without a local singularity installation, so in future this script should set everything up to run native theory and ATLAS.

This is my first "larger" bash script, so I am sure that there are better and/or more elegant ways to implemet the different steps. Any hints for improvements, bug reports, code suggestions or pull requests would be much appreciated.

Right now the script has only been tested on ubuntu 18.04.3 with native theory.

gyllic

ID: 39899 · Report as offensive     Reply Quote
Jim1348

Send message
Joined: 15 Nov 14
Posts: 330
Credit: 10,816,296
RAC: 18,876
Message 39912 - Posted: 13 Sep 2019, 2:01:04 UTC - in response to Message 39899.  

That is a lot of good work on your part. I already have CVMFS up and running on two machines, one for native Theory and the other for native ATLAS.
But I would like to implement a "local cache" for CVMFS.

Is there a way that I could add a local cache to an existing Ubuntu 18.04 installation?

Thanks.
ID: 39912 · Report as offensive     Reply Quote
computezrmle
Avatar

Send message
Joined: 15 Jun 08
Posts: 1144
Credit: 56,214,640
RAC: 96,639
Message 39913 - Posted: 13 Sep 2019, 7:07:59 UTC - in response to Message 39912.  

CVMFS is a local cache.
It serves all ATLAS native and Theory native instances running on the same computer.

If you need a more general local cache, e.g. to serve a couple of CVMFS caches either native or inside VMs, a squid proxy is recommended.
ID: 39913 · Report as offensive     Reply Quote
Jim1348

Send message
Joined: 15 Nov 14
Posts: 330
Credit: 10,816,296
RAC: 18,876
Message 39916 - Posted: 13 Sep 2019, 10:56:37 UTC - in response to Message 39913.  
Last modified: 13 Sep 2019, 11:01:01 UTC

CVMFS is a local cache.

OK, thanks. I was not sure whether I had it already or not, since he lists it as a separate item.
(I have had to wipe out my machines since last doing a squid proxy, and will need to give it a go again.)
ID: 39916 · Report as offensive     Reply Quote
wolfman1360

Send message
Joined: 17 Feb 17
Posts: 4
Credit: 12,660
RAC: 1
Message 40063 - Posted: 2 Oct 2019, 3:35:59 UTC

I want to give you a great big hug right now.
I am far from a Linux expert. I completely ( and I mean completely) borked up a machine running Linux (Ubuntu 1804) attempting to install this (apart from boinc) all on my own. At this point, I'm failing to remember the specifics, but I do remember quite clearly that I was having issues with CVMFS. I vaguely remember it attempting to replace various libraries having to do with boinc as well as a few other things. Thinking nothing of it, as the machine was running headless and Linux has never given me any issues before, I went ahead with it.
I received several errors after this - again, I don't specifically remember the steps taken here - only that when attempting to reboot the system, it failed to come back up, and physically getting to it later found it to be in an infinit bootloop. I do remember that cvmfs_config probe was getting me nowhere at the time. Said machine is still sitting here, so I will throw Ubuntu 1804 on it again and start from scratch.

Regardless, I look forward to giving this another shot. This is my excuse to delve into Linux and break things. I'm still confused as to what exactly this application does vs. virtualbox. Does this require an alternative version of Boinc to run alongside, or am I missing something completely and utterly obvious? Of course I am probably overcomplicating just about everything. Seems like this also uses a ton of internet traffic, which I have no issues with.
Sorry for the very drawn out post and perhaps silly questions.
thank you again for putting this together and if there is anything I can do to help let me know.
ID: 40063 · Report as offensive     Reply Quote

Message boards : Theory Application : Simple Bash script which sets everything up automatically to run native apps


©2019 CERN