1)
Message boards :
ATLAS application :
ATLAS vbox and native 3.01
(Message 48644)
Posted 23 Sep 2023 by zepingouin Post: I modified the command line to monitor ATLAS native 3.01: In this example, I used 2 CPUs per task hence the tail -n2. sudo watch -n10 "find /var/lib/boinc-client/slots/ \( -name \"log.EVNTtoHITS\" -o -name \"AthenaMP.log\" \) |sort |xargs -I {} -n1 sh -c \"egrep 'INFO.*Run:Event ' {} |tail -n2\"|sort -k 7,7" An example of output: 17:32:48 ISF_Kernel_FullG4MT_QS.ISF_LongLivedGeant4Tool 390 0 INFO Run:Event 450000:20848791 (200th event for this worker) took 82.67 s. New average 93.67 +- 3.91 17:32:44 ISF_Kernel_FullG4MT_QS.ISF_LongLivedGeant4Tool 391 1 INFO Run:Event 450000:20848792 (192th event for this worker) took 45.15 s. New average 98.49 +- 3.622 17:32:03 ISF_Kernel_FullG4MT_QS.ISF_LongLivedGeant4Tool 362 0 INFO Run:Event 450000:22570763 (186th event for this worker) took 40.7 s. New average 96.78 +- 3.699 17:32:53 ISF_Kernel_FullG4MT_QS.ISF_LongLivedGeant4Tool 363 1 INFO Run:Event 450000:22570764 (178th event for this worker) took 128.6 s. New average 102.1 +- 4.028 17:33:07 ISF_Kernel_FullG4MT_QS.ISF_LongLivedGeant4Tool 312 1 INFO Run:Event 450000:22644313 (159th event for this worker) took 209.2 s. New average 95.61 +- 3.997 17:33:01 ISF_Kernel_FullG4MT_QS.ISF_LongLivedGeant4Tool 313 0 INFO Run:Event 450000:22644314 (155th event for this worker) took 152.8 s. New average 99 +- 4.297 |
2)
Message boards :
Number crunching :
Setting up a local Squid to work with LHC@home - Comments and Questions
(Message 48454)
Posted 17 Aug 2023 by zepingouin Post: Is there an easier way to do this? I'm surprised I can't find a simple windows program to cache internet access from other computers on the local network. I compiled Squid for use with Windows 10 and Cygwin, you may find binaries and instruction on this thread. Latest is Squid-6.2 and works fine. |
3)
Message boards :
ATLAS application :
Creation of container failed
(Message 45060)
Posted 16 Jun 2021 by zepingouin Post: Working workarounds can be find in these posts. |
4)
Message boards :
ATLAS application :
Guide for building everything from sources to run native ATLAS on Debian 9 (Stretch) Version 2
(Message 45019)
Posted 27 May 2021 by zepingouin Post: Singularity 3.7.3 is current. I updated the procedure mentioned above. |
5)
Message boards :
ATLAS application :
A few errors, tried everything I can
(Message 44995)
Posted 21 May 2021 by zepingouin Post: Latest improvements : PATH does not need to be modified since we use package for Go : $ sudo echo 'export GOPATH=${HOME}/go' >> ~/.bashrc && \ $ source ~/.bashrc This command is useless since we install Singularity from the tarball : go get -d github.com/sylabs/singularity |
6)
Message boards :
ATLAS application :
A few errors, tried everything I can
(Message 44994)
Posted 21 May 2021 by zepingouin Post: @Jim1348 : Thanks for the tip As I always prioritize packages over sources, I modified your process as follow : Needed dependencies for both cases : build-essential uuid-dev squashfs-tools wget pkg-config Modified process validated on Ubuntu 20.04.2.
$ sudo apt install golang (2) Modified PATH without /usr/local/go/bin : $ sudo echo 'export GOPATH=${HOME}/go' >> ~/.bashrc && \ $ echo 'export PATH=${PATH}:${GOPATH}/bin' >> ~/.bashrc && \ $ source ~/.bashrc $ go get -u github.com/golang/dep/cmd/dep $ go version go version go1.13.8 linux/amd64
$ echo 'deb http://deb.debian.org/debian buster-backports main contrib non-free'|sudo tee -a /etc/apt/sources.list $ sudo apt update $ sudo apt install -t buster-backports golang $ sudo sed -i '$d' /etc/apt/sources.list $ sudo apt update (2) Modified PATH without /usr/local/go/bin : $ sudo echo 'export GOPATH=${HOME}/go' >> ~/.bashrc && \ $ echo 'export PATH=${PATH}:${GOPATH}/bin' >> ~/.bashrc && \ $ source ~/.bashrc $ go get -u github.com/golang/dep/cmd/dep $ go version go version go1.14 linux/amd64
$ singularity --version singularity version 3.7.3 |
7)
Message boards :
ATLAS application :
Creation of container failed
(Message 44897)
Posted 6 May 2021 by zepingouin Post: Looking into errors, I found that the problem is the use of wrong path by container : 09:56:16 (130425): wrapper (7.7.26015): starting 09:56:16 (130425): wrapper: running run_atlas (--nthreads 6) [2021-05-06 09:56:16] Arguments: --nthreads 6 [2021-05-06 09:56:16] Threads: 6 [2021-05-06 09:56:16] Checking for CVMFS [2021-05-06 09:56:16] Probing /cvmfs/atlas.cern.ch... OK [2021-05-06 09:56:16] Probing /cvmfs/atlas-condb.cern.ch... OK [2021-05-06 09:56:16] Running cvmfs_config stat atlas.cern.ch [2021-05-06 09:56:16] VERSION PID UPTIME(M) MEM(K) REVISION EXPIRES(M) NOCATALOGS CACHEUSE(K) CACHEMAX(K) NOFDUSE NOFDMAX NOIOERR NOOPEN HITRATE(%) RX(K) SPEED(K/S) HOST PROXY ONLINE [2021-05-06 09:56:16] 2.8.1.0 129901 5 25280 84062 2 3 4718377 6144001 0 130560 0 36 99.909 528 724 http://s1ral-cvmfs.openhtc.io/cvmfs/atlas.cern.ch http://192.168.2.1:3128 1 [2021-05-06 09:56:16] CVMFS is ok [2021-05-06 09:56:16] Using singularity image /cvmfs/atlas.cern.ch/repo/containers/images/singularity/x86_64-centos7.img [2021-05-06 09:56:16] Checking for singularity binary... [2021-05-06 09:56:16] Singularity is not installed, using version from CVMFS [2021-05-06 09:56:16] Checking singularity works with /cvmfs/atlas.cern.ch/repo/containers/sw/singularity/x86_64-el7/current/bin/singularity exec -B /cvmfs /cvmfs/atlas.cern.ch/repo/containers/images/singularity/x86_64-centos7.img hostname [2021-05-06 09:56:18] INFO: Converting SIF file to temporary sandbox... fonck INFO: Cleaning up image... [2021-05-06 09:56:18] Singularity works [2021-05-06 09:56:18] Set ATHENA_PROC_NUMBER=6 [2021-05-06 09:56:18] Starting ATLAS job with PandaID=5047215526 [2021-05-06 09:56:18] Running command: /cvmfs/atlas.cern.ch/repo/containers/sw/singularity/x86_64-el7/current/bin/singularity exec --pwd /var/lib/boinc-client/slots/7 -B /cvmfs,/var /cvmfs/atlas.cern.ch/repo/containers/images/singularity/x86_64-centos7.img sh start_atlas.sh [2021-05-06 09:56:20] Job failed [2021-05-06 09:56:20] INFO: Converting SIF file to temporary sandbox... [2021-05-06 09:56:20] INFO: Cleaning up image... [2021-05-06 09:56:20] FATAL: container creation failed: mount /cvmfs/atlas.cern.ch/repo/containers/sw/singularity/x86_64-el7/3.7.2/var/singularity/mnt/session/rootfs/var/lib/package-list->/cvmfs/atlas.cern.ch/repo/containers/sw/singularity/x86_64-el7/3.7.2/var/singularity/mnt/session/underlay/var/lib/package-list error: while mounting /cvmfs/atlas.cern.ch/repo/containers/sw/singularity/x86_64-el7/3.7.2/var/singularity/mnt/session/rootfs/var/lib/package-list: destination /cvmfs/atlas.cern.ch/repo/containers/sw/singularity/x86_64-el7/3.7.2/var/singularity/mnt/session/underlay/var/lib/package-list doesn't exist in container [2021-05-06 09:56:20] ./runtime_log.err [2021-05-06 09:56:20] ./runtime_log For testing, I changed temporarily write permission for /var/lib : sudo chmod o+w /var/lib/ As you can see, all files are created there ... $ ls -lh /var/lib/|grep boinc drwxr-xr-x 2 boinc boinc 4,0K mai 5 23:49 alternatives lrwxrwxrwx 1 boinc boinc 12 mai 22 2018 boinc -> boinc-client drwxr-xr-x 8 boinc boinc 4,0K mai 6 09:57 boinc-client drwxr-xr-x 2 boinc boinc 4,0K mai 6 00:12 condor drwxr-xr-x 2 boinc boinc 4,0K mai 6 00:28 cs drwxr-xr-x 2 boinc boinc 4,0K mai 6 09:47 games drwxr-xr-x 2 boinc boinc 4,0K mai 6 09:56 gssproxy drwxr-xr-x 2 boinc boinc 4,0K mai 6 09:56 initramfs drwxr-xr-x 2 boinc boinc 4,0K mai 6 09:56 machines drwxr-xr-x 2 boinc boinc 4,0K mai 6 09:56 ntp -rw-r--r-- 1 boinc boinc 0 mai 6 09:56 package-list drwxr-xr-x 2 boinc boinc 4,0K mai 6 09:56 rpcbind drwxr-xr-x 2 boinc boinc 4,0K mai 6 09:56 rpm drwxr-xr-x 2 boinc boinc 4,0K mai 6 09:56 rpm-state drwxr-xr-x 2 boinc boinc 4,0K mai 6 09:56 texmf There are some parts missing at the mount point : $ ls /cvmfs/atlas.cern.ch/repo/containers/sw/singularity/x86_64-el7/3.7.2/var/singularity/mnt/session/ $ |
8)
Message boards :
ATLAS application :
Ubuntu 20.04 support?
(Message 44622)
Posted 30 Mar 2021 by zepingouin Post: Atlas Native tasks are running fine under Ubuntu 20.04.2 without singularity. [2021-03-28 08:34:43] CVMFS is ok [2021-03-28 08:34:43] Using singularity image /cvmfs/atlas.cern.ch/repo/containers/images/singularity/x86_64-centos7.img [2021-03-28 08:34:43] Checking for singularity binary... [2021-03-28 08:34:43] Singularity is not installed, using version from CVMFS [2021-03-28 08:34:43] Checking singularity works with /cvmfs/atlas.cern.ch/repo/containers/sw/singularity/x86_64-el7/current/bin/singularity exec -B /cvmfs /cvmfs/atlas.cern.ch/repo/containers/images/singularity/x86_64-centos7.img hostname [2021-03-28 08:34:45] [34mINFO: [0m Convert SIF file to sandbox... fonck [34mINFO: [0m Cleaning up image... [2021-03-28 08:34:45] Singularity works Installed packages : $ dpkg -l |grep -i cvmfs ii cvmfs 2.8.0~1+ubuntu20.04 amd64 CernVM File System ii cvmfs-config-default 2.0-1 all CernVM File System Default Configuration ii cvmfs-fuse3 2.8.0~1+ubuntu20.04 amd64 CernVM File System fuse3 libraries ii cvmfs-release 3.2-1 all This package contains the apt configuration for the CernVM File System packages. |
9)
Message boards :
Theory Application :
Cannot run native theory after debian upgrade
(Message 44533)
Posted 22 Mar 2021 by zepingouin Post: Sorry for the fire at OVH datacenter, and it occured again ! About the kernel compilation, I followed instructions at the beginning of this thread. I adjusted some release numbers, using current kernel 4.19.0-14 : cp -p /boot/config-4.19.0-14-amd64 .config Then unleashed the CPU's : make -j 14 bindeb-pkg ....... dpkg-deb: building package 'linux-headers-4.19.171' in '../linux-headers-4.19.171_4.19.171-1_amd64.deb'. dpkg-deb: building package 'linux-libc-dev' in '../linux-libc-dev_4.19.171-1_amd64.deb'. dpkg-deb: building package 'linux-image-4.19.171' in '../linux-image-4.19.171_4.19.171-1_amd64.deb'. dpkg-deb: building package 'linux-image-4.19.171-dbg' in '../linux-image-4.19.171-dbg_4.19.171-1_amd64.deb'. dpkg-genbuildinfo --build=binary dpkg-genchanges --build=binary >../linux-4.19.171_4.19.171-1_amd64.changes dpkg-genchanges: info: envoi des binaires seulement (le code source n'est pas inclus) dpkg-source --after-build . dpkg-buildpackage: info: envoi d'un binaire seulement (aucune inclusion de code source) The result : ls -lh linux-*171* -rw-r--r-- 1 f6fgz f6fgz 4,9K mars 22 19:52 linux-4.19.171_4.19.171-1_amd64.buildinfo -rw-r--r-- 1 f6fgz f6fgz 2,3K mars 22 19:52 linux-4.19.171_4.19.171-1_amd64.changes -rw-r--r-- 1 f6fgz f6fgz 11M mars 22 19:46 linux-headers-4.19.171_4.19.171-1_amd64.deb -rw-r--r-- 1 f6fgz f6fgz 46M mars 22 19:47 linux-image-4.19.171_4.19.171-1_amd64.deb -rw-r--r-- 1 f6fgz f6fgz 660M mars 22 19:51 linux-image-4.19.171-dbg_4.19.171-1_amd64.deb -rw-r--r-- 1 f6fgz f6fgz 992K mars 22 19:46 linux-libc-dev_4.19.171-1_amd64.deb Check if you have the following package installed : dpkg -l libssl-dev Souhait=inconnU/Installé/suppRimé/Purgé/H=à garder | État=Non/Installé/fichier-Config/dépaqUeté/échec-conFig/H=semi-installé/W=attend-traitement-déclenchements |/ Err?=(aucune)/besoin Réinstallation (État,Err: majuscule=mauvais) ||/ Nom Version Architecture Description +++-================-===========================================-============-================================================ ii libssl-dev:amd64 1.1.1j-1+0~20210301.25+debian10~1.gbp2578a0 amd64 Secure Sockets Layer toolkit - development files |
10)
Message boards :
Theory Application :
Read-only file system error on native Theory
(Message 44444)
Posted 5 Mar 2021 by zepingouin Post: Have a look at this thread. |
11)
Message boards :
Theory Application :
Cannot run native theory after debian upgrade
(Message 44261)
Posted 4 Feb 2021 by zepingouin Post: Contrats ! You step forward in the Linux learning curve :) |
12)
Message boards :
Theory Application :
Cannot run native theory after debian upgrade
(Message 44259)
Posted 3 Feb 2021 by zepingouin Post: Verify that the /etc/systemd/system/multi-user.target.wants/boinc-client.service file has not been modified by the update process and looks like : [Unit] Be careful !! This service seems to be related to Automount NOT BOINC !! Yes, you will get the one needed as describe in this message which is at the top of this forum, or just copy/paste from the one provided above to replace the one you have. As I think you are not sure about what you are doing, I recommend you to apply what is described above if you don't want mess your systemd configuration. Here is an extract : sudo wget http://lhcathome.cern.ch/lhcathome/download/create-boinc-cgroup -O /sbin/create-boinc-cgroup sudo wget http://lhcathome.cern.ch/lhcathome/download/boinc-client.service -O /etc/systemd/system/boinc-client.service GRUB_CMDLINE_LINUX_DEFAULT="console=ttyS0 vsyscall=emulate quiet" I you want to stay close from the former one, you may omit the quiet parameter. |
13)
Message boards :
Theory Application :
Cannot run native theory after debian upgrade
(Message 44252)
Posted 2 Feb 2021 by zepingouin Post: dpkg-deb: building package 'linux-headers-4.19.160' in '../linux-headers-4.19.160_4.19.160-1_amd64.deb'. ~/KERNEL$ sudo dpkg -i linux-headers-4.19.132_4.19.132-1_amd64.deb As I said at the very beginning of this thread for my notes, MODIFY numbers according to the current release. You are doing copy/paste from the notes, you need to MODIFY the numbers ! sudo dpkg -i linux-headers-4.19.160_4.19.160-1_amd64.deb sudo dpkg -i linux-image-4.19.160_4.19.160-1_amd64.deb # ADAPTER les NUMEROS de VERSION de noyau selon la version en cours Et en plus, c'est écrit en français ;-) |
14)
Message boards :
Theory Application :
Cannot run native theory after debian upgrade
(Message 44246)
Posted 31 Jan 2021 by zepingouin Post: So I installed libelf-dev and now it's running, running, running, running... I'll keep you posted :) With 2 cores, you need to be patient ... after a few hours you should get the kernel packages compiled. make[4]: *** Aucune règle pour fabriquer la cible « debian/certs/debian-uefi-certs.pem », nécessaire pour « certs/x509_certificate_list ». Arrêt. I urge you to read again message 44079. # If you do this, ensure that you modify the configuration to set: # CONFIG_SYSTEM_TRUSTED_KEYS = "" "2 - Add vsyscall emulation to the kernel boot parameters" You have to edit the GRUB default file (use your preferred editor if unfamiliar with vi) $ sudo vi /etc/default/grub You should have a line with GRUB_CMDLINE_LINUX_DEFAULT parameter like : GRUB_CMDLINE_LINUX_DEFAULT="quiet" Just add vsyscall=emulate to it : GRUB_CMDLINE_LINUX_DEFAULT="vsyscall=emulate quiet" Then update GRUB : $ sudo update-grub Once the linux-headers and linux-image packages are ready, just install them as described in message 44079 and reboot. Now you should have these directories : $ find /sys/fs/cgroup/ -type d -name boinc -exec ls -ld {} \;|sort -k9,9 drwxrwxr-x 2 root boinc 0 janv. 31 14:38 /sys/fs/cgroup/blkio/boinc drwxrwxr-x 2 root boinc 0 janv. 31 14:38 /sys/fs/cgroup/cpu,cpuacct/boinc drwxrwxr-x 2 root boinc 0 janv. 31 14:38 /sys/fs/cgroup/cpuset/boinc drwxrwxr-x 2 root boinc 0 janv. 31 14:38 /sys/fs/cgroup/devices/boinc drwxrwxr-x 2 root boinc 0 janv. 31 14:38 /sys/fs/cgroup/freezer/boinc drwxrwxr-x 2 root boinc 0 janv. 31 14:38 /sys/fs/cgroup/hugetlb/boinc drwxrwxr-x 2 root boinc 0 janv. 31 14:38 /sys/fs/cgroup/memory/boinc drwxrwxr-x 2 root boinc 0 janv. 31 14:38 /sys/fs/cgroup/net_cls,net_prio/boinc drwxrwxr-x 2 root boinc 0 janv. 31 14:38 /sys/fs/cgroup/perf_event/boinc drwxrwxr-x 2 root boinc 0 janv. 31 14:38 /sys/fs/cgroup/pids/boinc And the added kernel parameter : $ sudo grep emulate /var/log/syslog Jan 31 11:24:42 pitot kernel: [ 0.000000] Command line: BOOT_IMAGE=/vmlinuz-4.19.160 root=UUID=107d3155-5b10-440d-91f9-a410f99a549d ro vsyscall=emulate quiet Jan 31 11:24:42 pitot kernel: [ 0.531334] Kernel command line: BOOT_IMAGE=/vmlinuz-4.19.160 root=UUID=107d3155-5b10-440d-91f9-a410f99a549d ro vsyscall=emulate quiet You are ready for Native Theory with Debian Buster. |
15)
Message boards :
Theory Application :
Cannot run native theory after debian upgrade
(Message 44243)
Posted 31 Jan 2021 by zepingouin Post: Everything is OK, tasks are successful ! Summary to run Native Theory with Debian Buster : 1 - Add HugeTLB support to the kernel 2 - Add vsyscall emulation to the kernel boot parameters Enjoy ! |
16)
Message boards :
Theory Application :
Cannot run native theory after debian upgrade
(Message 44241)
Posted 31 Jan 2021 by zepingouin Post: Exit Code 139[INFO] Container 'runc' finished with status code 139. You are right, browsing the syslog file I found this error at the end of execution time for each task: Jan 30 21:05:28 pitot kernel: [226518.645124] job[29027] vsyscall attempted with vsyscall=none ip:ffffffffff600400 cs:33 sp:7fffebd9dd48 ax:ffffffffff600400 si:7fffebd9ef82 di:0 Jan 30 21:05:28 pitot kernel: [226518.645128] job[29027]: segfault at ffffffffff600400 ip ffffffffff600400 sp 00007fffebd9dd48 error 15 Jan 30 21:05:28 pitot kernel: [226518.645130] Code: Bad RIP value. By the way, searching on the Net, I found this information about Debian kernel (source) : linux-latest (81) unstable; urgency=medium * From Linux 4.10, the old 'virtual syscall' interface on 64-bit PCs (amd64) is disabled. This breaks chroot environments and containers that use (e)glibc 2.13 and earlier, including those based on Debian 7 or RHEL/CentOS 6. To re-enable it, set the kernel parameter: vsyscall=emulate -- Ben Hutchings <ben@decadent.org.uk> Fri, 30 Jun 2017 23:50:03 +0100 I will try and report, hope to get the solution ... Breaking News It seems to point in the right direction, I have tasks running and runRivet.log indicates processing events ! |
17)
Message boards :
Theory Application :
Cannot run native theory after debian upgrade
(Message 44238)
Posted 30 Jan 2021 by zepingouin Post: dpkg-buildpackage: erreur: fakeroot non trouvé, veuillez soit installer le paquet fakeroot You need to install the fakeroot package : sudo apt install fakeroot sudo make -j 14 bindeb-pkg DON'T make the kernel with admin rights, you only need admin rights to install the DEB packages. Be aware that option -j number depends of how many cores are available on your computer. |
18)
Message boards :
Theory Application :
Cannot run native theory after debian upgrade
(Message 44220)
Posted 29 Jan 2021 by zepingouin Post: But if I understand what you say, despite of all this that you have done, it is still not working, right ? I downloaded some tasks and here is the error message for each one : [INFO] Container 'runc' finished with status code 139. |
19)
Message boards :
Theory Application :
Cannot run native theory after debian upgrade
(Message 44200)
Posted 26 Jan 2021 by zepingouin Post: xzcat : /usr/src/linux-config-4.19/config.amd64_none_amd64.xz: Aucun fichier ou dossier de ce type Oops, sorry ! I forgot to mention this needed package as well : sudo apt install linux-config-4.19 |
20)
Message boards :
Theory Application :
Cannot run native theory after debian upgrade
(Message 44178)
Posted 22 Jan 2021 by zepingouin Post: tar xaf /usr/src/linux-source-4.19.tar.xz -C KERNEL My mistake, I forgot to say that you have to first create the directory. The point is to do the process from a standard user account in a directory that means something. mkdir KERNEL (or whatever you want) But if I understand what you say, despite of all this that you have done, it is still not working, right ? Native Theory tasks start but end with error with Debian Buster. On the other hand, it works well with Ubuntu Focal. Finally, you mention to recompile the module for VB It is optional, just in case you use VB for other projects like Cosmology@Home for instance. |
©2024 CERN