21)
Message boards :
ATLAS application :
Error with 2 CPUs
(Message 46364)
Posted 25 Feb 2022 by AndreyOR Post: Change the following line to a high number in global_prefs_override.xml to prevent BOINC from switching between tasks. If the line is not there just add it.: <cpu_scheduling_period_minutes>10080.000000</cpu_scheduling_period_minutes> I have mine set to 10080 minutes (1 week). I have the same setting on my Windows and Linux BOINC setups. I don't see a good reason to switch between tasks mid-task, just let a task finish before moving to the next one. For a project like ATLAS and maybe Theory this setting is necessary. |
22)
Message boards :
ATLAS application :
Error with 2 CPUs
(Message 46359)
Posted 25 Feb 2022 by AndreyOR Post: The reason is because you're running it on WSL2. WSL2 is not exactly the same as regular Linux (Ubuntu in your case) because it has a custom kernel and it's init.d, not systemd. It's good to see others use WSL2 for BOINC projects but it does have its quirks in LHC. One of them is that native ATLAS can only be ran single core in WSL2 and fails when you try to run it multi-core. I've tried to figure out a solution to that a few times in the past but no success so far. Another quirk is that native Theory doesn't run on WSL2 without a modification (it's an easy one though), which took me a while to figure out. If you're thinking of running Theory check out this post: https://lhcathome.cern.ch/lhcathome/forum_thread.php?id=5777&postid=46031, it should be at the bottom of the thread. I like WSL2 because you can run Linux projects on Windows machines with minimal resources compared to regular virtual machines. I used to use Hyper-V before learning about WSL2 and now very rarely use Hyper-V. |
23)
Message boards :
ATLAS application :
Guide to Getting Quickly Started Running Native ATLAS (Ubuntu 20.04 on WSL2)
(Message 46101)
Posted 23 Jan 2022 by AndreyOR Post: I don't think nested virtualization is supported for AMD processors until Windows 11 (as a host OS), and I believe running WSL2 on Hyper-V Windows VM is considered nested virtualization. |
24)
Message boards :
ATLAS application :
Guide to Getting Quickly Started Running Native ATLAS (Ubuntu 20.04 on WSL2)
(Message 46095)
Posted 20 Jan 2022 by AndreyOR Post: Brummig, I had the chance to try installing Singularity like you described, from a package, instead of from source like the Singularity documentation describe, and it seems to be working. So far I have 2 completed single core ATLAS tasks with "HITS file was successfully produced" https://lhcathome.cern.ch/lhcathome/result.php?resultid=340312764. It's good to see that there's something that can be simplified for a project that's anything but simple. I mean, besides BOINC, for (native) Theory and ATLAS you need a specific OS (Linux), a software distribution service (CVMFS), a container (Singularity for ATLAS, runc for Theory), and if you regularly contribute 5 or more CPU threads - a caching proxy (Squid). Each must be properly configured but even then things don't always work right. The projects tried to simplify things by bundling up the containers but that doesn't always work and one still has to install them separately (mostly a problem in ATLAS). WSL2 presents additional challenges as it has a custom kernel and it's init.d, not systemd. Single thread ATLAS usually runs fine but I only recently was able to figure out how to get Theory to run and still don't know how to get multi-thread ATLAS to run. Something isn't working for you though. All of your ATLAS tasks are either errors or produce no HITS file. I don't know why sometimes tasks that should be errors show ups as Completed and Validated, that's definitely misleading and makes things even more complicated. May I suggest you switch to Theory for now until you can troubleshoot and figure out ATLAS. Otherwise your CPU time is of no benefit to you or the project. Theory won't work on WSL2 as is, you need to make one modification to WSL2 configuration (.wslconfig file). I assume you're familiar with it, if not, read here: https://docs.microsoft.com/en-us/windows/wsl/wsl-config. Add the following line at the end of .wslconfig kernelCommandLine = vsyscall=emulate Exit and shut down (wsl --shutdown) Ubuntu. Wait until it completely shuts down, Microsoft recommends 8 seconds but I've seen it take longer. Restart Ubuntu and run the following command to make sure the configuration took: cat /proc/self/maps | egrep 'vdso|vsyscall' If the output looks something like the following two lines - the configuration took. If the output is only like the first line - shutdown again and wait a little longer before restarting. 7fffe03fe000-7fffe0400000 r-xp 00000000 00:00 0 [vdso] ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] Make sure CVMFS is running, change your settings on the website to accept Theory and you should be good to go. I wonder why single thread ATLAS isn't working for you. I installed SingularityCE 3.9.3, maybe try uninstalling your older one and install this one. If it doesn't work, perhaps uninstall Singularity altogether and try running with the Singularity that comes bundled. Maybe one of the older versions will work; I'm considering trying various older versions in attempts to get multi-thread ATLAS to work. |
25)
Message boards :
Number crunching :
Setting up a local Squid to work with LHC@home - Comments and Questions
(Message 46085)
Posted 17 Jan 2022 by AndreyOR Post: I'm trying to install a newer version (4.17) of Squid from source but ./configure has many different options and I'm not sure which to use (except 2). Which should be used for a 1 to 2 PC home network to run LHC? I've previously, on a different system, successfully installed an older version that's available prepackaged and used the configuration file with appropriate modifications found on the forum. |
26)
Message boards :
ATLAS application :
Guide to Getting Quickly Started Running Native ATLAS (Ubuntu 20.04 on WSL2)
(Message 46072)
Posted 16 Jan 2022 by AndreyOR Post: I played around with WSL2 BOINC configurations, to be able to control it from Windows BOINC Manager and it doesn't seem to work cleanly. Configurations seem to be crisis-crossing somehow, WSL2 BOINC data directory seems to be getting changed or duplicated (still not sure which happens). Proxy settings on one affect the other. I'm still not all clear on what's going on but it doesn't seem to function like I expected, where you can manage both independently without one affecting the other. Unlike you, I don't have anything else connected, so I can see how your set up can get even more complicated. Do you have your BOINC clients set up to listen to different ports? That's what I had to do to prevent a problem in the last post. I changed the WSL2 one as it was easier but that made managing it through command line more difficult. |
27)
Message boards :
ATLAS application :
Guide to Getting Quickly Started Running Native ATLAS (Ubuntu 20.04 on WSL2)
(Message 46058)
Posted 14 Jan 2022 by AndreyOR Post: There's some kind of cross-communication happening between WSL2 & Windows BOINCs. I noticed that if BOINC is running on WSL2 first and then I try to start BOINC in Windows I get a Connection Error: Invalid client RPC password. BOINC has to be started on Windows first and then on WSL2, no problems if done in this order. I haven't tried to figure out what's going on yet. |
28)
Message boards :
ATLAS application :
Guide to Getting Quickly Started Running Native ATLAS (Ubuntu 20.04 on WSL2)
(Message 46045)
Posted 12 Jan 2022 by AndreyOR Post: maeax, yes, they're incompatible. WSL2 runs off of Hyper-V architecture. Hyper-V is a Type 1 hypervisor that comes as part of the Windows package. VirtualBox is a type 2 hypervisor and it doesn't work with Hyper-V enabled. There's work being done on that and I believe the newest version of VB might be able to work with Hyper-V although I don't know how well. To get back to being able to use VB I think you'll have to disable all 3 of the following in Windows features: Hyper-V, Virtual Machine Platform, and Windows Subsystem for Windows. Restart and try VB again. I don't believe there's a default distribution but it has, I believe, 11 distributions available, if you include the various versions. |
29)
Message boards :
ATLAS application :
Guide to Getting Quickly Started Running Native ATLAS (Ubuntu 20.04 on WSL2)
(Message 46041)
Posted 11 Jan 2022 by AndreyOR Post: Brummig, it's good to see someone else using WSL2 for BOINC and trying it for LHC. I've used it to run multiple projects. LHC is even harder to set up on WSL2 than on regular Linux, probably due to its custom kernel. I only recently figured out how to get native Theory to run on WSL2, still can't run multithread ATLAS (runs ok single thread only). The previous responses are correct and you have a point too. I should get a chance to post what I did to get things to work within a few days. On a quick note... -- maeax and computezrmle are right your ATLAS tasks aren't working correctly. To me it looks like Singularity isn't working properly. Based on your instructions, I'm guessing it wasn't installed correctly. I'd suggest you carefully go through the SingularityCE "Quick Start" up to (but not including) the "Overview of the SingularityCE Interface" section. https://sylabs.io/guides/3.9/user-guide/quick_start.html# -- As mentioned above, the following entries are important, I'd definitely modify the PROXY one and add the CDN one. CVMFS_HTTP_PROXY="auto;DIRECT" CVMFS_USE_CDN=yes |
30)
Message boards :
Theory Application :
Unable to run native Theory in WSL2 Ubuntu 20.04
(Message 46031)
Posted 10 Jan 2022 by AndreyOR Post: With computezrmle's troubleshooting help I was able to find a solution. Basically one needs to add a line to WSL2 configuration file (.wslconfig) to enable vsyscall emulation which solves the memory access violation (exit code 139) problem. See https://docs.microsoft.com/en-us/windows/wsl/wsl-config for details on WSL2 configurations. A simple .wslconfig file that works for native Theory might look like this: [wsl2] memory=16GB processors=8 kernelCommandLine = vsyscall=emulate |
31)
Message boards :
Theory Application :
Unable to run native Theory in WSL2 Ubuntu 20.04
(Message 46001)
Posted 5 Jan 2022 by AndreyOR Post: Yes, I read that WSL2 has a custom kernel so even if Ubuntu versions are identical the kernels are not. I ran the command you suggested and yes, WSL2 is init.d (Hyper-V setup is systemd). It seems you're right about the user namespaces thing, that it's not used anymore. I deleted the entry on my Hyper-V setup, restarted it and tried a couple of Theory tasks which finished with no problems. This at least eliminates that as a possible reason for why Theory won't work on WSL2. It seems like the instructions on that sticky post need an update. With the MillkyWay, I wanted to at least see if WSL2 Ubuntu could run multicore projects of any kind, and it can, the tasks finished without problems. I wonder if there's anything else to try. Maybe installing runc as opposed to using the one provided? I remember reading your suggestion when I first was trying to get everything set up that installing singularity instead of using the one provided is more reliable. That suggestion worked for me both in Hyper-V and WSL2. |
32)
Message boards :
Theory Application :
Unable to run native Theory in WSL2 Ubuntu 20.04
(Message 45994)
Posted 4 Jan 2022 by AndreyOR Post: So I'm a bit confused then. I get exactly the same outputs for those commands in Hyper-V Ubuntu 20.04.3 (same version as WSL2) set up on the same PC but have no problem running native Theory and multicore ATLAS. Here's an example Theory task: https://lhcathome.cern.ch/lhcathome/result.php?resultid=337970091. I'm going to try running MilkyWay multicore app on WSL2 and see if it works, that's the only other multicore app that I'm aware of. What do you think about the things I mentioned in the last 3 paragraphs: being unable to change user namespaces and that WSL2 Ubuntu is initi.d and not systemd like regular Ubuntu installations are? |
33)
Message boards :
Theory Application :
Unable to run native Theory in WSL2 Ubuntu 20.04
(Message 45989)
Posted 4 Jan 2022 by AndreyOR Post: Output for: grep cgroup /proc/filesystems nodev cgroup nodev cgroup2 There's no output for: command -v runc Output for: $(command -v runc) --version --version: command not found I don't know that much about Linux so don't know what all that means exactly since I get same outputs on Hyper-V Ubuntu and have no problem running Theory and multicore ATLAS there. I haven't ran ATLAS in a while so everything cleared out but I just ran a 2-core task and even though it says Completed and Validated, the log says No HITS result produced. https://lhcathome.cern.ch/lhcathome/result.php?resultid=338077983 I started a single core task but it'll take a few hours to finish. https://lhcathome.cern.ch/lhcathome/result.php?resultid=338078147 In regard to what maeax said... When I try to enable user namespaces from those instructions I get the following. Commands: sudo sed -i '$ a\kernel.unprivileged_userns_clone = 1' /etc/sysctl.conf sudo sysctl -p Output: sysctl: cannot stat /proc/sys/kernel/unprivileged_userns_clone: No such file or directory One thing I learned from some research is that WSL2 Ubuntu (and maybe other distributions) are init.d and not systemd. Could that be part of the problem? If so, is there still a way to get Theory and multicore ATLAS to work? I like WSL2 for Linux projects because of its low use of resources as opposed to using Hyper-V which I've done before for Theory and multicore ATLAS. |
34)
Message boards :
Theory Application :
Unable to run native Theory in WSL2 Ubuntu 20.04
(Message 45979)
Posted 3 Jan 2022 by AndreyOR Post: I was hoping to use WSL2 Ubuntu 20.04 setup to run native Theory tasks but they error out. Here's an example task with log: https://lhcathome.cern.ch/lhcathome/result.php?resultid=338017823 Plenty of resources on the system and cvmfs_config probe checks out ok. Any ideas on why this may be happening? By the way, I'm able to run a number of other linux projects including single core Atlas tasks (but not multicore) on this setup. |
35)
Message boards :
Number crunching :
Setting up a local Squid to work with LHC@home - Comments and Questions
(Message 45559)
Posted 27 Oct 2021 by AndreyOR Post: I actually don't have that many processors. Most of those are from a single PC with multiple installations (Windows, Hyper-V Ubuntu, WSL2s). That PC is Ryzen 5900X with 32 GB RAM which is what I use for Atlas & Theory. Due to RAM constraints I should be able to run anywhere form 8 1-core to 2 12-core native ATLAS tasks simultaneously or up to 24 native Theory tasks simultaneously. My understanding is that Hyper-V is a type 1 hypervisor so would be considered bare metal. That would make both Windows and Ubuntu installations VMs, even though that sounds strange/wrong. In that case it seems like installing Squid on Hyper-V Ubuntu would be the way to go. Is my thinking correct here? |
36)
Message boards :
Number crunching :
Setting up a local Squid to work with LHC@home - Comments and Questions
(Message 45531)
Posted 25 Oct 2021 by AndreyOR Post: Would like a clarification on whether to install local Squid. I've read that it's recommended for 5+ worker nodes but not recommended to install on VMs. If I'm running native Theory & Atlas tasks with 5+ worker nodes on Hyper-V Ubuntu, is it recommended to install local Squid? Thanks. |
37)
Message boards :
ATLAS application :
Unable to run multicore ATLAS on WSL2 but single core runs OK
(Message 45371)
Posted 21 Sep 2021 by AndreyOR Post: Hi all, I'm able to run native ATLAS on WSL2 Ubuntu 20.04 as single core only. When I change preferences to use 2 cores the tasks fail as "Validate Error" after about 15 min. of runtime. I was under the impression that once you get ATLAS running and with enough RAM you should be able to run any core configuration. I was able to run native multicore ATLAS on Hyper-V prior to learning of and switching to WSL2. Haven't been able to get Theory to run on WSL2 either but that's a separate question for a different thread. Is this a limitation of WSL2? Any ideas to try? Thank you. Here's an example of a failed task: https://lhcathome.cern.ch/lhcathome/result.php?resultid=325839400 |
©2024 CERN