LHC@home

Very nice; thanks.
But I think it should be pointed out that the automatic configuration download no longer applies, insofar as I can see.
(sudo wget https://lhcathome.cern.ch/lhcathome/download/default.local -O /etc/cvmfs/default.local)

Maybe it could be updated?

Hello,
Back crunching here a little after quite a long while, starting things off slow.
Currently have an duel Opteron 6128 and i7-3610qm.
They seem to be receiving a lot of Atlas tasks, which are multithreaded. I do not recall if there are others that are multi threaded as well - perhaps CMS?
In people's experience, what is more efficient? Less cores thus long runtimes or more cores and shorter runtimes, per task? I see these Atlas tasks run about 2.5 gb of ram per WU. is it just a matter of not oversaturating the ram, as it were?
Right now I am running 4 threads per task, as per project settings and max number of CPUs.

Welcome back.

Atlas is currently the only multi threaded application here. The most efficient usage of CPUs would be running with a single thread as the multi thread tasks have at the beginning and at the end of a task a section that uses only a single thread anyway. So other CPU threads reserved for that task are idling during that time. Also multiple threads are doing their job calculations independently from an other so they will not finish at the same time when all jobs have been done.

But CPU threads are not the only thing to worry about, there is also the memory consumption. The amount of memory an Atlas task uses is calculated like this: 3000 MB + n * 900 MB where n is number of threads. So a single CPU Atlas task uses 3900 MB and two CPU task uses 4800 MB. So if you have the cores available but are limited by the memory more threads are the way to go.

If you are keen on the credits you get then single CPU tasks earn more because credit is calculated from the run time and not the CPU time.

Good luck with your experiments.

The fallback proxies are configured by a script that is located on CERN's online repository cvmfs-config.cern.ch.
This is done because your defaul.local defines a squid that can't be accessed or does not even exist.
It looks like you simply copied the default.local file from the forum thread and did not read the comments inside.

There are 2 possible solutions:
1. To setup a local proxy and replace "squid" with it's hostname
2. To remove the "#" in front of CVMFS_HTTP_PROXY="auto;DIRECT"

(1.) would be the preferred method for clusters/single computers providing more than 5 worker nodes
(2.) is the simple solution
The limit of 5 should be seen as a magnitude rather than a sharp limit.


Don't forget an "[sudo] cvmfs_config reload" after you saved the changes.

<edit>
Sorry, checked the wrong logfile.
Your recent ones show that you are already running a proxy called "squid".
That's fine. Leave it this way.

[2021-02-25 18:47:47] 2.8.0.0 2129 1201 59336 79744 0 78 1907592 4096000 1399 130560 0 386235 98.3608 875279 454 http://s1fnal-cvmfs.openhtc.io/cvmfs/atlas.cern.ch http://squid:3128 0

</edit>

Any help appreciated.

Good to see that you got it running.

Nonetheless you may check your CVMFS setup.
It connects via a fallback proxy at Fermilab (131.225.188.245).
To get out why be so kind as to post the output of the following command:

cvmfs_config showconfig atlas.cern.ch |grep -E 'FALLBACK_PROXY|HTTP_PROXY|USE_CDN'

'FALLBACK_PROXY|HTTP_PROXY|USE_CDN'
CVMFS_EXTERNAL_FALLBACK_PROXY=
CVMFS_EXTERNAL_HTTP_PROXY=
CVMFS_FALLBACK_PROXY='http://cvmfsbproxy.cern.ch:3126;http://cvmfsbproxy.fnal.gov:3126'    # from /cvmfs/cvmfs-config.cern.ch/etc/cvmfs/domain.d/cern.ch.conf
CVMFS_HTTP_PROXY='http://squid:3128;DIRECT'    # from /cvmfs/cvmfs-config.cern.ch/etc/cvmfs/domain.d/cern.ch.conf
CVMFS_USE_CDN=yes    # from /etc/cvmfs/default.local

I'm not understanding your last point, but then I'm not a solid Linux user and a lot of things are still over my head - I'm assuming what you mean is the project was calling too much of the CPU into action and it couldn't keep up.
Is there a big difference in which tasks you run vs. credits? For instance does Atlas pay more than sixtrack? How are your Ryzen's configured? My machines are 16-32 gb ram with my Ryzen 7 having 64 gb total, so it should be able to handle 7 2-core Atlas's just fine with my other i7's having 1 or 2 core Atlas's running depending on 16 or 32 gb of ram and 75% CPU usage.

It's a quiet Christmas eve here, so I'm currently throwing together an old Xeon w3520 I found in the parts drawer. I think I have an i7 from the same time period here somewhere...will be two good space heaters in my office in the cold Canadian winter, I just need to find a cooler for the latter and see how my electric bill fairs. No cases on either of these - not enough room, ironically. They will of course both run Linux - my main desktop and laptop will continue running Windows, at least for now, while I slowly but surely navigate around the Linux command line and learn how to break things, then fix them. Old as these are every little bit will help is my motto.

Here is a short history of load averages in Linux. It has two things going for it: First, it is very well written, has a simple to understand summary on the first page, and provides concrete examples with code snippets later on. Second, it was the very first article that came up when I did a Google search for "Unix load average explained".

http://www.brendangregg.com/blog/2017-08-08/linux-load-averages.html

...I do have other machines that are in datacenters and internet isn't an issue. One day that 300/25 will come my way and I can be happy. And maybe a few more Ryzens too.
I'm aiming for theoretically using up around 75% of each machine if at all possible. Should I be aiming at more or less? Basically 2 threads not being used on an 8 threads machine and similar on 16 - though might bump that up to 3 not in use.

I started LHC@home with one Ryzen 3700X (my desktop) running at 25-75% load for 10 days straight and then added a second (my server) running at 75%-100% load. 15 days after that (25 days total) I had one million points for this project. So yes, running at less than 100% is very do-able especially if you have lots of threads, RAM, and need your systems for other tasks.

EDIT: I forgot to add that when running the server at 75% the "load average" (how many treads were asking my kernel for a CPU at once) was a little less than 13 on average. When I turned LHC up to 100% the load average would quickly spike into the low 20's and stay there. That's not good on a system that can only handle 16 threads.

I see you are on Ubuntu 18.04.3. I have not had much luck getting native ATLAS to work on recent installs (native Theory is OK).
Let me know what you come up with.

I can only speak for Linux:

Native ATLAS takes 2 GB, and CMS (VBox) takes 3 GB. At least that is what they show in BoincTasks, but actual usage is less. I expect you will need to plan for the worst-case in order to run the number you want.
Theory (native) is very small - averaging around 20 MB, though occasionally more, and SixTrack only about 100 MB in Linux.

The RAM usage is well known but I think the bandwidth is not. You can get a total daily transfer history from BOINC on the command line:

boinccmd --get_daily_xfer_history

But it will not be broken down by project and/or sub-project. It will just show the daily transfer totals for that host. If you have a proxy server running for your BOINC clients you might be able to save a lot of network traffic with it on the native workloads.

I guess this means 75 Mbit/s download and 3-4 Mbit/s upload?
75 down are far away from being a problem in this case.

3-4 up might be a problem.

I don't think anyone answered one of your questions:

... on the single little core 2 duo I have running on here, if there was an Asteroids at home task running, and boinc decided a native theory Atlas task should start, with max number of jobs CPUs set to 2, there would be 2 theory Atlas tasks using 0.5 CPUs (sharing one core) and asteroids would use the second if max CPUs was also set to 2. Am I understanding this correctly …?

No - the number of CPUs to be used by an (Atlas, either native or VBox) task is specified by that setting and then passed as a requirement down the chain. So if you set it to 2, the client will have to set aside two cores for it, and will therefore either wait until your Asteroids task has finished leaving the other core idle, or else find another single-core job (e.g. Sixtrack) to do in the meantime which won't finish at the same time and the cycle restarts...

So it's best not to set #CPUs to completely fill the machine, unless you dedicate it to only running Atlas. (In which case you also have to remember to untick the "If no work for selected applications is available, accept work from other applications?" box!)

you are already running Theory OK, you have CVMFS installed properly, so try native ATLAS.
And since you have 32 GB of memory, you can set Max # CPUs 1 and use some of it, as it is a little more efficient than using more cores per work unity (which is used if you want to save on memory).

BUT: I have found that to run native ATLAS, you have to grant additional permissions after attaching to LHC and downloading at least one native ATLAS.
Then: Your run "sudo chmod -R 777 /var/lib/boinc-client" and reboot. Then it might work.

And don't even think of upgrading to BOINC 7.16.3, or native ATLAS falls apart.

Everything else is pretty easy beyond that. To run CMS all you need is VirtualBox.
And for SixTrack you don't even need that.