1) Message boards : Number crunching : Not getting any tasks, though many are available (Message 49437)
Posted 7 Feb 2024 by Darrell
Post:
Some more to consider. I was having this problem after I decreased my MAX tasks from 18 to 17 in my perfectly working "app_config.xml" and had BOINC reload my config files. None of the suggestions in this thread had helped, but when I completely REMOVED the app-config and reloaded, this happened:

2/7/2024 10:52:20 AM | LHC@home | update requested by user
2/7/2024 10:52:23 AM | LHC@home | Sending scheduler request: Requested by user.
2/7/2024 10:52:23 AM | LHC@home | Not requesting tasks: don't need (CPU: ; NVIDIA GPU: )
2/7/2024 10:52:26 AM | LHC@home | Scheduler request completed
2/7/2024 10:52:26 AM | LHC@home | Project requested delay of 6 seconds
2/7/2024 10:52:43 AM | | Re-reading cc_config.xml
2/7/2024 10:52:43 AM | | Using proxy info from GUI
2/7/2024 10:52:43 AM | | Config file proxy info overridden by GUI
2/7/2024 10:52:43 AM | | Using HTTP proxy 192.168.1.125:3128
2/7/2024 10:52:43 AM | | Config: GUI RPC allowed from any host
2/7/2024 10:52:43 AM | | Config: use all coprocessors
2/7/2024 10:52:43 AM | | log flags: file_xfer, sched_ops, task, unparsed_xml
2/7/2024 10:52:46 AM | LHC@home | Sending scheduler request: To fetch work.
2/7/2024 10:52:46 AM | LHC@home | Requesting new tasks for CPU
2/7/2024 10:52:47 AM | LHC@home | update requested by user
2/7/2024 10:52:56 AM | LHC@home | Scheduler request completed: got 11 new tasks


Perhaps there is a clue here for whomever is maintaining BOINC these days. And another workaround for others. I now use the "% CPUs" to control the number running.
2) Questions and Answers : Windows : LHC@Home Computation error (Message 49057)
Posted 21 Dec 2023 by Darrell
Post:
To me this appears to be a case of the VM files being moved from "C:" to "E:" outside of the VM utilities. VB does not like that.
3) Message boards : Theory Application : Problem of the day (Message 48952)
Posted 21 Nov 2023 by Darrell
Post:
I tried running the benchmarks, but it made no difference. Besides, another of my computers had not had the benchmarks run and it was processing just fine.

I did try another thing, though. I drained my (one) WU, cancelled a replacement that had started and reset the project. Then I removed app_config, adjusted the preferences to only 50% of the CPUs, and restarted. This worked so far, but I have NO idea why.

I plan to follow up on this approach to see what I can see.
4) Message boards : Theory Application : Problem of the day (Message 48947)
Posted 20 Nov 2023 by Darrell
Post:
ALL of my preferences have NO LIMIT for number of jobs and number of CPUs. My other computers are downloading and running 15-20 WUs each, so I know the preferences work.

I did a "diff" between this and another computer's app_config. They were identical except the other (running) computer did NOT have the entry, below:


<app>
<name>Theory</name>
</app>
<app_version>
<app_name>Theory</app_name>
<plan_class>vbox64_theory</plan_class>
<avg_ncpus>1.0</avg_ncpus>
</app_version>

After removing this, and having BOINC reload configs, I got 4 new tasks. Is this mis-coded in some way? I am curious!
5) Message boards : Theory Application : Problem of the day (Message 48944)
Posted 19 Nov 2023 by Darrell
Post:
Perhaps you looked at one of my other computers.

401889030 217094866 19 Nov 2023, 6:11:52 UTC 30 Nov 2023, 6:11:52 UTC In progress --- --- --- Theory Simulation v300.07 (vbox64_theory) windows_x86_64

is the ONLY WU in process for the computer I can't get/keep more than 1 WU running. Please take another look, and I have had NO ISP problems. Thanks.
6) Message boards : Theory Application : Problem of the day (Message 48942)
Posted 19 Nov 2023 by Darrell
Post:
No. All are set to the same - home, school, work, default. There is something strange going on today, though. Only a few are actually making progress. The others are ... stalled? Of the 17 active on my A3900X1, only 3 are showing more than a fraction of a percent of a CPU time in my monitor. Those three are going 100% of a cpu each. Peculiar, but I have a meeting to run in a few minutes so I can't pursue just now. I haven't looked at the other computers yet.
7) Message boards : Theory Application : Problem of the day (Message 48939)
Posted 19 Nov 2023 by Darrell
Post:
I'm having a problem with WUs not being downloaded due to "11/19/2023 8:40:03 AM | LHC@home | Not requesting tasks: don't need (CPU: ; NVIDIA GPU: )" even though I ask for "<project_max_concurrent>18</project_max_concurrent>" in app_config.xml, and have "use 100%" for both CPU # and CPU time. This is computer ASUS570 ID=10837826, 32GB, AMD 5950X. I get only one at a time when the previous WU ends. I only run Theory VBOX64 on this computer.

My other three computers are working fine and work is flowing. What else do I need to check?
8) Message boards : Theory Application : Theory simulation takes way too long (Message 48893)
Posted 4 Nov 2023 by Darrell
Post:
Ahh, thanks for the better way to close the window. Now all the info to check INSIDE the VB to see if progress is being made is in one place. Thanks for the improvement.
9) Message boards : Theory Application : Theory simulation takes way too long (Message 48890)
Posted 4 Nov 2023 by Darrell
Post:
Just to expand a little for the sake of NOOBs:
My config: BOINC 7.24.1, Virtualbox 7.0.12, Windows 10

1. In BOINC Manager window, select the task you want to view
2. On the left, click on Properties. Near the bottom, note the slot number
listed alongside Directory
3 Navigate to that slot number in "[drive letter]:\ProgramData\BOINC\slots" and
note the folder name starting with "boinc_"
4 In Virtualbox, find that same name.
5 Click "SHOW" on the menu bar
6 Press ALT-F2 to display the running status.
NOTE WELL! If you think is is running too long, the entries in the above window will only
update infrequently, perhaps a minute or two per line, so be patient.
7 When done, close the window and be sure the top option is selected to CONTINUE RUNNING
the Theory application.

Thanks to all the many posters here who helped me get going again.
10) Message boards : Number crunching : Setting up a local squid cache for a home cluster - old comments (Message 39104)
Posted 12 Jun 2019 by Darrell
Post:
@ computezrmle:
PurpleHat wrote:

You would go to boinc manager to each computer at Options->Other Options-> HTTP Proxy and put in ip and port to host that are running squid.
If squid is not running boinc-manager would tell you that need internet access and if you are running more projects then LHC it is possible to exclude this in same section. Simply add projecturl in [Don´t use proxy for]


True.

If using Windows and a Client Configuration file, adding the following may be easier if multiple machines use the same configuration:

<cc_config>
<options>
<proxy_info>
<http_server_name>[your server here]</http_server_name>
<http_server_port>[your server port number here]</http_server_port>
<no_proxy>[comma separated list of URLs to not use the proxy]</no_proxy>
<no_autodetect>0</no_autodetect>
</proxy_info>
</options>
</cc_config>
11) Message boards : Number crunching : Setting up a local squid cache for a home cluster - old comments (Message 39098)
Posted 10 Jun 2019 by Darrell
Post:
@ computezrmle:

I already tried removing "dns_nameservers 1.1.1.1 1.0.0.1 # fast DNS resolvers", and it fails with
"CONNECT lhcathome.cern.ch:443 HTTP/1.1" 503 151 "-" "BOINC client (windows_x86_64 7.14.2)" TAG_NONE:HIER_NONE

Replacing it gives
"CONNECT lhcathome.cern.ch:443 HTTP/1.1" 200 45331 "-" "BOINC client (windows_x86_64 7.14.2)" TCP_TUNNEL:HIER_DIRECT
so it appears it must operate differently in Windows vs. Linux.

I will test your suggestions to see which works and which others don't when I have more time available. As for right now, I am running.

Ahh, so much to learn, so little time to do so.
12) Message boards : Number crunching : Setting up a local squid cache for a home cluster - old comments (Message 39092)
Posted 10 Jun 2019 by Darrell
Post:
@ computezrmle:

I agree that local caching doesn't work with "localhost" or "127.x.y.z" or "::".

It DOES work for me with the NAME of the local computer, e.g., "DDW3770K" in the HTTP Proxy address.
NOTE HOWEVER, it uses the second LAN interface IP address on this computer EVEN THOUGH IT IS DISABLED.

I don't have a computer with only a single interface available at the moment for testing, so YMMV.

Anyone able to confirm using the computer name will work with a single LAN interface on the same computer?

Also, using the name from another computer on the LAN instead of an IP address also works.
13) Message boards : Number crunching : Setting up a local squid cache for a home cluster - old comments (Message 39091)
Posted 10 Jun 2019 by Darrell
Post:
This is what I did to install under Windows:

I started here: http://www.squid-cache.org/ then clicked "download" in the menu

Since I cannot compile Squid, I clicked on "binary package of Squid." part way down

Then clicked link "Squid-3.5" in 1.2.17.1 to get the 3.5 version for Windows

Under the 64bit, I clicked "http://squid.diladele.com/"

Then selected "Squid for Windows" "DOWNLOAD MSI" which starts a 33.6mb download of Squid

I installed it on disk "S" (requires Admin privilege)

Using Notepad, I modified "S:\Squid\etc\squid\squid.conf" as per the below:


# Squid configuration for BOINC under Windows Squid configuration for BOINC under Windows Squid configuration for BOINC under Windows Squid configuration for BOINC under Windows
# Based on squid version 3.5
# See also: http://www.squid-cache.org/

# NOTE: I AM NOT AN EXPERT IN SQUID, and don't know exactly what is and is not required. However, it is sufficient even if not all necessary for my use.

# Where I found info about it: https://lhcathome.cern.ch/lhcathome/forum_thread.php?id=4611&postid=36101#36101
# Many thanks!

# Define your local hosts/networks here
# Examples:
# acl crunchers src 172.16.0.20
# acl crunchers src 172.16.0.25
# acl localnet src 172.16.0.0/12
# acl localnet src fc00::/7
acl crunchers src 192.168.1.120-192.168.1.128 # my local LAN addresses
acl localnet src 192.168.0.0/16 # RFC1918 possible internal network
acl localnet src fc00::/7 # RFC 4193 local private network range

# required for some extras
acl to_httpport port 80
acl to_http8000 port 8000
acl to_squidport port 3128

acl SSL_ports port 443 # ssl
acl Safe_ports port 443 # https
acl Safe_ports port 80 # http
acl Safe_ports port 1025-65535 # unregistered ports

acl CONNECT method CONNECT

follow_x_forwarded_for allow localhost
follow_x_forwarded_for deny all

#
# Start of extra section 1
# Requests that need special handling

# worldcommunitygrid doesn't like data from the local cache
# use the following lines as template if other projects also have problems
acl wcg_nocache dstdomain .worldcommunitygrid.org
always_direct allow wcg_nocache
cache deny wcg_nocache

# if CVMFS uses geoapi, ensure it's checked directly
acl cvmfs_geoapi urlpath_regex -i ^/+cvmfs/+[0-9a-z._~-]+/+api/+[0-9a-z._~-]+/+geo/+[0-9a-z._~-]+/+[0-9a-z.,_~-]+
always_direct allow cvmfs_geoapi
cache deny cvmfs_geoapi

# avoids polluting the disk cache with typical onetimers, e.g. ATLAS job data
acl boinc_nocache urlpath_regex -i /download[0-9a-z._~-]*/+[0-9a-z._~-]+/+.+
cache deny boinc_nocache

# seriously: do NOT cache that!
# Based on a frontier cache suggestion from Fermilab
acl PragmaNoCache req_header Pragma no-cache
cache deny PragmaNoCache

# End of extra section 1
#

#
# Start of extra section 2
# not used in this basic configuration
# include /etc/squid/extensions.d/cern_extensions.conf
# End of extra section 2
#

#
# Start of extra section 3
# not used in this basic configuration
# parent cache configuration
# doesn't improve performance but gains more data for analysis
#include /etc/squid/parents.d/s1x-cvmfs_openhtc_io.conf
#include /etc/squid/parents.d/lhcb-portal-dirac_cern_ch.conf
#include /etc/squid/parents.d/cvmfs-stratum-one_cern_ch.conf
#include /etc/squid/parents.d/cmsfrontier_cern_ch.conf
#include /etc/squid/parents.d/lcgft-atlas_gridpp_rl_ac_uk.conf
#include /etc/squid/parents.d/lhchomeproxy_cern_ch.conf
# End of extra section 3
#

# Deny requests to certain unsafe ports
http_access deny !Safe_ports

# Deny CONNECT to other than secure SSL ports
http_access deny CONNECT !SSL_ports

http_access allow localhost manager
http_access deny manager

# The following should be uncommented to protect innocent
# web applications running on the proxy server who think the only
# one who can access services on "localhost" is a local user
#http_access deny to_localhost

#
# INSERT YOUR OWN RULE(S) HERE TO ALLOW ACCESS FROM YOUR CLIENTS
# see ACL definition above
# Examples:
# http_access allow crunchers
# http_access allow localnet
http_access allow crunchers
http_access allow localnet
http_access allow localhost
http_access deny all

# http_port
# don't bind it to an IP that is accessible from outside unless you know what you do.
# Examples:
# http_port 172.16.0.77:3128
http_port 3128

#
# Start of extra section 4
# not used in this basic configuration
# used for additional ports
# Example:
# http_port 172.16.0.77:3129 intercept
# End of extra section 4
#

# if your machine has more than one IP
# Example:
# tcp_outgoing_address 172.16.0.88


# Required OFF for intercepted traffic from LHCb VMs
client_dst_passthru off


# You don't believe this is enough?
# For sure, it is!
cache_mem 192 MB
maximum_object_size_in_memory 24 KB
memory_replacement_policy heap GDSF

# Keep it large enough to store vdi files in the cache.
# See extra section 1 how to avoid onetimers eating up your storage.
# min-size=xxx keeps very small files away from your disk
cache_replacement_policy heap LFUDA
maximum_object_size 6144 MB
cache_dir aufs /var/cache/squid 32000 16 64 min-size=7937

# logformat has to be changed according to your needs and the capabilities of your logfile analyser
# See: http://www.squid-cache.org/Versions/v3/3.5/cfgman/logformat.html
# if unsure, use the default setting
logformat my_awstats %>A %lp %ui %un [%tl] "%rm %ru HTTP/%rv" %>Hs %st "%{Referer}>h" "%{User-Agent}>h" %Ss:%Sh
access_log stdio:/var/log/squid/access_squid.log logformat=my_awstats
strip_query_terms off
netdb_filename none

coredump_dir none
ftp_user anonymous@

# max_stale 1 week #default
# extended to be prepared for a project reset
max_stale 37 days

# 1 line is required to avoid the ancient default settings
# be conservative
# don't violate the HTTP standards
refresh_pattern . 0 0% 0

# Required for Windows (but perhaps not for Linux?)
dns_nameservers 1.1.1.1 1.0.0.1 # fast DNS resolvers

store_avg_object_size 1 MB

# booster 1!
collapsed_forwarding on

# booster 2!
client_persistent_connections on
server_persistent_connections on

digest_generation off
log_icp_queries off

# use your preferred language here
error_default_language en

dns_defnames on

#depends on your local setup
dns_v4_first on

forwarded_for transparent

##### End of squid.conf End of squid.conf End of squid.conf End of squid.conf End of squid.conf End of squid.conf End of squid.conf End of squid.conf End of squid.conf End of squid.conf End of squid.conf

I started the Squid program (S:\Squid\bin\Diladele.Squid.Tray.exe or link on the desktop).

Then on my OTHER machines under BOINC Manager, options, Other options, HTTP Proxy entered the IPAddress of THIS machine and port 3128.

Local caching doesn't seem to work but from the other machines, it does.
14) Message boards : Number crunching : Setting up a local squid cache for a home cluster - old comments (Message 39090)
Posted 9 Jun 2019 by Darrell
Post:
@ computezrmle:

I am a rank noobie in Squid knowledge, but I got it "running" under Windows. This means I don't get error messages or crashes, BOINC runs and gets data, etc.

However, I also don't have tools or techniques to know if it is effective or not. Where do I get the programs to analyze its logs?

Also, with regard to your post just above, it appears to me that using the "::" IPV6 form for "this computer" is also not effective. Can you confirm?

Thanks.
15) Message boards : Number crunching : Setting up a local squid cache for a home cluster - old comments (Message 39084)
Posted 9 Jun 2019 by Darrell
Post:
@ PurpleHat:

Ahh, I see a bit of light!

Thanks for pointing out that the post IS the config (for Linux-type). I will have to try to translate into Windowese to try it.

And also your pointers.

Thanks again, PurpleHat
16) Message boards : Number crunching : Setting up a local squid cache for a home cluster - old comments (Message 39079)
Posted 8 Jun 2019 by Darrell
Post:
I second what Dingo wrote.

I am running 4x16GB i7 computers and 2x32GB Threadrippers with LHC in about half their capacities, mostly Theory and a little Atlas. I am knowledgeable in computing and Windows, but I am not in Squid.

Perhaps one of the experts here could write a tutorial for us non-experts on how to do it. It would help us AND Cern by reducing traffic.
17) Message boards : ATLAS application : Very long tasks in the queue (Message 29542)
Posted 22 Mar 2017 by Darrell
Post:
@ Yeti:

I think NOT DONE! I am running my first 16 core Atlas with an estimated runtime of 1hr 20min, and it is already at 13hr 47min.
18) Questions and Answers : Windows : Unable to set priority of Virtualbox processing (Message 29234)
Posted 13 Mar 2017 by Darrell
Post:
@ marmot:

Thanks for the reference. It looks to be quite powerful and I will need some time to check it out. Sorry for the slow reply.
19) Questions and Answers : Windows : Unable to set priority of Virtualbox processing (Message 29233)
Posted 13 Mar 2017 by Darrell
Post:
@ Crystal Pellet:

Thank you for the technique - it works great, and I apologize for being so slow returning to thank you.
20) Questions and Answers : Windows : Unable to set priority of Virtualbox processing (Message 28723)
Posted 30 Jan 2017 by Darrell
Post:
When the Virtualbox applications run on all or nearly all cores, they lockout other processing, including the keyboard, and other applications.

Can the applications/Virtualbox processing priority be set lower to allow low CPU usage above them (e.g., windows explorer)?



©2024 CERN