LHC@home

Yes I have 32 cores running these tasks and there is always one of the finished longer tasks stuck on the upload (Transfers) page.
And now the same thing for several of those short running *workplace* tasks.

1/6/2018 2:12:23 AM | | Project communication failed: attempting access to reference site
1/6/2018 2:12:27 AM | | Internet access OK - project servers may be temporarily down.

After 2am so.....goodnight

---

Volunteer Mad Scientist For Life

Have a SixTrack task that thas been trying to download for about 90 minutes now with failures:

06-Jan-2018 20:07:08 [LHC@home] [http] HTTP_OP::init_get(): http://lhcathome-upload.cern.ch/lhcathome/download/2dd/workspace1_hl13_collision_scan_62.3225_60.3275_chrom_15_oct_-300_B1__35__s__62.31_60.32__10_12__5__30_1_sixvf_boinc3051.zip
06-Jan-2018 20:07:08 [LHC@home] [http] HTTP_OP::libcurl_exec(): ca-bundle 'C:\BOINC\ca-bundle.crt'
06-Jan-2018 20:07:08 [LHC@home] [http] HTTP_OP::libcurl_exec(): ca-bundle set
06-Jan-2018 20:07:08 [LHC@home] Started download of workspace1_hl13_collision_scan_62.3225_60.3275_chrom_15_oct_-300_B1__35__s__62.31_60.32__10_12__5__30_1_sixvf_boinc3051.zip
06-Jan-2018 20:07:09 [LHC@home] [http] [ID#10685] Info: Connection 1588 seems to be dead!
06-Jan-2018 20:07:09 [LHC@home] [http] [ID#10685] Info: Closing connection 1588
06-Jan-2018 20:07:09 [LHC@home] [http] [ID#10685] Info: TLSv1.2 (OUT), TLS alert, Client hello (1):
06-Jan-2018 20:07:37 [LHC@home] [http] [ID#10685] Info: Trying 137.138.156.50...
06-Jan-2018 20:07:38 [LHC@home] [http] [ID#10685] Info: Connected to lhcathome-upload.cern.ch (137.138.156.50) port 80 (#1590)
06-Jan-2018 20:07:38 [LHC@home] [http] [ID#10685] Sent header to server: GET /lhcathome/download/2dd/workspace1_hl13_collision_scan_62.3225_60.3275_chrom_15_oct_-300_B1__35__s__62.31_60.32__10_12__5__30_1_sixvf_boinc3051.zip HTTP/1.1

06-Jan-2018 20:07:38 [LHC@home] [http] [ID#10685] Sent header to server: Host: lhcathome-upload.cern.ch

06-Jan-2018 20:07:38 [LHC@home] [http] [ID#10685] Sent header to server: User-Agent: BOINC client (windows_intelx86 7.8.3)

06-Jan-2018 20:07:38 [LHC@home] [http] [ID#10685] Sent header to server: Accept: */*

06-Jan-2018 20:07:38 [LHC@home] [http] [ID#10685] Sent header to server: Accept-Encoding: deflate, gzip

06-Jan-2018 20:07:38 [LHC@home] [http] [ID#10685] Sent header to server: Content-Type: application/x-www-form-urlencoded

06-Jan-2018 20:07:38 [LHC@home] [http] [ID#10685] Sent header to server: Accept-Language: en_US

06-Jan-2018 20:07:38 [LHC@home] [http] [ID#10685] Sent header to server:

06-Jan-2018 20:07:38 [LHC@home] [http] [ID#10685] Sent header to server: án@ºm']÷[-³ZÇQ_«¤2¦a‡¶n†Ò0—ø×iW¡9]jdyÆ
Y¬<1J8|Ã’Ã’K(è _;¢ÌMsÛó®OÇVÕZ§–‰úó«hÔ#†Y'Ï '¼¬nrÆ’0rßà¢éÒátu½*ž{T×C9­U(Åâ»ôÀä®8I3ÃŒv…Ÿ9EÒ¤žòÅ’Qø|ä-õ¬_ëٱ-Ѻ̑BwL%ɐ8oÛðÌûŽ—Y>Õ`Næ(íIyKºHÛ/ùrÓ9ÊþØ4rõ´@Ï1
ÃìÞ-]¸PÑ^§iÞ3(ÊP•ù§TËT†PE©ùI

06-Jan-2018 20:08:38 [LHC@home] [http] [ID#10685] Info: Operation too slow. Less than 10 bytes/sec transferred the last 60 seconds
06-Jan-2018 20:08:38 [LHC@home] [http] [ID#10685] Info: Closing connection 1590
06-Jan-2018 20:08:38 [LHC@home] [http] HTTP error: Timeout was reached
06-Jan-2018 20:08:38 [LHC@home] Temporarily failed download of workspace1_hl13_collision_scan_62.3225_60.3275_chrom_15_oct_-300_B1__35__s__62.31_60.32__10_12__5__30_1_sixvf_boinc3051.zip: transient HTTP error
06-Jan-2018 20:08:38 [LHC@home] Backing off 02:13:10 on download of workspace1_hl13_collision_scan_62.3225_60.3275_chrom_15_oct_-300_B1__35__s__62.31_60.32__10_12__5__30_1_sixvf_boinc3051.zip

---

Can't request work: too many uploads in progress.

And I'm out of tasks.

Well, in fact, almost all of LHC has been pretty much out of order for about 3 weeks.

It all started out when they swamped the server with an extremely high number of ATLAS tasks - and everything got stuck.
Then they swamped the server with over one million of Sixtrack tasks - again, everything got stuck.
CMS has no tasks since 2 days ago, only Theory and LHCb tasks are available, from which so and so many fail due to connection problems with the "Condor" server :-(

Let's hope that from tomorrow on, when the CERN people are back from vacation, things will improve.

Just got it:
"07.01.2018 14:52:49 | LHC@home | [error] Error reported by file upload server: Server is out of disk space"

Just got it:
"07.01.2018 14:52:49 | LHC@home | [error] Error reported by file upload server: Server is out of disk space"

Well, the indication "Server out of disk space" may or may not be true.
I remember when all these problems at LHC started mit-December, we've seen all kinds of error messages, which were partly wrong. It seems that BOINC at times is messed up by what's going on at LHC :-)

My stuck SixTrack task download finally finished downloading about 10 minutes ago after another forced retry.

---

My stuck SixTrack task download finally finished downloading about 10 minutes ago after another forced retry.

here, an ATLAS task which got finished 2 days ago is still waiting for upload, despite of numerous forced retries :-(

I was finally able to get new work by spamming the upload button until enough -- still not all -- tasks had uploaded.

I should add that a lot of tasks were uploaded over night. Also I now find that downloads don't finish either.

Also I now find that downloads don't finish either.

Yesterday, I got 2 ATLAS tasks which took some 15 minutes to download; however, the download was incomplete, and so the tasks failed a few minutes after they started.
Once more I am saying that we can only hope that if the CERN people are back from vacation now, they'll try to straighten out all that mess.

Thanks to the IT infrastructure guys, the situation is getting better - please see:
https://lhcathome.cern.ch/lhcathome/forum_thread.php?id=4567#33713

... the situation is getting better - please see:
https://lhcathome.cern.ch/lhcathome/forum_thread.php?id=4567#33713

well, they are saying that they will try to fix some issues. Which may, of course, take time. So far I did not notice any improvement yet.

Indeed my task that is already 15 days trying to upload is still not working and I did many manual tries too.
Another one is for about a week trying to upload, so we still have to wait and see.

---

Greetings from,
TJ

I've uploaded all computed tasks but I can not download any

2018-01-08 15:39:59 | LHC@home | Sending scheduler request: To fetch work.
2018-01-08 15:39:59 | LHC@home | Requesting new tasks for CPU
2018-01-08 15:40:01 | LHC@home | Scheduler request completed: got 0 new tasks
2018-01-08 15:40:01 | LHC@home | No tasks sent
2018-01-08 15:40:01 | LHC@home | No tasks are available for SixTrack
2018-01-08 15:40:01 | LHC@home | Wiadomość z serwera: VirtualBox is not installed

I am trying to upload an Atlas task already past its deadline on a Linux laptop and 8 SixTrack tasks on my main Linux box. Strangely enough, all SixTrack tasks on the laptop uploaded and were reported. Atlas tasks and SixTrack tasks are the only one not failing on my 3 PCs. The third and newest is a Windows 10 PC which has survived, on its A10-6700 CPU and 22 GB RAM the Windows Meltdown/Spectre patch.
Tullio

please see https://lhcathome.cern.ch/lhcathome/forum_thread.php?id=4567&postid=33736

Impossible to send back sixtrack tasks:
19/01/2018 06:18:53 | LHC@home | Started upload of w-c2_job.B1inj_c2.2158__23__s__64.28_59.31__11.1_12.1__6__24_1_sixvf_boinc13712_0_r471927011_0
19/01/2018 06:18:53 | LHC@home | Started upload of w-c2_job.B1inj_c2.2158__28__s__64.28_59.31__7.1_8.1__6__57_1_sixvf_boinc18808_1_r1137666119_0
19/01/2018 06:19:03 | LHC@home | [error] Error reported by file upload server: [w-c2_job.B1inj_c2.2158__23__s__64.28_59.31__11.1_12.1__6__24_1_sixvf_boinc13712_0_r471927011_0] locked by file_upload_handler PID=-1
19/01/2018 06:19:03 | LHC@home | [error] Error reported by file upload server: [w-c2_job.B1inj_c2.2158__28__s__64.28_59.31__7.1_8.1__6__57_1_sixvf_boinc18808_1_r1137666119_0] locked by file_upload_handler PID=-1
19/01/2018 06:19:03 | LHC@home | Temporarily failed upload of w-c2_job.B1inj_c2.2158__23__s__64.28_59.31__11.1_12.1__6__24_1_sixvf_boinc13712_0_r471927011_0: transient upload error
19/01/2018 06:19:03 | LHC@home | Backing off 04:32:29 on upload of w-c2_job.B1inj_c2.2158__23__s__64.28_59.31__11.1_12.1__6__24_1_sixvf_boinc13712_0_r471927011_0
19/01/2018 06:19:03 | LHC@home | Temporarily failed upload of w-c2_job.B1inj_c2.2158__28__s__64.28_59.31__7.1_8.1__6__57_1_sixvf_boinc18808_1_r1137666119_0: transient upload error
19/01/2018 06:19:03 | LHC@home | Backing off 04:43:40 on upload of w-c2_job.B1inj_c2.2158__28__s__64.28_59.31__7.1_8.1__6__57_1_sixvf_boinc18808_1_r1137666119_0

Is it a task that your BOINC client has been trying to upload in the past days (and most probably got interrupted the first time) or a recent one?
My pc successfully updated a recent result this morning:

Fri 19 Jan 2018 06:29:21 AM CET | LHC@home | Started upload of LHC_2015_LHC_2015_234_BOINC_errors__33__s__62.31_60.32__7.2_7.3__5__64.5_1_sixvf_boinc116391_0_r100608959_0
Fri 19 Jan 2018 06:29:28 AM CET | LHC@home | Finished upload of LHC_2015_LHC_2015_234_BOINC_errors__33__s__62.31_60.32__7.2_7.3__5__64.5_1_sixvf_boinc116391_0_r100608959_0
Fri 19 Jan 2018 06:29:28 AM CET | LHC@home | Sending scheduler request: To report completed tasks.

... locked by file_upload_handler PID=-1

This looks like the (meanwhile) well known file lock issue.
David Cameron wrote about it in the ATLAS MB.

@PFLIEGER Guy
There's nothing you can do to solve the problem.

@Alessio Mereghetti
If not already done the server admins at CERN may check the service that periodically unlocks those files and perhaps run it more often.

... and perhaps run it more often.

this may seen to be necessary