LHC@home

Since last night, uploads of finished tasks get stuck in "backoff" Status.
Any problem with the ATLAS upload server?

got the same problem, upload goes to 100% but will not finish.
At morning i thought first it will be a problem of the Windows 10 update last night.

upload goes to 100% but will not finish.

here, the progress bar in the BOINC Manager shows 0,00% upload :-(

tested with Primegrid, no upload problem.

with my Telefon DSL line, 600 kbit upload speed, the upload won't start.
with the Kabel-DSL line, 10Mbit upload Speed, upload starts and goes to 100% but will not finish.
When Boinc restarts the upload, the upload will be reset and start at 0%
Upload Speed seems to be OK, limited to 200 KB for each Boinc session so 2 upload Task per session will run run with 100KB each.

I have one that just got stuck too. After two automatic upload attempts, I tried a manual retry. It started out at 350 Kbps and slowed down to 200 Kpbs at the end, and finally stuck at 100%. So it is not a network problem.
EDIT: Finally, after sitting a 100% for a minute or two, it completed and has been reported. It is now waiting for validation.

When Boinc restarts the upload, the upload will be reset and start at 0%

same here: I just noticed the following: an upload which was in "retry" status started again, very slowly, with many interrruptions inbetween, then it stopped at 99,73% with the status reverting to "retry", and after it restarted from there, the progress status fell back to 0.00 %. And here it stopped again.

would be interesting to know how many users are affected by this problem.
Only very few ones? Or more, and many of them have not noticed it?

Same issue here. But, like the download issue the upload issues are erratic. 1 WU finished and checked out, others didn't.

My BOINC log says (paraphrasing) "Project Communication failed ... Internet Access OK. Must be server side. Retrying later".

EDIT: Looking at the status page you can now see a lot of WUs are queuing. A lot of volunteers should be experiencing the same issue.

Temporarily failed upload of ...ATLAS_result: transient HTTP error

I now checked the BOINC event log. The first such entry here was last night at 00:22:59 hrs (UTC+1).

I just could observe another try of a task getting sent: the progress bar proceeds to 100%, then the upload stops, and a few minutes later it's back at 0.00%. Always the same game.
I'm wondering whether no one at CERN has found out about this problem yet. Are there no bells around that would start ringing if something like this happens?

The bad thing is that as long as uploads don't get finished, no new tasked are being downloaded :-(

One of our file servers crashed during the night, and the fail-over server took over. Should be back to normal soon.

There is an increase of load with the current ATLAS simulation campaign, so we will add another file server to increase capacity.

Nils, many thanks for the Information :-)

Thanks for the update. I have one stuck as well.

Theory and LHCb are uploading on the same PC for me.

.....There is an increase of load with the current ATLAS simulation campaign, so we will add another file server to increase capacity.

Thanks for Information.

i'll hope the new file Server will work soon, at the moment all uploads will stuck at 100% and not finish.
Intersting that upload will work but not the finishing. Seems to be the Database entry is blocked.

My WU-Cache will be empty in 10 hours and it's snowing......

Nils Høimyr wrote at 10:36 hrs:

One of our file servers crashed during the night, and the fail-over server took over. Should be back to normal soon.
There is an increase of load with the current ATLAS simulation campaign, so we will add another file server to increase capacity.

Nils, any idea when you will be back to normal?
At this point, uploads are still not working.

So far, none of my finished ATLAS tasks could be uploaded succesfully.
How is the situation with the other crunchers who had reported here the same problem during the course of the day?
Are your finished tasks also still waiting, or have they been uploaded meanwhile?

First task was uploading successful at 17 UTC.
https://lhcathome.cern.ch/lhcathome/workunit.php?wuid=81710691

Thank you Nils and your Team.

Still have four tasks, which are not uploaded. Stops at 100% or less or does not start. No upload at the moment.

Still have four tasks, which are not uploaded. Stops at 100% or less or does not start. No upload at the moment.

same Situation here!
Nils, is someone out there taking care of this problem?

It's still hit and miss for me too. The assurance that the issue is being worked on is all I wanted really. It could have come a day earlier when downloads started failing, but ... oh well.

You can always check the server status page to see if the problem has been resolved. The WUs in question remain stuck "in progress". Once that number falls to reasonable levels you can assume the problem has been fixed for most users.

On the upside: time for maintenance!

It's still hit and miss for me too. The assurance that the issue is being worked on is all I wanted really.

Well, fact is: the whole day is over, and nothing has happened :-(
So, I am not so sure that the issue has been worked on.