Message boards : ATLAS application : Download failures
Message board moderation

To post messages, you must log in.

Previous · 1 · 2 · 3 · 4 · 5 · 6 · 7 · 8 · Next

AuthorMessage
Profile Nils Høimyr
Volunteer moderator
Project administrator
Project developer
Project tester

Send message
Joined: 15 Jul 05
Posts: 247
Credit: 5,974,599
RAC: 0
Message 36549 - Posted: 24 Aug 2018, 6:51:31 UTC - in response to Message 36548.  

Thanks guys for letting us know. One of our 3 file servers was sick. It is being fixed now.
ID: 36549 · Report as offensive     Reply Quote
computezrmle
Volunteer moderator
Volunteer developer
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 15 Jun 08
Posts: 2521
Credit: 252,640,499
RAC: 138,028
Message 36550 - Posted: 24 Aug 2018, 6:52:35 UTC
Last modified: 24 Aug 2018, 6:53:34 UTC

Don't know if this has something to do with the download errors:
This morning 8:25 CEST my ISP forced a DSL reconnect including a fresh public IP for my router.
Th connection to the outdoor cabinet remained untouched an is error free for more than 3 months.

<edit>
Thank you Nils.
I was a bit too slow.
</edit>
ID: 36550 · Report as offensive     Reply Quote
Erich56

Send message
Joined: 18 Dec 15
Posts: 1788
Credit: 117,781,886
RAC: 85,266
Message 36690 - Posted: 12 Sep 2018, 20:19:51 UTC

Are the ATLAS download problems back?

One of my PCs is downloading a new task right now, and has been taking about 5 minutes for the first 2%.

Any new problems around?
ID: 36690 · Report as offensive     Reply Quote
Jim1348

Send message
Joined: 15 Nov 14
Posts: 602
Credit: 24,371,321
RAC: 0
Message 36691 - Posted: 12 Sep 2018, 21:51:28 UTC - in response to Message 36690.  
Last modified: 12 Sep 2018, 22:08:39 UTC

Well, since you asked, I am uploading a native ATLAS now at 1000 Kbps.
But I am downloading another one at 19 Kbps.

So yes, the problems are back. (Please don't ask next time.)

EDIT: But another one just started downloading at 2000 Kbps, but going down very rapidly to about 85 Kbps now.
ID: 36691 · Report as offensive     Reply Quote
bronco

Send message
Joined: 13 Apr 18
Posts: 443
Credit: 8,438,885
RAC: 0
Message 36692 - Posted: 12 Sep 2018, 22:04:03 UTC - in response to Message 36690.  

I am seeing extremely slow rates, 25 to 60 KBps, but at least they are completing.
ID: 36692 · Report as offensive     Reply Quote
Jim1348

Send message
Joined: 15 Nov 14
Posts: 602
Credit: 24,371,321
RAC: 0
Message 36693 - Posted: 13 Sep 2018, 1:02:32 UTC

Uploads are still OK for me, but the latest download stuck with 0 progress for three minutes, and then started very slowly. It is now up to only 4 Kbps after 10 minutes.
If Erich had not alerted us, I would be out of work by morning.
ID: 36693 · Report as offensive     Reply Quote
Erich56

Send message
Joined: 18 Dec 15
Posts: 1788
Credit: 117,781,886
RAC: 85,266
Message 36694 - Posted: 13 Sep 2018, 5:23:34 UTC

a minute ago, I watched the upload of a finished task and the download of a new task - this time, it all went well, without any delay.
ID: 36694 · Report as offensive     Reply Quote
Jim1348

Send message
Joined: 15 Nov 14
Posts: 602
Credit: 24,371,321
RAC: 0
Message 36695 - Posted: 13 Sep 2018, 5:32:43 UTC - in response to Message 36694.  
Last modified: 13 Sep 2018, 5:33:07 UTC

I just started a download 12 minutes ago. It is still at only 18 Kbps.
ID: 36695 · Report as offensive     Reply Quote
maeax

Send message
Joined: 2 May 07
Posts: 2230
Credit: 173,865,740
RAC: 16,799
Message 36696 - Posted: 13 Sep 2018, 5:43:17 UTC - in response to Message 36549.  

Thanks guys for letting us know. One of our 3 file servers was sick. It is being fixed now.

Nils,
is it the same problem now?
The Atlas Dashboard show a downgrading of the slots since yesterday from 10k to 8k!
ID: 36696 · Report as offensive     Reply Quote
computezrmle
Volunteer moderator
Volunteer developer
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 15 Jun 08
Posts: 2521
Credit: 252,640,499
RAC: 138,028
Message 36772 - Posted: 19 Sep 2018, 6:33:51 UTC

<core_client_version>7.8.4</core_client_version>
<![CDATA[
<message>
WU download error: couldn't get input files:
<file_xfer_error>
  <file_name>J3nLDmjEVMtnlyackoJh5iwnABFKDmABFKDmMNxTDmABFKDmlfonCn_EVNT.15252537._000268.pool.root.1</file_name>
  <error_code>-224 (permanent HTTP error)</error_code>
  <error_message>permanent HTTP error</error_message>
</file_xfer_error>
</message>
]]>

Got 3 in a row since 6:00 UTC
ID: 36772 · Report as offensive     Reply Quote
Harri Liljeroos
Avatar

Send message
Joined: 28 Sep 04
Posts: 722
Credit: 48,442,809
RAC: 26,597
Message 36780 - Posted: 19 Sep 2018, 19:58:27 UTC - in response to Message 36772.  

I've got a few of those as well and just as I type this two more.
ID: 36780 · Report as offensive     Reply Quote
maeax

Send message
Joined: 2 May 07
Posts: 2230
Credit: 173,865,740
RAC: 16,799
Message 36904 - Posted: 27 Sep 2018, 6:31:38 UTC - in response to Message 36780.  

ID: 36904 · Report as offensive     Reply Quote
maeax

Send message
Joined: 2 May 07
Posts: 2230
Credit: 173,865,740
RAC: 16,799
Message 36952 - Posted: 4 Oct 2018, 17:02:03 UTC - in response to Message 36904.  

ID: 36952 · Report as offensive     Reply Quote
maeax

Send message
Joined: 2 May 07
Posts: 2230
Credit: 173,865,740
RAC: 16,799
Message 37016 - Posted: 14 Oct 2018, 5:39:54 UTC
Last modified: 14 Oct 2018, 5:42:22 UTC

ID: 37016 · Report as offensive     Reply Quote
maeax

Send message
Joined: 2 May 07
Posts: 2230
Credit: 173,865,740
RAC: 16,799
Message 37331 - Posted: 13 Nov 2018, 10:05:00 UTC

ID: 37331 · Report as offensive     Reply Quote
computezrmle
Volunteer moderator
Volunteer developer
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 15 Jun 08
Posts: 2521
Credit: 252,640,499
RAC: 138,028
Message 37332 - Posted: 13 Nov 2018, 11:04:50 UTC - in response to Message 37331.  

Your wingcomputers failed due to misconfigured setups, e.g. disabled VT-x or CVMFS/Singularity errors.
All of them had lots of errors in a row.
Only one of them looks OK now.

Your computer failed because ... hm ... I don't know.
It seems like it was one of those (most likely) local errors that can't be completely explained.
ID: 37332 · Report as offensive     Reply Quote
Darren Jones

Send message
Joined: 22 Aug 09
Posts: 5
Credit: 192,011
RAC: 0
Message 37534 - Posted: 4 Dec 2018, 18:16:56 UTC

I have just had 2 Atlas download fail:

04/12/2018 18:14:03 | LHC@home | update requested by user
04/12/2018 18:14:05 | LHC@home | Sending scheduler request: Requested by user.
04/12/2018 18:14:05 | LHC@home | Requesting new tasks for CPU and AMD/ATI GPU
04/12/2018 18:14:06 | LHC@home | Scheduler request completed: got 2 new tasks
04/12/2018 18:14:08 | LHC@home | Started download of xzeKDmD9RotnyYickojUe11pABFKDmABFKDm5fgLDmABFKDmrh2V2m_EVNT.16161801._001501.pool.root.1
04/12/2018 18:14:08 | LHC@home | Started download of xzeKDmD9RotnyYickojUe11pABFKDmABFKDm5fgLDmABFKDmrh2V2m_input.tar.gz
04/12/2018 18:14:10 | LHC@home | Giving up on download of xzeKDmD9RotnyYickojUe11pABFKDmABFKDm5fgLDmABFKDmrh2V2m_EVNT.16161801._001501.pool.root.1: permanent HTTP error
04/12/2018 18:14:10 | LHC@home | Finished download of xzeKDmD9RotnyYickojUe11pABFKDmABFKDm5fgLDmABFKDmrh2V2m_input.tar.gz
04/12/2018 18:14:10 | LHC@home | Started download of rte_xzeKDmD9RotnyYickojUe11pABFKDmABFKDm5fgLDmABFKDmrh2V2m.tar.gz
04/12/2018 18:14:10 | LHC@home | Started download of boinc_job_script.f4qLmV
04/12/2018 18:14:12 | LHC@home | Finished download of boinc_job_script.f4qLmV
04/12/2018 18:14:12 | LHC@home | Started download of 5mNNDmIUNotnlyackoJh5iwnABFKDmABFKDm80lNDmABFKDmPWWtdm_EVNT.16161801._001435.pool.root.1
04/12/2018 18:14:13 | LHC@home | Finished download of rte_xzeKDmD9RotnyYickojUe11pABFKDmABFKDm5fgLDmABFKDmrh2V2m.tar.gz
04/12/2018 18:14:13 | LHC@home | Giving up on download of 5mNNDmIUNotnlyackoJh5iwnABFKDmABFKDm80lNDmABFKDmPWWtdm_EVNT.16161801._001435.pool.root.1: permanent HTTP error
04/12/2018 18:14:13 | LHC@home | Started download of 5mNNDmIUNotnlyackoJh5iwnABFKDmABFKDm80lNDmABFKDmPWWtdm_input.tar.gz
04/12/2018 18:14:13 | LHC@home | Started download of rte_5mNNDmIUNotnlyackoJh5iwnABFKDmABFKDm80lNDmABFKDmPWWtdm.tar.gz
04/12/2018 18:14:14 | LHC@home | Finished download of rte_5mNNDmIUNotnlyackoJh5iwnABFKDmABFKDm80lNDmABFKDmPWWtdm.tar.gz
04/12/2018 18:14:14 | LHC@home | Started download of boinc_job_script.a59jrI
04/12/2018 18:14:15 | LHC@home | Finished download of 5mNNDmIUNotnlyackoJh5iwnABFKDmABFKDm80lNDmABFKDmPWWtd


Darren
ID: 37534 · Report as offensive     Reply Quote
computezrmle
Volunteer moderator
Volunteer developer
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 15 Jun 08
Posts: 2521
Credit: 252,640,499
RAC: 138,028
Message 37535 - Posted: 4 Dec 2018, 18:43:25 UTC - in response to Message 37534.  

Your wingnodes also failed.
Most likely a server error.
ID: 37535 · Report as offensive     Reply Quote
computezrmle
Volunteer moderator
Volunteer developer
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 15 Jun 08
Posts: 2521
Credit: 252,640,499
RAC: 138,028
Message 37539 - Posted: 4 Dec 2018, 21:13:08 UTC

The rate of ATLAS download errors is rapidly increasing.
Could it be a faulty batch?
ID: 37539 · Report as offensive     Reply Quote
Jim1348

Send message
Joined: 15 Nov 14
Posts: 602
Credit: 24,371,321
RAC: 0
Message 37544 - Posted: 5 Dec 2018, 12:08:21 UTC - in response to Message 37539.  

I am seeing that also.. In the last three hours, I have gotten five:
Giving up on download of 48ZMDmCtcotnlyackoJh5iwnABFKDmABFKDmgEFaDmABFKDmllmCpm_EVNT.16161801._001700.pool.root.1: permanent HTTP error


It has not interrupted crunching yet however.
ID: 37544 · Report as offensive     Reply Quote
Previous · 1 · 2 · 3 · 4 · 5 · 6 · 7 · 8 · Next

Message boards : ATLAS application : Download failures


©2024 CERN