Message boards : Number crunching : Error reported by file upload server: Server is out of disk space
Message board moderation

To post messages, you must log in.

AuthorMessage
Toby Broom
Volunteer moderator

Send message
Joined: 27 Sep 08
Posts: 931
Credit: 780,987,563
RAC: 81,604
Message 53023 - Posted: 12 Feb 2026, 21:22:04 UTC

As title
ID: 53023 · Report as offensive     Reply Quote
Crystal Pellet
Volunteer moderator
Volunteer tester

Send message
Joined: 14 Jan 10
Posts: 1551
Credit: 10,067,691
RAC: 816
Message 53026 - Posted: 13 Feb 2026, 9:26:50 UTC

Solved for the moment . . .
ID: 53026 · Report as offensive     Reply Quote
casi34

Send message
Joined: 16 Nov 25
Posts: 3
Credit: 3,232,836
RAC: 15,231
Message 53198 - Posted: 19 Mar 2026, 17:17:41 UTC
Last modified: 19 Mar 2026, 17:19:21 UTC

The problem seems to have appeared again today with ATLAS tasks.

jeu. 19 mars 2026 17:37:40 | LHC@home | [error] Error reported by file upload server: Server is out of disk space
jeu. 19 mars 2026 17:37:40 | LHC@home | Temporarily failed upload of XaMKDmA05M9nsSi4ap6QjLDmwznN0nGgGQJmIVkRDmh31LDmqjhiln_0_r2007412987_ATLAS_result: transient upload error
ID: 53198 · Report as offensive     Reply Quote
dskagcommunity
Avatar

Send message
Joined: 17 Oct 11
Posts: 13
Credit: 1,568,472
RAC: 4,436
Message 53212 - Posted: 20 Mar 2026, 8:57:34 UTC

Not just atlas :(
---------- 24/7 Crunching since 2011 -----------
DSKAG Austria: http://www.dskag.at

ID: 53212 · Report as offensive     Reply Quote
Erich56

Send message
Joined: 18 Dec 15
Posts: 1978
Credit: 160,452,304
RAC: 49,488
Message 53213 - Posted: 20 Mar 2026, 9:54:50 UTC

what surprises me is that so far no one over there seems to have noticed that the disks are full :-(
Besides, I wonder how this can happen - are there no alert provisions in place, e.g. automatic notification to the person in charge once the disks reach a certain filling percentage, or other things (blinking light, sound alarm, ... :-)
ID: 53213 · Report as offensive     Reply Quote
Ivan Kartsev

Send message
Joined: 7 Aug 13
Posts: 6
Credit: 1,134,166
RAC: 9,090
Message 53214 - Posted: 20 Mar 2026, 9:58:24 UTC - in response to Message 53213.  

I have same questions)
ID: 53214 · Report as offensive     Reply Quote
dskagcommunity
Avatar

Send message
Joined: 17 Oct 11
Posts: 13
Credit: 1,568,472
RAC: 4,436
Message 53215 - Posted: 20 Mar 2026, 10:58:50 UTC

We just can wait and hope, weekend is coming :/

But yeah in most projects i wonder sometimes, but then i realize they does no have the personal for such things, but CERN? O.o

Set up a PRTG instance or somethin for your IT Support.
---------- 24/7 Crunching since 2011 -----------
DSKAG Austria: http://www.dskag.at

ID: 53215 · Report as offensive     Reply Quote
Erich56

Send message
Joined: 18 Dec 15
Posts: 1978
Credit: 160,452,304
RAC: 49,488
Message 53216 - Posted: 20 Mar 2026, 11:07:08 UTC - in response to Message 53215.  

In reply to dskagcommunity's message of 20 Mar 2026:
We just can wait and hope, weekend is coming :/

But yeah in most projects i wonder sometimes, but then i realize they does no have the personal for such things, but CERN? O.o

Set up a PRTG instance or somethin for your IT Support.
I am used to the "disk full" problem with GPUGRID - there is happens quite often, and I am wondering that still they haven't set up procedures to avoid this problem (However, they seem to have quite different problems lately - the availability of tasks has decreaded markedly within the past 1 year - sometimes I have the impression that the project does no longer exist).

But that CERN would ever run out of disk space - that's new to me :-)
ID: 53216 · Report as offensive     Reply Quote
Erich56

Send message
Joined: 18 Dec 15
Posts: 1978
Credit: 160,452,304
RAC: 49,488
Message 53222 - Posted: 20 Mar 2026, 15:12:31 UTC

short time ago, back-upped tasks started to upload :-)

However, no new ones available for download :-(
ID: 53222 · Report as offensive     Reply Quote
Anne Havinga

Send message
Joined: 4 Mar 20
Posts: 23
Credit: 7,998,613
RAC: 8,357
Message 53285 - Posted: 25 Mar 2026, 18:47:43 UTC - in response to Message 53222.  

Happening again.

25/03/2026 19:42:19 | LHC@home | [error] Error reported by file upload server: Server is out of disk space
ID: 53285 · Report as offensive     Reply Quote
Erich56

Send message
Joined: 18 Dec 15
Posts: 1978
Credit: 160,452,304
RAC: 49,488
Message 53286 - Posted: 25 Mar 2026, 19:17:51 UTC

same here :-(

Unbelieveable that there is no pre-alert system in place.
What has changed vis-a-vis all the years before? I don't remember having experienced this problem at any earlier time.
ID: 53286 · Report as offensive     Reply Quote
dskagcommunity
Avatar

Send message
Joined: 17 Oct 11
Posts: 13
Credit: 1,568,472
RAC: 4,436
Message 53287 - Posted: 25 Mar 2026, 20:09:04 UTC

Ooohhh boiiiiiiii
---------- 24/7 Crunching since 2011 -----------
DSKAG Austria: http://www.dskag.at

ID: 53287 · Report as offensive     Reply Quote
Ivan Kartsev

Send message
Joined: 7 Aug 13
Posts: 6
Credit: 1,134,166
RAC: 9,090
Message 53288 - Posted: 25 Mar 2026, 20:17:57 UTC

same here
ID: 53288 · Report as offensive     Reply Quote
ProfileMagic Quantum Mechanic
Avatar

Send message
Joined: 24 Oct 04
Posts: 1300
Credit: 95,664,935
RAC: 22,681
Message 53291 - Posted: 25 Mar 2026, 22:43:38 UTC
Last modified: 25 Mar 2026, 22:58:30 UTC

I have 2 -dev Dockers and one Docker for here that have been trying to send in for a couple hours but no luck (and some 8-core Atlas)
ID: 53291 · Report as offensive     Reply Quote
Garrulus glandarius

Send message
Joined: 5 Apr 25
Posts: 82
Credit: 2,389,259
RAC: 7,919
Message 53295 - Posted: 26 Mar 2026, 7:11:56 UTC - in response to Message 53291.  

In reply to Magic Quantum Mechanic's message of 25 Mar 2026:
(and some 8-core Atlas)

I'll take the opportunity to veer off-topic a bit. Is it worth trying to run 8-core ATLAS instead of quad? Does CPU efficiency remain linear? At one point I set the max # to 8 but still kept getting quads (on a 8C/16T CPU) so I reverted the settings and left it at 4.
ID: 53295 · Report as offensive     Reply Quote
Crystal Pellet
Volunteer moderator
Volunteer tester

Send message
Joined: 14 Jan 10
Posts: 1551
Credit: 10,067,691
RAC: 816
Message 53299 - Posted: 26 Mar 2026, 8:18:37 UTC - in response to Message 53291.  

In reply to Magic Quantum Mechanic's message of 25 Mar 2026:
I have 2 -dev Dockers and one Docker for here that have been trying to send in for a couple hours but no luck (and some 8-core Atlas)
Sending Theory results to the dev-system is no problem for me.
What I've understood, that there's a problem with storage further in the chain for ATLAS jobs that should be transferrred from the BOINC server to their system.
ID: 53299 · Report as offensive     Reply Quote
Garrulus glandarius

Send message
Joined: 5 Apr 25
Posts: 82
Credit: 2,389,259
RAC: 7,919
Message 53302 - Posted: 26 Mar 2026, 10:27:36 UTC
Last modified: 26 Mar 2026, 10:37:08 UTC

Managed to upload and report 4 of 8 tasks waiting. Now the disk is full again. Retries sometimes help, maybe they started clearing up the clutter.

After a few minutes and a few more retries: 8/8
ID: 53302 · Report as offensive     Reply Quote
ProfileMagic Quantum Mechanic
Avatar

Send message
Joined: 24 Oct 04
Posts: 1300
Credit: 95,664,935
RAC: 22,681
Message 53303 - Posted: 26 Mar 2026, 10:57:47 UTC - in response to Message 53299.  
Last modified: 26 Mar 2026, 11:31:16 UTC

In reply to Crystal Pellet's message of 26 Mar 2026:
In reply to Magic Quantum Mechanic's message of 25 Mar 2026:
I have 2 -dev Dockers and one Docker for here that have been trying to send in for a couple hours but no luck (and some 8-core Atlas)
Sending Theory results to the dev-system is no problem for me.
What I've understood, that there's a problem with storage further in the chain for ATLAS jobs that should be transferrred from the BOINC server to their system.


It was a certain time that they would not send them in where I would get it at -dev for both Docker and Atlas 3/25/2026 11:23:21 PM | lhcathome-dev | Backing off 04:32:12 on upload of Theory_2922-4790729-720_0_r950267645_result
But since I am always awake and checking after 3am I tried and they are all sent in finally.......although uploads of Atlas from my Starlink is a bit slow and about twice as long as a d/l
But they are in and I am going to run a few less right now than usual for a while
It was during a time when you didn't have any to send in at dev

Edit ; it just did it again at -dev with a Docker 3/26/2026 4:23:11 AM | lhcathome-dev | Backing off 00:15:31 on upload of Theory_2922-4896208-720_0_r1298637244_result
Internet speed test

263.3 Mbps download
13.4 Mbps upload ..........(4:30am goodnight)
ID: 53303 · Report as offensive     Reply Quote
Harri Liljeroos
Avatar

Send message
Joined: 28 Sep 04
Posts: 802
Credit: 65,414,561
RAC: 24,704
Message 53313 - Posted: 27 Mar 2026, 8:38:50 UTC

Upload server has some new error:

5007	LHC@home	27-03-2026 10:34	[error] Error reported by file upload server: EOF on socket read : asked for 262144, got 46720	

ID: 53313 · Report as offensive     Reply Quote

Message boards : Number crunching : Error reported by file upload server: Server is out of disk space


©2026 CERN