LHC@home

Hi Everyone,

I've got a completed task that is trying to upload but has encountered messages about a 'permissions' failure, as below:

09/07/2012 17:20:57 | LHC@home 1.0 | Computation for task w1jul_niebb0d__22__s__64.28_59.31__9.4_9.6__6__15_1_sixvf_boinc38018_2 finished
09/07/2012 17:21:00 | LHC@home 1.0 | Started upload of w1jul_niebb0d__22__s__64.28_59.31__9.4_9.6__6__15_1_sixvf_boinc38018_2_0
09/07/2012 17:21:04 | LHC@home 1.0 | [error] Error reported by file upload server: can't open file /data/boinc/project/sixtrack/upload/17a/w1jul_niebb0d__22__s__64.28_59.31__9.4_9.6__6__15_1_sixvf_boinc38018_2_0: Permission denied
09/07/2012 17:21:04 | LHC@home 1.0 | Temporarily failed upload of w1jul_niebb0d__22__s__64.28_59.31__9.4_9.6__6__15_1_sixvf_boinc38018_2_0: transient upload error
09/07/2012 17:21:04 | LHC@home 1.0 | Backing off 2 min 3 sec on upload of w1jul_niebb0d__22__s__64.28_59.31__9.4_9.6__6__15_1_sixvf_boinc38018_2_0
09/07/2012 17:23:06 | LHC@home 1.0 | Started upload of w1jul_niebb0d__22__s__64.28_59.31__9.4_9.6__6__15_1_sixvf_boinc38018_2_0
09/07/2012 17:23:10 | LHC@home 1.0 | [error] Error reported by file upload server: can't open file /data/boinc/project/sixtrack/upload/17a/w1jul_niebb0d__22__s__64.28_59.31__9.4_9.6__6__15_1_sixvf_boinc38018_2_0: Permission denied
09/07/2012 17:23:10 | LHC@home 1.0 | Temporarily failed upload of w1jul_niebb0d__22__s__64.28_59.31__9.4_9.6__6__15_1_sixvf_boinc38018_2_0: transient upload error
09/07/2012 17:23:10 | LHC@home 1.0 | Backing off 5 min 9 sec on upload of w1jul_niebb0d__22__s__64.28_59.31__9.4_9.6__6__15_1_sixvf_boinc38018_2_0
09/07/2012 17:25:32 | LHC@home 1.0 | Started upload of w1jul_niebb0d__22__s__64.28_59.31__9.4_9.6__6__15_1_sixvf_boinc38018_2_0
09/07/2012 17:25:36 | LHC@home 1.0 | [error] Error reported by file upload server: can't open file /data/boinc/project/sixtrack/upload/17a/w1jul_niebb0d__22__s__64.28_59.31__9.4_9.6__6__15_1_sixvf_boinc38018_2_0: Permission denied
09/07/2012 17:25:36 | LHC@home 1.0 | Temporarily failed upload of w1jul_niebb0d__22__s__64.28_59.31__9.4_9.6__6__15_1_sixvf_boinc38018_2_0: transient upload error
09/07/2012 17:25:36 | LHC@home 1.0 | Backing off 9 min 24 sec on upload of w1jul_niebb0d__22__s__64.28_59.31__9.4_9.6__6__15_1_sixvf_boinc38018_2_0

Anyone else seeing this issue? My concern is that it willi exponentially continue to 'back-off'....

Best regards,

Ged

See News post http://lhcathomeclassic.cern.ch/sixtrack/forum_thread.php?id=3455&nowrap=true#24191. Pete, project admin has now corrected the permissions problem.

For me it is still not working that uploading...

---

Greetings from,
TJ

Ditto,here.
Greetings.

As per Pete's comments on the News board, the issue I mentioned in this post seems now to be resolved.

Thanks,

Ged

If some users are still experiencing problems then please post the main error messages. The filesystem full issue has been resolved and so has the write permissions. Any details would be helpful

If some users are still experiencing problems then please post the main error messages. The filesystem full issue has been resolved and so has the write permissions. Any details would be helpful

this machine has pending tasks that are of the 1-2 second variety (about 84 of them).

heres an example of the message from one of them (task 3965742):

Stderr output

<core_client_version>7.0.25</core_client_version>
<![CDATA[
<stderr_txt>
[../../projects/lhcathomeclassic.cern.ch_sixtrack/w1jul_niebb1d__27__s__64.28_59.31__10.2_10.4__6__11_1_sixvf_boinc47270.zip]
End-of-central-directory signature not found. Either this file is not
a zipfile, or it constitutes one disk of a multi-part archive. In the
latter case the central directory and zipfile comment will be found on
the last disk(s) of this archive.
unzip: cannot find zipfile directory in ../../projects/lhcathomeclassic.cern.ch_sixtrack/w1jul_niebb1d__27__s__64.28_59.31__10.2_10.4__6__11_1_sixvf_boinc47270.zip,
and cannot find ../../projects/lhcathomeclassic.cern.ch_sixtrack/w1jul_niebb1d__27__s__64.28_59.31__10.2_10.4__6__11_1_sixvf_boinc47270.zip.zip, period.
17:05:40 (3796): called boinc_finish

</stderr_txt>
]]>

im pretty sure some of these tasks occured prior to the disk full problem but some of them are also from a few minutes ago which makes them after the point you mention this issue was corrected.

i do not know what other information you would like regarding this but ill provide any further information you may find helpful, just let me know.

i got the same issues today on several other boxes as well. just posting the results from one machine.

thanks.

rob

EDIT:

currently, my main machines do not have any lhc work, their caches were depleted and when they request more work, the server does not send any tasks. this occurred on machines with no cpu work at the time as they are set up to only crunch lhc so i know its not a full work cache issue. these machines are set up with a 2.0/0.1 work cache.

---

If some users are still experiencing problems then please post the main error messages. The filesystem full issue has been resolved and so has the write permissions. Any details would be helpful

Pete, I have copied this error messages over and over and over at several threads with no replay from admins.

10/07/2012 02:51:43 LHC@home 1.0 Started upload of w1jul_niebb1d__19__s__64.28_59.31__8.8_9__6__74_1_sixvf_boinc32470_1_0
10/07/2012 02:59:31 Project communication failed: attempting access to reference site
10/07/2012 02:59:31 LHC@home 1.0 Temporarily failed upload of w1jul_niebb1d__19__s__64.28_59.31__8.8_9__6__74_1_sixvf_boinc32470_1_0: HTTP error
10/07/2012 02:59:31 LHC@home 1.0 Backing off 55 min 54 sec on upload of w1jul_niebb1d__19__s__64.28_59.31__8.8_9__6__74_1_sixvf_boinc32470_1_0
10/07/2012 02:59:32 Internet access OK - project servers may be temporarily down.
10/07/2012 03:55:26 LHC@home 1.0 Started upload of w1jul_niebb1d__19__s__64.28_59.31__8.8_9__6__74_1_sixvf_boinc32470_1_0
10/07/2012 04:03:31 Project communication failed: attempting access to reference site
10/07/2012 04:03:31 LHC@home 1.0 Temporarily failed upload of w1jul_niebb1d__19__s__64.28_59.31__8.8_9__6__74_1_sixvf_boinc32470_1_0: HTTP error
10/07/2012 04:03:31 LHC@home 1.0 Backing off 2 hr 54 min 28 sec on upload of w1jul_niebb1d__19__s__64.28_59.31__8.8_9__6__74_1_sixvf_boinc32470_1_0
10/07/2012 04:03:32 Internet access OK - project servers may be temporarily down.
10/07/2012 06:58:00 LHC@home 1.0 Started upload of w1jul_niebb1d__19__s__64.28_59.31__8.8_9__6__74_1_sixvf_boinc32470_1_0
10/07/2012 07:03:10 Project communication failed: attempting access to reference site
10/07/2012 07:03:10 LHC@home 1.0 Temporarily failed upload of w1jul_niebb1d__19__s__64.28_59.31__8.8_9__6__74_1_sixvf_boinc32470_1_0: HTTP error
10/07/2012 07:03:10 LHC@home 1.0 Backing off 41 min 45 sec on upload of w1jul_niebb1d__19__s__64.28_59.31__8.8_9__6__74_1_sixvf_boinc32470_1_0
10/07/2012 07:03:12 Internet access OK - project servers may be temporarily down.
10/07/2012 07:44:56 LHC@home 1.0 Started upload of w1jul_niebb1d__19__s__64.28_59.31__8.8_9__6__74_1_sixvf_boinc32470_1_0
10/07/2012 07:50:05 Project communication failed: attempting access to reference site
10/07/2012 07:50:05 LHC@home 1.0 Temporarily failed upload of w1jul_niebb1d__19__s__64.28_59.31__8.8_9__6__74_1_sixvf_boinc32470_1_0: HTTP error
10/07/2012 07:50:05 LHC@home 1.0 Backing off 1 hr 23 min 22 sec on upload of w1jul_niebb1d__19__s__64.28_59.31__8.8_9__6__74_1_sixvf_boinc32470_1_0
10/07/2012 07:50:07 Internet access OK - project servers may be temporarily down.
10/07/2012 07:52:10 LHC@home 1.0 Started upload of w1jul_niebb1d__19__s__64.28_59.31__8.8_9__6__74_1_sixvf_boinc32470_1_0

---

Greetings from,
TJ

Multiple machines in different locations all trying to upload.

Upload: Pending(project backoff: 01:xx:xx:xx) errors

-------------------------------------------------------------------
10/23/2013 2:37:06 AM | | No config file found - using defaults
10/23/2013 2:37:06 AM | | Starting BOINC client version 7.0.64 for windows_x86_64
10/23/2013 2:37:06 AM | | log flags: file_xfer, sched_ops, task
10/23/2013 2:37:06 AM | | Libraries: libcurl/7.25.0 OpenSSL/1.0.1 zlib/1.2.6
10/23/2013 2:37:06 AM | | Data directory: D:\DATA\BOINC
10/23/2013 2:37:06 AM | | Running under account user
10/23/2013 2:37:06 AM | | Processor: 4 AuthenticAMD AMD Phenom(tm) II X4 945 Processor [Family 16 Model 4 Stepping 3]
10/23/2013 2:37:06 AM | | Processor features: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 htt pni cx16 popcnt syscall nx lm svm sse4a osvw ibs skinit wdt page1gb rdtscp 3dnowext 3dnow
10/23/2013 2:37:06 AM | | OS: Microsoft Windows 7: Professional x64 Edition, (06.01.7600.00)
10/23/2013 2:37:06 AM | | Memory: 8.00 GB physical, 16.00 GB virtual
10/23/2013 2:37:06 AM | | Disk: 116.34 GB total, 40.48 GB free
10/23/2013 2:37:06 AM | | Local time is UTC -4 hours
10/23/2013 2:37:06 AM | | CUDA: NVIDIA GPU 0: GeForce GT 620 (driver version 320.49, CUDA version 5.50, compute capability 2.1, 1024MB, 939MB available, 269 GFLOPS peak)
10/23/2013 2:37:06 AM | | OpenCL: NVIDIA GPU 0: GeForce GT 620 (driver version 320.49, device version OpenCL 1.1 CUDA, 1024MB, 939MB available, 269 GFLOPS peak)
10/23/2013 2:37:06 AM | LHC@home 1.0 | URL http://lhcathomeclassic.cern.ch/sixtrack/; Computer ID 10293034; resource share 100
10/23/2013 2:37:06 AM | | Reading preferences override file
10/23/2013 2:37:06 AM | | Preferences:
10/23/2013 2:37:06 AM | | max memory usage when active: 4095.65MB
10/23/2013 2:37:06 AM | | max memory usage when idle: 7372.17MB
10/23/2013 2:37:06 AM | | max disk usage: 10.00GB
10/23/2013 2:37:06 AM | | suspend work if non-BOINC CPU load exceeds 20 %
10/23/2013 2:37:06 AM | | (to change preferences, visit a project web site or select Preferences in the Manager)
10/23/2013 2:37:06 AM | | Not using a proxy
10/23/2013 2:37:13 AM | LHC@home 1.0 | Started upload of sd_sixt42_500_2.3_6D_e2.62__1__s__62.31_60.32__2_4__6__20_1_sixvf_boinc4_1_0
10/23/2013 2:37:13 AM | LHC@home 1.0 | Started upload of sd_sixt42_500_2.3_6D_e2.62__1__s__62.31_60.32__2_4__6__20_1_sixvf_boinc4_1_6
10/23/2013 2:37:55 AM | LHC@home 1.0 | [error] Error reported by file upload server: can't open log file '../log_boinc05/file_upload_handler.log' (errno: 9)
10/23/2013 2:37:55 AM | LHC@home 1.0 | [error] Error reported by file upload server: can't open log file '../log_boinc05/file_upload_handler.log' (errno: 9)
10/23/2013 2:37:55 AM | LHC@home 1.0 | Temporarily failed upload of sd_sixt42_500_2.3_6D_e2.62__1__s__62.31_60.32__2_4__6__20_1_sixvf_boinc4_1_0: transient upload error
10/23/2013 2:37:55 AM | LHC@home 1.0 | Backing off 5 min 9 sec on upload of sd_sixt42_500_2.3_6D_e2.62__1__s__62.31_60.32__2_4__6__20_1_sixvf_boinc4_1_0
10/23/2013 2:37:55 AM | LHC@home 1.0 | Temporarily failed upload of sd_sixt42_500_2.3_6D_e2.62__1__s__62.31_60.32__2_4__6__20_1_sixvf_boinc4_1_6: transient upload error
10/23/2013 2:37:55 AM | LHC@home 1.0 | Backing off 4 min 52 sec on upload of sd_sixt42_500_2.3_6D_e2.62__1__s__62.31_60.32__2_4__6__20_1_sixvf_boinc4_1_6
10/23/2013 2:37:56 AM | LHC@home 1.0 | Started upload of sd_sixt42_500_2.3_6D_e2.62__1__s__62.31_60.32__2_4__6__20_1_sixvf_boinc4_1_9
10/23/2013 2:37:56 AM | LHC@home 1.0 | Started upload of sd_sixt15_440_0.9_4D__1__s__62.31_60.32__4_6__6__45_1_sixvf_boinc26_1_0
10/23/2013 2:38:13 AM | LHC@home 1.0 | [error] Error reported by file upload server: can't open log file '../log_boinc05/file_upload_handler.log' (errno: 9)
10/23/2013 2:38:13 AM | LHC@home 1.0 | [error] Error reported by file upload server: can't open log file '../log_boinc05/file_upload_handler.log' (errno: 9)
10/23/2013 2:38:13 AM | LHC@home 1.0 | Temporarily failed upload of sd_sixt42_500_2.3_6D_e2.62__1__s__62.31_60.32__2_4__6__20_1_sixvf_boinc4_1_9: transient upload error
10/23/2013 2:38:13 AM | LHC@home 1.0 | Backing off 7 min 2 sec on upload of sd_sixt42_500_2.3_6D_e2.62__1__s__62.31_60.32__2_4__6__20_1_sixvf_boinc4_1_9
10/23/2013 2:38:13 AM | LHC@home 1.0 | Temporarily failed upload of sd_sixt15_440_0.9_4D__1__s__62.31_60.32__4_6__6__45_1_sixvf_boinc26_1_0: transient upload error
10/23/2013 2:38:13 AM | LHC@home 1.0 | Backing off 7 min 19 sec on upload of sd_sixt15_440_0.9_4D__1__s__62.31_60.32__4_6__6__45_1_sixvf_boinc26_1_0
10/23/2013 2:38:24 AM | | Suspending computation - CPU is busy
10/23/2013 2:38:54 AM | | Resuming computation
10/23/2013 2:39:05 AM | | Suspending computation - CPU is busy
10/23/2013 2:39:25 AM | | Resuming computation
10/23/2013 2:39:32 AM | LHC@home 1.0 | update requested by user
10/23/2013 2:39:44 AM | LHC@home 1.0 | Sending scheduler request: Requested by user.
10/23/2013 2:39:44 AM | LHC@home 1.0 | Not requesting tasks: too many uploads in progress
10/23/2013 2:39:45 AM | LHC@home 1.0 | Scheduler request completed
10/23/2013 2:40:48 AM | | Suspending computation - CPU is busy
10/23/2013 2:41:08 AM | | Resuming computation
10/23/2013 2:42:59 AM | LHC@home 1.0 | update requested by user
10/23/2013 2:43:00 AM | LHC@home 1.0 | Sending scheduler request: Requested by user.
10/23/2013 2:43:00 AM | LHC@home 1.0 | Not requesting tasks: too many uploads in progress
10/23/2013 2:43:02 AM | LHC@home 1.0 | Scheduler request completed
10/23/2013 2:46:38 AM | | Suspending computation - CPU is busy
10/23/2013 2:46:48 AM | | Resuming computation
10/23/2013 2:47:56 AM | LHC@home 1.0 | Started upload of sd_sixt42_500_2.3_6D_e2.62__1__s__62.31_60.32__2_4__6__20_1_sixvf_boinc4_1_0
10/23/2013 2:47:56 AM | LHC@home 1.0 | Started upload of sd_sixt42_500_2.3_6D_e2.62__1__s__62.31_60.32__2_4__6__20_1_sixvf_boinc4_1_6
10/23/2013 2:47:58 AM | LHC@home 1.0 | [error] Error reported by file upload server: can't open log file '../log_boinc05/file_upload_handler.log' (errno: 9)
10/23/2013 2:47:58 AM | LHC@home 1.0 | [error] Error reported by file upload server: can't open log file '../log_boinc05/file_upload_handler.log' (errno: 9)
10/23/2013 2:47:58 AM | LHC@home 1.0 | Temporarily failed upload of sd_sixt42_500_2.3_6D_e2.62__1__s__62.31_60.32__2_4__6__20_1_sixvf_boinc4_1_0: transient upload error
10/23/2013 2:47:58 AM | LHC@home 1.0 | Backing off 13 min 41 sec on upload of sd_sixt42_500_2.3_6D_e2.62__1__s__62.31_60.32__2_4__6__20_1_sixvf_boinc4_1_0
10/23/2013 2:47:58 AM | LHC@home 1.0 | Temporarily failed upload of sd_sixt42_500_2.3_6D_e2.62__1__s__62.31_60.32__2_4__6__20_1_sixvf_boinc4_1_6: transient upload error
10/23/2013 2:47:58 AM | LHC@home 1.0 | Backing off 10 min 51 sec on upload of sd_sixt42_500_2.3_6D_e2.62__1__s__62.31_60.32__2_4__6__20_1_sixvf_boinc4_1_6
10/23/2013 2:48:00 AM | LHC@home 1.0 | Started upload of sd_sixt42_500_2.3_6D_e2.62__1__s__62.31_60.32__2_4__6__20_1_sixvf_boinc4_1_9
10/23/2013 2:48:00 AM | LHC@home 1.0 | Started upload of sd_sixt15_440_0.9_4D__1__s__62.31_60.32__4_6__6__45_1_sixvf_boinc26_1_0
10/23/2013 2:48:02 AM | LHC@home 1.0 | [error] Error reported by file upload server: can't open log file '../log_boinc05/file_upload_handler.log' (errno: 9)
10/23/2013 2:48:02 AM | LHC@home 1.0 | [error] Error reported by file upload server: can't open log file '../log_boinc05/file_upload_handler.log' (errno: 9)
10/23/2013 2:48:02 AM | LHC@home 1.0 | Temporarily failed upload of sd_sixt42_500_2.3_6D_e2.62__1__s__62.31_60.32__2_4__6__20_1_sixvf_boinc4_1_9: transient upload error
10/23/2013 2:48:02 AM | LHC@home 1.0 | Backing off 15 min 14 sec on upload of sd_sixt42_500_2.3_6D_e2.62__1__s__62.31_60.32__2_4__6__20_1_sixvf_boinc4_1_9
10/23/2013 2:48:02 AM | LHC@home 1.0 | Temporarily failed upload of sd_sixt15_440_0.9_4D__1__s__62.31_60.32__4_6__6__45_1_sixvf_boinc26_1_0: transient upload error
10/23/2013 2:48:02 AM | LHC@home 1.0 | Backing off 15 min 19 sec on upload of sd_sixt15_440_0.9_4D__1__s__62.31_60.32__4_6__6__45_1_sixvf_boinc26_1_0
10/23/2013 3:11:54 AM | LHC@home 1.0 | Started upload of sd_sixt42_500_2.3_6D_e2.62__1__s__62.31_60.32__2_4__6__20_1_sixvf_boinc4_1_0
10/23/2013 3:11:54 AM | LHC@home 1.0 | Started upload of sd_sixt42_500_2.3_6D_e2.62__1__s__62.31_60.32__2_4__6__20_1_sixvf_boinc4_1_6
10/23/2013 3:11:56 AM | LHC@home 1.0 | [error] Error reported by file upload server: can't open log file '../log_boinc05/file_upload_handler.log' (errno: 9)
10/23/2013 3:11:56 AM | LHC@home 1.0 | [error] Error reported by file upload server: can't open log file '../log_boinc05/file_upload_handler.log' (errno: 9)
10/23/2013 3:11:56 AM | LHC@home 1.0 | Temporarily failed upload of sd_sixt42_500_2.3_6D_e2.62__1__s__62.31_60.32__2_4__6__20_1_sixvf_boinc4_1_0: transient upload error
10/23/2013 3:11:56 AM | LHC@home 1.0 | Backing off 16 min 10 sec on upload of sd_sixt42_500_2.3_6D_e2.62__1__s__62.31_60.32__2_4__6__20_1_sixvf_boinc4_1_0
10/23/2013 3:11:56 AM | LHC@home 1.0 | Temporarily failed upload of sd_sixt42_500_2.3_6D_e2.62__1__s__62.31_60.32__2_4__6__20_1_sixvf_boinc4_1_6: transient upload error
10/23/2013 3:11:56 AM | LHC@home 1.0 | Backing off 30 min 58 sec on upload of sd_sixt42_500_2.3_6D_e2.62__1__s__62.31_60.32__2_4__6__20_1_sixvf_boinc4_1_6

@Costa

See what I mean about kittens? The problem has been reported in 3 other threads with posts that already give the pertinent error message and now a 4th thread with a post that contains the whole Messages. Lucky he doesn't know about stderrdae.txt.

It's spreading faster than a common cold in a daycare.

Yeah, I don't think this is just you. I have three linux boxes with WUs to upload stacking up, and the same error 9.

I would tentatively guess the partition on the upload server where the log files live has run out of space, but hopefully Eric and the others will see this and work on a fix.

---

Yes. Same here.:(