Message boards : News : CERN AFS problems
Message board moderation

To post messages, you must log in.

1 · 2 · Next

AuthorMessage
Eric Mcintosh
Volunteer moderator
Project administrator
Project developer
Project tester
Project scientist

Send message
Joined: 12 Jul 11
Posts: 857
Credit: 1,619,050
RAC: 0
Message 26859 - Posted: 10 Oct 2014, 15:54:03 UTC

We seem to be having intermittent? problems with our local
file system. Server running but.....will fix soonest.
ID: 26859 · Report as offensive     Reply Quote
Profile Robert Pick

Send message
Joined: 1 Dec 05
Posts: 62
Credit: 11,398,274
RAC: 261
Message 26863 - Posted: 11 Oct 2014, 13:00:18 UTC - in response to Message 26859.  

I fired up this morning and received this message which also includes the fact that I reset the project, then reset the modem and router. Restarted computer and everything else, and still no love! So I detached from the project, reattached and here's what I got. 10/11/2014 5:26:58 AM | | cc_config.xml not found - using defaults
10/11/2014 5:26:59 AM | | Starting BOINC client version 7.2.42 for windows_x86_64
10/11/2014 5:26:59 AM | | log flags: file_xfer, sched_ops, task
10/11/2014 5:26:59 AM | | Libraries: libcurl/7.25.0 OpenSSL/1.0.1 zlib/1.2.6
10/11/2014 5:26:59 AM | | Data directory: C:\ProgramData\BOINC
10/11/2014 5:26:59 AM | | Running under account Robert Pick
10/11/2014 5:26:59 AM | | CUDA: NVIDIA GPU 0: GeForce GTS 450 (driver version 344.11, CUDA version 6.5, compute capability 2.1, 1024MB, 942MB available, 632 GFLOPS peak)
10/11/2014 5:26:59 AM | | OpenCL: NVIDIA GPU 0: GeForce GTS 450 (driver version 344.11, device version OpenCL 1.1 CUDA, 1024MB, 942MB available, 632 GFLOPS peak)
10/11/2014 5:26:59 AM | | Host name: Pick1
10/11/2014 5:26:59 AM | | Processor: 8 GenuineIntel Intel(R) Core(TM) i7 CPU 950 @ 3.07GHz [Family 6 Model 26 Stepping 5]
10/11/2014 5:26:59 AM | | Processor features: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss htt tm pni ssse3 cx16 sse4_1 sse4_2 popcnt syscall nx lm vmx tm2 pbe
10/11/2014 5:26:59 AM | | OS: Microsoft Windows 7: Professional x64 Edition, Service Pack 1, (06.01.7601.00)
10/11/2014 5:26:59 AM | | Memory: 8.99 GB physical, 17.98 GB virtual
10/11/2014 5:26:59 AM | | Disk: 931.51 GB total, 784.33 GB free
10/11/2014 5:26:59 AM | | Local time is UTC -7 hours
10/11/2014 5:26:59 AM | Collatz Conjecture | URL http://boinc.thesonntags.com/collatz/; Computer ID 150817; resource share 100
10/11/2014 5:26:59 AM | Einstein@Home | URL http://einstein.phys.uwm.edu/; Computer ID 11662743; resource share 100
10/11/2014 5:26:59 AM | LHC@home 1.0 | URL http://lhcathomeclassic.cern.ch/sixtrack/; Computer ID 10190874; resource share 100
10/11/2014 5:26:59 AM | Milkyway@Home | URL http://milkyway.cs.rpi.edu/milkyway/; Computer ID 461418; resource share 100
10/11/2014 5:26:59 AM | SETI@home | URL http://setiathome.berkeley.edu/; Computer ID 6743758; resource share 100
10/11/2014 5:26:59 AM | SETI@home | General prefs: from SETI@home (last modified 26-Sep-2014 11:42:12)
10/11/2014 5:26:59 AM | SETI@home | Computer location: home
10/11/2014 5:26:59 AM | SETI@home | General prefs: no separate prefs for home; using your defaults
10/11/2014 5:26:59 AM | | Reading preferences override file
10/11/2014 5:26:59 AM | | Preferences:
10/11/2014 5:26:59 AM | | max memory usage when active: 8285.51MB
10/11/2014 5:26:59 AM | | max memory usage when idle: 8285.51MB
10/11/2014 5:26:59 AM | | max disk usage: 100.00GB
10/11/2014 5:26:59 AM | | max download rate: 10240000 bytes/sec
10/11/2014 5:26:59 AM | | max upload rate: 10240000 bytes/sec
10/11/2014 5:26:59 AM | | (to change preferences, visit a project web site or select Preferences in the Manager)
10/11/2014 5:26:59 AM | | Not using a proxy
10/11/2014 5:27:05 AM | LHC@home 1.0 | Sending scheduler request: Requested by project.
10/11/2014 5:27:05 AM | LHC@home 1.0 | Requesting new tasks for CPU and NVIDIA
10/11/2014 5:27:10 AM | LHC@home 1.0 | Scheduler request completed: got 0 new tasks
10/11/2014 5:27:10 AM | LHC@home 1.0 | Server can't open log file (../log_boinc05/scheduler.log)
10/11/2014 5:30:03 AM | SETI@home | Computation for task 28fe08ac.19218.183565.438086664195.12.103_0 finished
10/11/2014 5:30:03 AM | SETI@home | Starting task 27mr08am.3408.23794.438086664206.12.230_1
10/11/2014 5:30:05 AM | SETI@home | Started upload of 28fe08ac.19218.183565.438086664195.12.103_0_0
10/11/2014 5:30:06 AM | SETI@home | Temporarily failed upload of 28fe08ac.19218.183565.438086664195.12.103_0_0: can't resolve hostname
10/11/2014 5:30:06 AM | SETI@home | Backing off 00:02:00 on upload of 28fe08ac.19218.183565.438086664195.12.103_0_0
10/11/2014 5:30:06 AM | | Project communication failed: attempting access to reference site
10/11/2014 5:30:07 AM | | BOINC can't access Internet - check network connection or proxy configuration.
10/11/2014 5:32:07 AM | SETI@home | Started upload of 28fe08ac.19218.183565.438086664195.12.103_0_0
10/11/2014 5:32:10 AM | SETI@home | Finished upload of 28fe08ac.19218.183565.438086664195.12.103_0_0
10/11/2014 5:32:12 AM | SETI@home | Sending scheduler request: To report completed tasks.
10/11/2014 5:32:12 AM | SETI@home | Reporting 1 completed tasks
10/11/2014 5:32:12 AM | SETI@home | Not requesting tasks: "no new tasks" requested via Manager
10/11/2014 5:32:14 AM | SETI@home | Scheduler request completed
10/11/2014 5:34:48 AM | LHC@home 1.0 | update requested by user
10/11/2014 5:34:49 AM | LHC@home 1.0 | Sending scheduler request: Requested by user.
10/11/2014 5:34:49 AM | LHC@home 1.0 | Requesting new tasks for CPU and NVIDIA
10/11/2014 5:34:52 AM | LHC@home 1.0 | Scheduler request completed: got 0 new tasks
10/11/2014 5:34:52 AM | LHC@home 1.0 | Server can't open log file (../log_boinc05/scheduler.log)
10/11/2014 5:36:10 AM | SETI@home | General prefs: from SETI@home (last modified 26-Sep-2014 11:42:12)
10/11/2014 5:36:10 AM | SETI@home | Computer location: home
10/11/2014 5:36:10 AM | SETI@home | General prefs: no separate prefs for home; using your defaults
10/11/2014 5:36:10 AM | | Reading preferences override file
10/11/2014 5:36:10 AM | | Preferences:
10/11/2014 5:36:10 AM | | max memory usage when active: 8285.51MB
10/11/2014 5:36:10 AM | | max memory usage when idle: 8285.51MB
10/11/2014 5:36:10 AM | | max disk usage: 100.00GB
10/11/2014 5:36:10 AM | | max download rate: 10240000 bytes/sec
10/11/2014 5:36:10 AM | | max upload rate: 10240000 bytes/sec
10/11/2014 5:36:10 AM | | (to change preferences, visit a project web site or select Preferences in the Manager)
10/11/2014 5:36:21 AM | LHC@home 1.0 | update requested by user
10/11/2014 5:36:22 AM | LHC@home 1.0 | Sending scheduler request: Requested by user.
10/11/2014 5:36:22 AM | LHC@home 1.0 | Requesting new tasks for CPU and NVIDIA
10/11/2014 5:36:25 AM | LHC@home 1.0 | Scheduler request completed: got 0 new tasks
10/11/2014 5:36:25 AM | LHC@home 1.0 | Server can't open log file (../log_boinc05/scheduler.log)
10/11/2014 5:36:55 AM | LHC@home 1.0 | update requested by user
10/11/2014 5:37:00 AM | LHC@home 1.0 | Sending scheduler request: Requested by user.
10/11/2014 5:37:00 AM | LHC@home 1.0 | Requesting new tasks for CPU and NVIDIA
10/11/2014 5:37:02 AM | LHC@home 1.0 | Scheduler request completed: got 0 new tasks
10/11/2014 5:37:02 AM | LHC@home 1.0 | Server can't open log file (../log_boinc05/scheduler.log)
10/11/2014 5:37:06 AM | LHC@home 1.0 | Resetting project
10/11/2014 5:37:06 AM | LHC@home 1.0 | Detaching from project
10/11/2014 5:37:31 AM | | Fetching configuration file from http://lhcathomeclassic.cern.ch/sixtrack/get_project_config.php
10/11/2014 5:39:09 AM | LHC@home 1.0 | Master file download succeeded
10/11/2014 5:39:14 AM | LHC@home 1.0 | Sending scheduler request: Project initialization.
10/11/2014 5:39:14 AM | LHC@home 1.0 | Requesting new tasks for CPU and NVIDIA
10/11/2014 5:39:16 AM | LHC@home 1.0 | Scheduler request completed: got 0 new tasks
10/11/2014 5:39:16 AM | LHC@home 1.0 | Server can't open log file (../log_boinc05/scheduler.log)
10/11/2014 5:39:46 AM | LHC@home 1.0 | update requested by user
10/11/2014 5:39:52 AM | LHC@home 1.0 | Sending scheduler request: Requested by user.
10/11/2014 5:39:52 AM | LHC@home 1.0 | Requesting new tasks for CPU and NVIDIA
10/11/2014 5:39:53 AM | LHC@home 1.0 | Scheduler request completed: got 0 new tasks
10/11/2014 5:39:53 AM | LHC@home 1.0 | Server can't open log file (../log_boinc05/scheduler.log)
What might be a fix for this problem? I'm running Seti and LHC at this time. Seti runs just fine at this moment. When I shut down last night all was well with the world of Boinc but this morning LHC has taken a hike to-----------!
Pick

ID: 26863 · Report as offensive     Reply Quote
Sunny129
Avatar

Send message
Joined: 12 Dec 05
Posts: 31
Credit: 9,709,398
RAC: 0
Message 26864 - Posted: 11 Oct 2014, 13:59:14 UTC - in response to Message 26863.  

^ well it looks like that failed SETI@Home upload was just a glitch b/c the file uploaded without a problem 2 minutes after the first attempt.

regarding LHC@Home, i'm relieved that i'm not the only one getting the "LHC@home 1.0 | Server can't open log file (../log_boinc05/scheduler.log)" message...i.e. glad its a server issue and not an issue with my particular host.
ID: 26864 · Report as offensive     Reply Quote
alvin
Avatar

Send message
Joined: 12 Mar 12
Posts: 128
Credit: 20,013,377
RAC: 0
Message 26866 - Posted: 11 Oct 2014, 14:21:01 UTC - in response to Message 26864.  

as it might be problem with File system any uploads could be stopped for not loosing results.
and any transfers too
I see my numbers aren't changed for a while which I assume is good sign of works to fix it
ID: 26866 · Report as offensive     Reply Quote
[AF>FAH-Addict.net]toTOW

Send message
Joined: 9 Oct 10
Posts: 77
Credit: 3,671,357
RAC: 0
Message 26873 - Posted: 12 Oct 2014, 10:00:37 UTC

I'm receiving the same error as you do when trying to request some work ...
ID: 26873 · Report as offensive     Reply Quote
Profile KWSN The Holy Hand Grenade!

Send message
Joined: 29 Jul 07
Posts: 4
Credit: 404,661
RAC: 0
Message 26874 - Posted: 12 Oct 2014, 11:44:10 UTC

"LHC@home 1.0 | Server can't open log file (../log_boinc05/scheduler.log)"


this (in other projects that I run) usually means that the log file has expanded to the point that it has run out of disk space... drain the log file and we should be back in business...
ID: 26874 · Report as offensive     Reply Quote
Forest T.

Send message
Joined: 25 Nov 05
Posts: 3
Credit: 3,814,966
RAC: 0
Message 26875 - Posted: 12 Oct 2014, 16:14:42 UTC - in response to Message 26874.  

I'm having the same problem. Can someone tell me where to find the scheduler.log file so I can drain it? I did a search in Dolphin and came up empty. Thanks.
ID: 26875 · Report as offensive     Reply Quote
Ano

Send message
Joined: 29 Nov 09
Posts: 42
Credit: 229,229
RAC: 0
Message 26876 - Posted: 12 Oct 2014, 17:33:01 UTC - in response to Message 26875.  

I think the message is just a log from the server explaining his error, so having the file on your side (client side) wouldn't help in this case.
Also having this message and not receiving work.
ID: 26876 · Report as offensive     Reply Quote
Forest T.

Send message
Joined: 25 Nov 05
Posts: 3
Credit: 3,814,966
RAC: 0
Message 26877 - Posted: 12 Oct 2014, 23:38:49 UTC - in response to Message 26876.  

Thanks Ano. Also, I now have 32 work units reporting 100% complete and UPLOADING, but they've been like that for hours. Rebooted and they still come up as UPLOADING. Is this going to be fixed so they finish uploading, or is there something we need to do on our end to get them to restart uploading?
ID: 26877 · Report as offensive     Reply Quote
alvin
Avatar

Send message
Joined: 12 Mar 12
Posts: 128
Credit: 20,013,377
RAC: 0
Message 26878 - Posted: 13 Oct 2014, 0:41:38 UTC - in response to Message 26877.  

Server and project stopped receiving jobs to not loose them.
So we just do other projects till they fixed AFS and everything related I think.
ID: 26878 · Report as offensive     Reply Quote
Robi

Send message
Joined: 2 Sep 04
Posts: 14
Credit: 178,532
RAC: 0
Message 26879 - Posted: 13 Oct 2014, 3:24:46 UTC

Project status claims to be OK (Running) on all levels, but still no upload. I guess in a few hours, when they open the doors at CERN and start their daily routines, they will see it and fix it. (it's 5:30 there, so it shouldn't take much longer...)
ID: 26879 · Report as offensive     Reply Quote
Profile Viking69
Avatar

Send message
Joined: 24 Jul 05
Posts: 56
Credit: 5,602,722
RAC: 4
Message 26883 - Posted: 13 Oct 2014, 14:28:30 UTC

I do not have any uploads waiting, but I am also not getting any new work. The server status screen does take a long time to open though. 117,000 + WU's prepared, but my agent logs state:
10/13/2014 6:59:51 AM | LHC@home 1.0 | Server can't open log file (../log_boinc05/scheduler.log)

Let's crunch for our future.
ID: 26883 · Report as offensive     Reply Quote
Yacob

Send message
Joined: 1 Dec 12
Posts: 11
Credit: 5,844,526
RAC: 0
Message 26884 - Posted: 13 Oct 2014, 16:59:46 UTC

I could get new tasks, but my old ones are still unable to be uploaded.
ID: 26884 · Report as offensive     Reply Quote
Richard Haselgrove

Send message
Joined: 27 Oct 07
Posts: 186
Credit: 3,297,640
RAC: 0
Message 26885 - Posted: 13 Oct 2014, 17:00:13 UTC

Scheduler contacts (requesting new work and potentially reporting completed work) are running smoothly:

13/10/2014 17:54:56 | LHC@home 1.0 | Sending scheduler request: Requested by user.
13/10/2014 17:54:56 | LHC@home 1.0 | Requesting new tasks for CPU
13/10/2014 17:54:56 | LHC@home 1.0 | [sched_op] CPU work request: 1875.28 seconds; 0.00 devices
13/10/2014 17:54:56 | LHC@home 1.0 | [sched_op] NVIDIA GPU work request: 0.00 seconds; 0.00 devices
13/10/2014 17:54:56 | LHC@home 1.0 | [sched_op] Intel GPU work request: 0.00 seconds; 0.00 devices
13/10/2014 17:54:58 | LHC@home 1.0 | Scheduler request completed: got 1 new tasks
13/10/2014 17:54:58 | LHC@home 1.0 | [sched_op] estimated total CPU task duration: 18441 seconds
13/10/2014 17:55:00 | LHC@home 1.0 | Started download of sd_HL_15_440_2.2_6D_cc_err_D2__13__s__62.31_60.32__6_8__6__60_1_sixvf_boinc3342.zip
13/10/2014 17:55:02 | LHC@home 1.0 | Finished download of sd_HL_15_440_2.2_6D_cc_err_D2__13__s__62.31_60.32__6_8__6__60_1_sixvf_boinc3342.zip

but it's the uploading of data from completed tasks which is failing:

13/10/2014 17:52:58 | LHC@home 1.0 | Started upload of sd_HL_15_440_2.2_6D_cc_err_IT__26__s__62.31_60.32__10_12__6__60_1_sixvf_boinc2137_1_0
13/10/2014 17:52:58 | LHC@home 1.0 | [http] [ID#1458] Info: About to connect() to lhcathomeclassic.cern.ch port 80 (#1619)
13/10/2014 17:52:58 | LHC@home 1.0 | [http] [ID#1458] Info: Trying 128.142.138.22...
13/10/2014 17:52:58 | LHC@home 1.0 | [http] [ID#1458] Info: Connected to lhcathomeclassic.cern.ch (128.142.138.22) port 80 (#1619)
13/10/2014 17:52:59 | LHC@home 1.0 | [http] [ID#1458] Received header from server: Date: Mon, 13 Oct 2014 16:53:00 GMT
13/10/2014 17:52:59 | LHC@home 1.0 | [http] [ID#1458] Received header from server: Connection: close
13/10/2014 17:52:59 | LHC@home 1.0 | [error] Error reported by file upload server: can't open log file '../log_boinc05/file_upload_handler.log' (errno: 9)
13/10/2014 17:52:59 | LHC@home 1.0 | Temporarily failed upload of sd_HL_15_440_2.2_6D_cc_err_IT__26__s__62.31_60.32__10_12__6__60_1_sixvf_boinc2137_1_0: transient upload error
ID: 26885 · Report as offensive     Reply Quote
Ano

Send message
Joined: 29 Nov 09
Posts: 42
Credit: 229,229
RAC: 0
Message 26886 - Posted: 13 Oct 2014, 17:18:54 UTC

It's funny how this error message is "high class".Instead of "transient", it could just have been "temporary" (according to the dictionary I just checked).
Well besides that, I'm glad I get tasks again, though the Tasks tab is quickly filling up ^^
ID: 26886 · Report as offensive     Reply Quote
Eric Mcintosh
Volunteer moderator
Project administrator
Project developer
Project tester
Project scientist

Send message
Joined: 12 Jul 11
Posts: 857
Credit: 1,619,050
RAC: 0
Message 26887 - Posted: 13 Oct 2014, 19:46:24 UTC

Think it is OK now (but we are still looking at AFS). Eric.
ID: 26887 · Report as offensive     Reply Quote
Profile Robert Pick

Send message
Joined: 1 Dec 05
Posts: 62
Credit: 11,398,274
RAC: 261
Message 26889 - Posted: 13 Oct 2014, 21:03:46 UTC - in response to Message 26887.  

I did get several WU today but after working several through none will upload and this seems to be the reason! 10/13/2014 2:02:06 PM | LHC@home 1.0 | [error] Error reported by file upload server: can't open log file '../log_boinc05/file_upload_handler.log' (errno: 9)
10/13/2014 2:02:06 PM | LHC@home 1.0 | [error] Error reported by file upload server: can't open log file '../log_boinc05/file_upload_handler.log' (errno: 9)
10/13/2014 2:02:06 PM | LHC@home 1.0 | Temporarily failed upload of sd_HL_15_440_2.2_6D_cc_err_Q5__7__s__62.31_60.32__6_8__6__40_1_sixvf_boinc161_0_0: transient upload error
10/13/2014 2:02:06 PM | LHC@home 1.0 | Backing off 05:00:16 on upload of sd_HL_15_440_2.2_6D_cc_err_Q5__7__s__62.31_60.32__6_8__6__40_1_sixvf_boinc161_0_0
10/13/2014 2:02:06 PM | LHC@home 1.0 | Temporarily failed upload of sd_HL_15_440_2.2_6D_cc_err_Q5__7__s__62.31_60.32__6_8__6__30_1_sixvf_boinc159_0_0: transient upload error
10/13/2014 2:02:06 PM | LHC@home 1.0 | Backing off 03:29:42 on upload of sd_HL_15_440_2.2_6D_cc_err_Q5__7__s__62.31_60.32__6_8__6__30_1_sixvf_boinc159_0_0

ID: 26889 · Report as offensive     Reply Quote
Richard Haselgrove

Send message
Joined: 27 Oct 07
Posts: 186
Credit: 3,297,640
RAC: 0
Message 26890 - Posted: 13 Oct 2014, 21:18:06 UTC - in response to Message 26887.  

Think it is OK now (but we are still looking at AFS). Eric.

Sorry, no. Still getting

13/10/2014 22:16:06 | LHC@home 1.0 | [error] Error reported by file upload server: can't open log file '../log_boinc05/file_upload_handler.log' (errno: 9)
13/10/2014 22:16:06 | LHC@home 1.0 | Temporarily failed upload of sd_HL_15_440_2.2_6D_cc_err_Q5__14__s__62.31_60.32__8_10__6__85_1_sixvf_boinc3727_1_0: transient upload error

when retesting after your message.
ID: 26890 · Report as offensive     Reply Quote
Yacob

Send message
Joined: 1 Dec 12
Posts: 11
Credit: 5,844,526
RAC: 0
Message 26891 - Posted: 13 Oct 2014, 21:50:04 UTC

Still the problem is there.

ID: 26891 · Report as offensive     Reply Quote
Forest T.

Send message
Joined: 25 Nov 05
Posts: 3
Credit: 3,814,966
RAC: 0
Message 26892 - Posted: 13 Oct 2014, 23:32:28 UTC - in response to Message 26891.  

Yes still there. WU not completing uploading. I now have WU's that failed to upload and are now past their due dates. I hope everyone's work isn't lost because of this.
ID: 26892 · Report as offensive     Reply Quote
1 · 2 · Next

Message boards : News : CERN AFS problems


©2024 CERN