LHC@home

This is our current To Do List:
* Database backup and restore
* Server backup
* Get test system working

* Find and compile and test:
- Transitioner
- Validator
- Assimilator
- File deleter

* Ensure e-mail is working properly
* Test migrate
* Real migration
* Solve any other problems that come up!

Cheers,
Neasan and Alex.

This is our current To Do List:
[snip]

* Find and compile and test:
- Transitioner
- Validator
- Assimilator
- File deleter

[snip]

Find???
Does this mean they have been lost on their way from Geneva to Oxford?
Or were they rejected by the english customs office?

---

Grüße vom Sänger

This is our current To Do List:
[snip]

* Find and compile and test:
- Transitioner
- Validator
- Assimilator
- File deleter

[snip]

Find???
Does this mean they have been lost on their way from Geneva to Oxford?
Or were they rejected by the english customs office?

Most/all of those programs had custom versions at this project, the file deleter is probably the only one even close to the standard version. So the source additions must be found and then figured out. And finaly updated to work with the current BOINC standard code.

---

BOINC WIKI

BOINCing since 2002/12/8

Oxford?

Where exactly do you think Queen Mary, University of London is?

This is our current To Do List:
* Database backup and restore
* Server backup
* Get test system working

* Find and compile and test:
- Transitioner
- Validator
- Assimilator
- File deleter

* Ensure e-mail is working properly
* Test migrate
* Real migration
* Solve any other problems that come up!

Cheers,
Neasan and Alex.

Hmm ok..

So that's a pretty large 'To-Do' list.

What's been done already? Anything?

---

I would suggest that you need to identify and cure the cause of the proliferation of hostids before you do the real migrate, and in addition to prune out the millions of unwanted hosts from the db before the real migrate, perhaps only migrating those hostids that have ever submitted work.

It would be possible to transfer this bug across and then fix it later, but, in my opinion, that would cause you more difficulties in the long run than solving the issue first.

Having a wildly excessive number of hosts in the db certainly skews any optimisations you may try to test, and may well cause other bugs that would not arise with a plausible number of hosts.

One bug which may (or may not) be a knock on from the mega-ghost-host issue is the stats export issue. Stats export is not going to be high up the list of practical priorities, so I don't anticipate you will spend a lot of time on stats export itself.

On the other hand stats come high on the personal priotirites of many crunchers and if a solution to that came easily out your ghostbusting of the mega-hosts then many crunchers would be delighted, and there would be a corresponding saving in winges on these boards...

Best regards,
River~~

---

Out of curiosity: how big was the database dump that you had to transfer to UK? Considering you have 5739543 entries on host table...

---

This is our current To Do List:
* Database backup and restore
* Server backup
* Get test system working

* Find and compile and test:
- Transitioner
- Validator
- Assimilator
- File deleter

* Ensure e-mail is working properly
* Test migrate
* Real migration
* Solve any other problems that come up!

Cheers,
Neasan and Alex.

Hmm ok..

So that's a pretty large 'To-Do' list.

What's been done already? Anything?

I will refer you to this post:
http://lhcathome.cern.ch/forum_thread.php?id=2406&nowrap=true#16298

it will be finished in a month of Sundays ;)
pls, transfer the project to Poland, will be similarly but cheaper :D

This is our current To Do List:
* Database backup and restore
* Server backup
* Get test system working

* Find and compile and test:
- Transitioner
- Validator
- Assimilator
- File deleter

* Ensure e-mail is working properly
* Test migrate
* Real migration
* Solve any other problems that come up!

Cheers,
Neasan and Alex.

Hmm ok..

So that's a pretty large 'To-Do' list.

What's been done already? Anything?

I will refer you to this post:
http://lhcathome.cern.ch/forum_thread.php?id=2406&nowrap=true#16298

---

[...]prune out the millions of unwanted hosts from the db before the real migrate, perhaps only migrating those hostids that have ever submitted work.

I like that idea. For those hosts that never submitted work but still want to participate, they can manually add "themselves" back to the project post-migration.

---

_______

"Three quarks for Muster Mark!"
. . . . . . . - James Joyce, Finnegans Wake . . . .

TWDYSTI* Friday and we have an update (also on front page):

Yes it is going slower than expected. It is a unique installation of BOINC (very unique) but it is moving ahead and we give you our current To Do List:
* Initial Database Migration
* Get test system working with database
* Ensure e-mail is working properly(trickier than it sounds)
* Test migrate again
* Real migration
* Sort out job submission over AFS by SixTrack User

This is going ahead at a pace to make sure that everything is working at every step of the way as opposed to trying to sort it out post a hasty install. So in a few weeks time the problems of yesteryear will seem like a bad dream that never happened and we'll all be happily crunching away**.
Later days,
Neasan and Alex.



*=Thank Whatever Deity You Subscribe To It's
**=Not a guarantee, the value of investments may rise or fall, terms and conditions apply.

[...]prune out the millions of unwanted hosts from the db before the real migrate, perhaps only migrating those hostids that have ever submitted work.

I like that idea. For those hosts that never submitted work but still want to participate, they can manually add "themselves" back to the project post-migration.

I joined before Christmas, leave my laptop running and ready for any LHC work but haven't received my first WU yet. System regulary checks now, while I run other BOINC projects.
I'd hate to be deleted after patiently waiting and have to wait for some sort of post-migration period to be over before I can re-apply. Maybe there needs to be a list of ID's that are still active because within the last x months either they submitted WU's or they have been polling for WU's.

[...]prune out the millions of unwanted hosts from the db before the real migrate, perhaps only migrating those hostids that have ever submitted work.

I like that idea. For those hosts that never submitted work but still want to participate, they can manually add "themselves" back to the project post-migration.

I joined before Christmas, leave my laptop running and ready for any LHC work but haven't received my first WU yet. System regulary checks now, while I run other BOINC projects.
I'd hate to be deleted after patiently waiting and have to wait for some sort of post-migration period to be over before I can re-apply. Maybe there needs to be a list of ID's that are still active because within the last x months either they submitted WU's or they have been polling for WU's.

Just to clarify:

There are a huge number of ghost hosts in the system, due to a bug that makes / made some hosts forget their identity. The proposal is aimed at the most straightforward way of pruning them out of the database to make it more manageable.

The deletion of your host, if you have not got any work by then would simply meant that next time your tired for work, the host would be re-initialised. This would happen on the very next connection.

The date shown for that computer joining the project would be the date it was given the new identity, but you would still have your original joining date shown on the forum boards nect to every post.

It is not a perfect solution, but in my opinion offers the most practical balance between taking up the admins time and bringing the database back to something manageable.

R~~

[...]prune out the millions of unwanted hosts from the db before the real migrate, perhaps only migrating those hostids that have ever submitted work.

I like that idea. For those hosts that never submitted work but still want to participate, they can manually add "themselves" back to the project post-migration.

I joined before Christmas, leave my laptop running and ready for any LHC work but haven't received my first WU yet. System regulary checks now, while I run other BOINC projects.
I'd hate to be deleted after patiently waiting and have to wait for some sort of post-migration period to be over before I can re-apply. Maybe there needs to be a list of ID's that are still active because within the last x months either they submitted WU's or they have been polling for WU's.

Just to clarify:

There are a huge number of ghost hosts in the system, due to a bug that makes / made some hosts forget their identity. The proposal is aimed at the most straightforward way of pruning them out of the database to make it more manageable.

The deletion of your host, if you have not got any work by then would simply meant that next time your tired for work, the host would be re-initialised. This would happen on the very next connection.

The date shown for that computer joining the project would be the date it was given the new identity, but you would still have your original joining date shown on the forum boards nect to every post.

It is not a perfect solution, but in my opinion offers the most practical balance between taking up the admins time and bringing the database back to something manageable.

R~~

Too bad about that ghost hosts programming error - costly if it clogs up the database.
Solution sounds good to me - I don't care that much about joining dates, just as long as I get my first WU and can start crunching it.
Thanks for the update.

[...]prune out the millions of unwanted hosts from the db before the real migrate, perhaps only migrating those hostids that have ever submitted work.

Need to do some searching to find the info again, but this ghosting is showing up in other projects. At least one 'got rid of the ghosts' by running an automated merge based on simple criteria

All hosts with same owner, same machine specs and have no overlapping dates were considered one machine and merged.

This would effectively rid the system of almost all ghosts. Pruning would only eliminate the ghosts that did no work. Merging then pruning systems that had never received work would get few more since there will be cases where the machines reported specs changed.

---

Any chance of another update?

---

[...]prune out the millions of unwanted hosts from the db before the real migrate, perhaps only migrating those hostids that have ever submitted work.

I like that idea. For those hosts that never submitted work but still want to participate, they can manually add "themselves" back to the project post-migration.

I joined before Christmas, leave my laptop running and ready for any LHC work but haven't received my first WU yet. System regulary checks now, while I run other BOINC projects.
I'd hate to be deleted after patiently waiting and have to wait for some sort of post-migration period to be over before I can re-apply. Maybe there needs to be a list of ID's that are still active because within the last x months either they submitted WU's or they have been polling for WU's.

Just to clarify:

There are a huge number of ghost hosts in the system, due to a bug that makes / made some hosts forget their identity. The proposal is aimed at the most straightforward way of pruning them out of the database to make it more manageable.

The deletion of your host, if you have not got any work by then would simply meant that next time your tired for work, the host would be re-initialised. This would happen on the very next connection.

The date shown for that computer joining the project would be the date it was given the new identity, but you would still have your original joining date shown on the forum boards nect to every post.

It is not a perfect solution, but in my opinion offers the most practical balance between taking up the admins time and bringing the database back to something manageable.

R~~

I have a slightly different solution that may generate a smaller table. Force a merge between all hosts that are normally allowed to merge that have the same name. This will get rid of more hosts as some of those host will still have WUs attached. Then auto delete any hosts that can be deleted that have not reported in the last 30 days or so that have no WUs attached. This should generate a fairly small table of hosts.

---

BOINC WIKI

What about people who have attached, zero credit, zero rac for the simple fact they couldn`t get any work.

---

I would assume that Bird-Dog's concerns would be taken care of by examing the last time individuals with zero credit and zero RAC polled the site in an effort to obtain work. Anything older than some set point could be discarded

---

If I've lived this long, I've gotta be that old

This discussion on purging hosts from the database is valuable but a bit premature as long as there is still the old software at work...

Time for an update from the proj admins then? It's been another 2 weeks ...