Message boards : News : Good news for the CMS@Home application
Message board moderation

To post messages, you must log in.

AuthorMessage
ivan
Volunteer moderator
Project tester
Volunteer developer
Volunteer tester
Project scientist
Avatar

Send message
Joined: 29 Aug 05
Posts: 1061
Credit: 7,737,455
RAC: 298
Message 28698 - Posted: 27 Jan 2017, 20:59:36 UTC

This afternoon we demonstrated the final link in the chain of producing Monte Carlo data for CMS using this project (and the -dev project too, of course), namely the transfer of result files from the temporary Data Bridge storage to a CMS Tier 2 site's storage element (SE). To summarise, the steps are:

o Creating a configuration script defining the process(es) to be simulated
o Submitting a batch of jobs of duration and result-file size suitable for running by volunteers
o Having those jobs picked up by volunteers running BOINC and the CMS@Home application, and the result files returned to the Data Bridge
o Running "merge" jobs on a small cluster at CERN to collect the smaller files into larger files (~2.2 GB) -- this step has to be done at CERN as most volunteers will not have the bandwidth (or data plan!) to handle the data volumes required. This step also serves to a large extent as the verification step required to satisfy CMS of the result files' integrity.
o Transferring the merged files into the Grid environment where they are then readily available to CMS researchers around the world

Thanks, everybody. From here on it gets more political, but we've been garnering support as the project progressed. We now need to move into a more "production" environment and convince central powers-that-be to take over the responsibility of submitting suitable workflows and collecting the results. You will still see some changes in the future, especially as we bring some of the more-advanced features across here from the -dev project.
ID: 28698 · Report as offensive     Reply Quote
computezrmle
Volunteer moderator
Volunteer developer
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 15 Jun 08
Posts: 2541
Credit: 254,608,838
RAC: 34,609
Message 28730 - Posted: 30 Jan 2017, 13:46:36 UTC

Are there any consequences for normal volunteers, e.g.:
- new ports to be opened in the firewall
- new data bridge
- how large are the downloads/uploads
- etc. ...
ID: 28730 · Report as offensive     Reply Quote
ivan
Volunteer moderator
Project tester
Volunteer developer
Volunteer tester
Project scientist
Avatar

Send message
Joined: 29 Aug 05
Posts: 1061
Credit: 7,737,455
RAC: 298
Message 28732 - Posted: 30 Jan 2017, 16:31:24 UTC - in response to Message 28730.  

Are there any consequences for normal volunteers, e.g.:
- new ports to be opened in the firewall
- new data bridge
- how large are the downloads/uploads
- etc. ...

At the moment I don't expect firewall or data bridge changes. We would have to "educate" the production team on the range of suitable output file sizes and job CPU duration.
ID: 28732 · Report as offensive     Reply Quote

Message boards : News : Good news for the CMS@Home application


©2024 CERN