Message boards :
ATLAS application :
ATLAS vbox and native 3.01
Message board moderation
Author | Message |
---|---|
Send message Joined: 13 May 14 Posts: 387 Credit: 15,314,184 RAC: 0 |
Hi all, We have just made release 3.01 for vbox and native apps in preparation for running updated simulation software. Until now all ATLAS tasks have been simulations of the ATLAS detector during "Run 2" of the LHC that ran from 2015 to 2018. Even 5 years later ATLAS physicists are still analysing the data from that period and require new simulations to be done. However last year "Run 3" of the LHC started and it will last until the end of 2025. Soon we will switch the ATLAS tasks here to Run 3 simulations, where many things have changed in the detector and the software used to simulate and analyse data. Version 3 of the app allows us to run both Run 2 and Run 3 simulations in the same version during the transition period, however this does make the image for vbox a bit larger (4.4GB, or 1.8GB compressed). The Linux native app is the same as before with one very minor change to set an environment variable required by the new software. The main benefit of the software used for Run 3 simulations is that it uses far less memory, less than 3GB for an 8-core task. At the moment we are not yet submitting Run 3 tasks here so everything should work exactly the same as before, but we'll start doing this soon and let you know. We have been doing some extensive testing on the LHC-dev project and there are still a couple of things to fix. |
Send message Joined: 2 May 07 Posts: 2242 Credit: 173,899,580 RAC: 2,816 |
Have in the last two hours 100 Tasks with download Error. <core_client_version>7.20.2</core_client_version> <![CDATA[ <message> app_version download error: couldn't get input files: <file_xfer_error> <file_name>ATLAS_vbox_3.01_image.vdi</file_name> <error_code>-224 (permanent HTTP error)</error_code> <error_message>permanent HTTP error</error_message> </file_xfer_error> </message> ]]> |
Send message Joined: 13 May 14 Posts: 387 Credit: 15,314,184 RAC: 0 |
Sorry, the file had the wrong permissions on the server. It should work now. |
Send message Joined: 15 Jun 08 Posts: 2530 Credit: 253,722,201 RAC: 51,175 |
vdi as well as vdi.gz are now allowed to be downloaded (tested with wget). |
Send message Joined: 2 Sep 04 Posts: 455 Credit: 201,147,322 RAC: 31,183 |
I have got a new 3.01 WU, but it seems still to contain this big 1,2 GB file. What will happen with it, as far as I understood it is useless for BOINC-Crunchers? It would be a waste of resources, electricity, time, bandwith and lifetime of all SSDs if it is still downloaded even it isn't needed. Supporting BOINC, a great concept ! |
Send message Joined: 15 Jun 08 Posts: 2530 Credit: 253,722,201 RAC: 51,175 |
as far as I understood it is useless for BOINC-Crunchers? I must have missed that. Where did you get that idea from? |
Send message Joined: 2 Sep 04 Posts: 455 Credit: 201,147,322 RAC: 31,183 |
Where did you get that idea from?https://lhcathome.cern.ch/lhcathome/forum_thread.php?id=5976&postid=47891#47891 Supporting BOINC, a great concept ! |
Send message Joined: 15 Jun 08 Posts: 2530 Credit: 253,722,201 RAC: 51,175 |
I agree, those huge EVNT files are annoying and shouldn't have been sent to BOINC for a couple of reasons. But I still can't find any words that claim they are useless for science. |
Send message Joined: 2 May 07 Posts: 2242 Credit: 173,899,580 RAC: 2,816 |
Sorry, the file had the wrong permissions on the server. It should work now. Download now ok, Important is the work we do with this Tasks and not the size of the input file. |
Send message Joined: 2 Sep 04 Posts: 455 Credit: 201,147,322 RAC: 31,183 |
|
Send message Joined: 2 Sep 04 Posts: 455 Credit: 201,147,322 RAC: 31,183 |
Important is the work we do with this TasksShure and not the size of the input file. If I can do the same science with 130 / 200 / 300 MB Downloads like before, why should we now download 1,2 GB for the same amount of science? Maybe it is okay for you, fine, but for me it is a wastefulness Supporting BOINC, a great concept ! |
Send message Joined: 2 May 07 Posts: 2242 Credit: 173,899,580 RAC: 2,816 |
Therefore we need a comment from science group generating the input. As long as the Errors are smaller than the successful Tasks, no problem so long. |
Send message Joined: 13 May 14 Posts: 387 Credit: 15,314,184 RAC: 0 |
The science output is the same regardless of the size of input file. The unfortunate large input files are the result of accidentally creating files with too many events instead of splitting them into smaller chunks. But they are not a waste or useless for science since the processing and output is the same. |
Send message Joined: 15 Jun 08 Posts: 2530 Credit: 253,722,201 RAC: 51,175 |
Thanks. Just wondering why this is mentioned in (all?) log.EVNTtoHITS: maxEvents = 200 <edit> How many of those big dogs are left? </edit> |
Send message Joined: 13 May 14 Posts: 387 Credit: 15,314,184 RAC: 0 |
Thanks. This is because each task is configured to simulate 200 events, even though there are more events than that in the input EVNT file. The EVNT file may contain 10000 events for example, so this would be split into 50 tasks of 200 events and each of those 50 tasks would download the same file.
I think this may be all of the remaining tasks for Run 2 simulation, which is why I would like to get the Run 3 simulation going as soon as possible. |
Send message Joined: 2 May 07 Posts: 2242 Credit: 173,899,580 RAC: 2,816 |
Is it a problem to do 500 instead of 200 events for this tasks, or have this input file not the data therefore? |
Send message Joined: 17 Sep 04 Posts: 105 Credit: 32,824,853 RAC: 705 |
I agree with this. Thanks. Regards, Bob P. |
Send message Joined: 15 Jun 08 Posts: 2530 Credit: 253,722,201 RAC: 51,175 |
As far as I understand the task generation process this is not possible for tasks that are already in the queue. At least not without time intensive and error-prone manual intervention. |
Send message Joined: 14 Jan 10 Posts: 1417 Credit: 9,441,018 RAC: 1,047 |
I ran a few tasks with the new image-vdi and exe and had setup the VM to run with 4096 MB RAM and 8 threads. Not much swapping is used. The ALT-F2 monitoring seems to work OK here. Not stuck like on -dev. |
Send message Joined: 2 May 07 Posts: 2242 Credit: 173,899,580 RAC: 2,816 |
+1 |
©2024 CERN