Message boards : CMS Application : WMAgent Jobs
Message board moderation

To post messages, you must log in.

AuthorMessage
Profile Laurence
Project administrator
Project developer

Send message
Joined: 20 Jun 14
Posts: 336
Credit: 237,918
RAC: 0
Message 28453 - Posted: 12 Jan 2017, 9:11:12 UTC

New CMS tasks will from now on receive jobs from the WMAgent rather than CRAB. This means that a different system is being used to inject the jobs. It is a big step towards integration with CMS computing operations.
ID: 28453 · Report as offensive     Reply Quote
Crystal Pellet
Volunteer moderator
Volunteer tester

Send message
Joined: 14 Jan 10
Posts: 955
Credit: 6,326,103
RAC: 1,819
Message 28455 - Posted: 12 Jan 2017, 14:20:32 UTC

How many events did Ivan put in 1 wmagent job?
ID: 28455 · Report as offensive     Reply Quote
ivan
Volunteer moderator
Project tester
Volunteer developer
Volunteer tester
Project scientist
Avatar

Send message
Joined: 29 Aug 05
Posts: 672
Credit: 5,334,629
RAC: 10,080
Message 28457 - Posted: 12 Jan 2017, 15:36:05 UTC - in response to Message 28455.  

How many events did Ivan put in 1 wmagent job?

There are 4,000 events in these jobs, and the output files are larger than the CRAB3 jobs, so watch your bandwidth usage. I haven't had a chance to tune these jobs yet, things are moving pretty fast on the ground. I want to find another useful workflow, these are an older script I was given to try to get WMAgent working. Now it is, we are moving closer to full CMS production.
ID: 28457 · Report as offensive     Reply Quote
Crystal Pellet
Volunteer moderator
Volunteer tester

Send message
Joined: 14 Jan 10
Posts: 955
Credit: 6,326,103
RAC: 1,819
Message 28459 - Posted: 12 Jan 2017, 15:59:04 UTC - in response to Message 28457.  

There are 4,000 events in these jobs, ...

Thanks Ivan!

I got the answer myself meanwhile.

First job ended with:

Begin processing the 4000th record. Run 1, Event 18612000, LumiSection 74448 at 12-Jan-2017 16:32:20.772 CET
.
.
.
MemoryReport> Peak virtual size 938.547 Mbytes
Key events increasing vsize:
[13] run: 1 lumi: 74433 event: 18608013 vsize = 816.039 deltaVsize = 0.25 rss = 552.418 delta = 0.117188
[5] run: 1 lumi: 74433 event: 18608005 vsize = 815.789 deltaVsize = 1.25 rss = 552.301 delta = 1.28125
[74] run: 1 lumi: 74433 event: 18608074 vsize = 848.438 deltaVsize = 31.8984 rss = 584.578 delta = 32.2773
[282] run: 1 lumi: 74434 event: 18608282 vsize = 903.254 deltaVsize = 29.3164 rss = 641.176 delta = 56.5977
[251] run: 1 lumi: 74434 event: 18608251 vsize = 865.938 deltaVsize = 8.25 rss = 603.043 delta = 18.4648
[3919] run: 1 lumi: 74448 event: 18611919 vsize = 937.797 deltaVsize = -0.75 rss = 641.961 delta = -2.55078
[3883] run: 1 lumi: 74448 event: 18611883 vsize = 938.297 deltaVsize = -0.25 rss = 642.379 delta = -2.13281
[3827] run: 1 lumi: 74448 event: 18611827 vsize = 938.547 deltaVsize = 1.25 rss = 644.512 delta = -2.92188
TimeReport> Time report complete in 13849 seconds
Time Summary:
- Min event: 0.00384307
- Max event: 1045.72
- Avg event: 3.46225
- Total job: 13849
CPU Summary:
- Min event: 0
- Max event: 650.53
- Avg event: 3.11171
- Total job: 12604
- Total event: 12446.8
ID: 28459 · Report as offensive     Reply Quote
ivan
Volunteer moderator
Project tester
Volunteer developer
Volunteer tester
Project scientist
Avatar

Send message
Joined: 29 Aug 05
Posts: 672
Credit: 5,334,629
RAC: 10,080
Message 28462 - Posted: 12 Jan 2017, 17:00:29 UTC - in response to Message 28459.  

Yes, I haven't massaged the output of these jobs yet, because I haven't worked out exactly where their config file is!
ID: 28462 · Report as offensive     Reply Quote
ivan
Volunteer moderator
Project tester
Volunteer developer
Volunteer tester
Project scientist
Avatar

Send message
Joined: 29 Aug 05
Posts: 672
Credit: 5,334,629
RAC: 10,080
Message 28477 - Posted: 13 Jan 2017, 20:07:40 UTC - in response to Message 28462.  

Yes, I haven't massaged the output of these jobs yet, because I haven't worked out exactly where their config file is!

I've now found that the config files live on a CouchDB database instance and are referred to by a random-looking hexadecimal string as a key. Still trying to find out how to access it and whether it's possible to modify it...
ID: 28477 · Report as offensive     Reply Quote

Message boards : CMS Application : WMAgent Jobs


©2020 CERN