All issues for this milestone are closed. You may close the milestone now.
Expired
Milestone Oct 31, 2023–Feb 29, 2024

End of Year Review 2023

This milestone is for keeping track of plans and developments discussed in the restarted biweekly AvailSim4 meetings.

  • Ongoing activities: finalize and test existing merge requests o Availsim4 on CVMFS o Random Seed Handling  now they are random by default o Input sheets are made robust against missing inputs (to account for all the features that a beginner does not need to use)
  • Improve HTCondor situation o Simplify experience when running on HTCondor [Milosz, Jack, Lukas, Cedric & Daniel – couple of days]  Get Openstack VM with GUI working on AFS (to avoid syncing back and forth between eos and afs) – done • Explore (Lukas makes issue) o Htcondor  works as expected now o Local execution o VScode or FastX  we are happy with VScode, we did not try FastX yet  FastX may allow running online Excel (allowing great extension) - mp-linux-ts-availability-01.cern.ch:3300  Limitation of VScode is filtering results Excel files  Need to check whether modified Excel files may cause hiccups with AS4  Be part of https://mp-computing.docs.cern.ch/projects/ and follow workflow there – • Mostly links to existing stuff (Lukas makes issue) • Availsim4 intro o How to run on • Working on cluster intro  Should we configure the virtual machine to run out of the box/terminal  Have a meeting with Cedric to discuss possibilities - done  Check use-cases for DagMan and possible benefits (or alternatives for handling sequences of jobs on the cluser) – • DagMan might help re-running job that did not finish? Maybe not • Maybe there is something else – • Can just check if results is missing in output and where missing resubmit submission files o Reduced output file sizes [Milosz, Lukas; in case it impacts availsim4core, proposal should be checked with SW team within a ~ month]  Wait for merge requests and CVMFS (try to have files pushed there each release) done (Milosz makes issue)  Option A: allow deactivating any sheet in results.xlsx  Option B: use root-cause-analysis to select which events should be tracked  Option C: allow to inject filtering class/function at the level of pandas dataframes that availsim4 generates before it gets exported to xlsx.  Option D: have a script as part of the htcondor submission that postprocesses what availsim4 outputs to reduce the file size
  • Getting started with availasim4 document [Jack mainly, Lukas helps] o First keep exploring o Will try to have it in January for newcomer (Jack makes issue)
  • Jack will think of a script for simplifying the FCC-ee use case submission and post-processing (Jack makes issue) o Done but may need to be simplified in view of new openstack machine o May provide re-running of jobs o Done and available https://gitlab.cern.ch/availsim4/fcc-ee-sensitivity-availability-study/-/tree/streamlined2?ref_type=heads
  • Simpler/faster input files o Have possibility to substitute simulation.xlsx by dictionary o (if needed) Allow “for loop” based sensitivity analysis without storing and loading files
  • The great extension (Milosz makes issue) o Currently needs to be locally installed o Check how to “ship” with AS4 in future
  • Open issues for each of them in gitlab – Milosz sorts how to assign to “AvailSim4 review” category
  • Work items 4
  • Merge requests 0
  • Participants 2
  • Labels 4
Loading
Loading
Loading
Loading
100% complete
100%
Start date
Oct 31, 2023
Oct 31 2023
-
Feb 29 2024
Due date
Feb 29, 2024 (Past due)
4
Work items 4
Open: 0 Closed: 4
None
Total weight
None
2
Merge requests 2
Open: 0 Closed: 0 Merged: 2
0
Releases
None
Reference: availsim4%"End of Year Review 2023"