Validation of simulation
Summary
To derive the sensitivity analysis (as ultimate goal), a full proton, gamma, and diffuse gamma is needed at a certain threshold. This threshold is determine via the safe threshold determination method, which aims to find at which threshold the NSB rate scan at 2 times the nominal NSB value intersects the 1.5 times the proton rate scan. Therefore, a production of the NSB rate scans and proton rate scan is needed.
This is the main purpose of sim-runner. However, a validation of the simulations. By validation in the sense of having enough statistics to be able to obtain reliable results
This is currently done in the following steps :
- NSB rate scan production :
- Simulate the NSB rate scans via sim-runnner. These runs should provide at their end a log files for each threshold simulated.
- Use the
rate_scan_logfile_reader.sh
file in/work/se1/unige/software/prog/simtel_chain/validation/safe_threshold
to extract the relevant data of the log files and construct the a file containing the threshold and rate information
- Proton rate scan production :
- Simulate the proton rate scans via sim-runner
- These runs take N different corsika files per threshold (N currently at 10), outputting N simtel files, each one yields a log file which are not interested to get the rates
- Conversion of the simtel files to npz file must be done via
read_simtel_all.py
script in/work/se1/unige/software/prog/simtel_chain/validation/convert_simtel_to_npz
, each file yields a log file containing relevant information - To get the rates the script
effective_area.py
must be run, this will yield also log files as well as plots of the trigger probability histograms (to verify the statistics), the effective area computation per energy bin and its fit, and the total rate. - The conversion to npz files as well as the rate computation scripts to run sge jobs are included, they utilize
read_simtel_all.py
andeffective_area.py
- Use the
rate_scan_logfile_reader.sh
file in/work/se1/unige/software/prog/simtel_chain/validation/safe_threshold
to extract the relevant data of the log files (noz log output and effective area log output) and construct the a file containing the threshold and rate information
- Safe Threshold determination :
- Steps 1 and 3 yield 2 files containing the rates for NSBx2 and protonx1 (not 1.5)
- An additional script to compute and make the safe threshold plots must be provided (it is implemented but it needs to be integrated to the validation chain, ask @ldmedina for it). This scripts introduce the 1.5 factor in the proton rate!
- Determine the safe threshold value running this script and input it as the threshold for the proton, gamma, diffuse gamma production
- Proton, Gamma, Diffuse Gamma production :
Once this productions are done via sim-runner, for each one of them we will repeat the steps of validation of the statistics and rates done in the proton rate scans, in other words :
- Conversion of the simtel files to npz file must be done via
read_simtel_all.py
script in/work/se1/unige/software/prog/simtel_chain/validation/convert_simtel_to_npz
, each file yields a log file containing relevant information - To get the rates the script
effective_area.py
must be run, this will yield also log files as well as plots of the trigger probability histograms (to verify the statistics), the effective area computation per energy bin and its fit, and the total rate. - The conversion to npz files as well as the rate computation scripts to run sge jobs are included, they utilize
read_simtel_all.py
andeffective_area.py
- Verify the well fulfillment of the histograms to validate the simulation
- Do your analysis
What is the expected correct behavior?
We would like to implement the validation chain accordingly to the sim-runner framework, not only for commodity but also because it's key to obtain a simulation production