Skip to content

add event count and update the executor

Jie Xiao requested to merge 35-count-the-number-of-events-in-a-dataset into master

Closes #35 (closed)

Add event count in higgs_dna/workflows/base.py and higgs_dna/workflows/nevent.py (in case people just want to count the events). People can use --count option to name a json file that documents the number of events.

I also update the executor definitions in higgs_dna/utils/runner_utils.py, making them are able to accept parquet format.

I've tested futures, iterative, and dask/slurm executors with this branch. E.g.,

python ../scripts/run_analysis.py --json-analysis test_v11.json --executor futures -j 3 --skipCQR --count evecount.json --no-trigger --dump output
python ../scripts/run_analysis.py --json-analysis test_v11.json --executor dask/slurm -m 10GB -s 4 -j 1 --skipCQR --count evecount2.json --no-trigger --dump output

Merge request reports