Annoyance: tests depend on several packages

Several DaVinci tests assume the presence of a local checkout of another package. This always forces users to check out DaVinciExamples, DaVinciTests and Phys/DaVinci. I understand this is a consequence of the removal of $DAVINCIROOT and alike. If no solution is found at framework level, it would be good to avoid such cross-dependencies.

added DPA-WP3 label

assigned to @pkoppenb and @dfazzini

changed title from Annoyance: tests depend on several pacakges to Annoyance: tests depend on several packages

@pkoppenb, yes this is due to the fact that we are not using anymore the environment variables $*ROOT. I don't know if there is any other method that we can use but probably the easier is, as you said, avoiding any cross-dependencies.
These dependencies are mostly due to the fact that many options files, stored in Phys/DaVinci/options/ and DaVinciExamples/python/DaVinciExamples/, are shared between different packages in order to avoid code duplication. For example, the same options files used to test some DaVinci features can also work as working examples for the users: i.e.example-tupling-basic-run-mc.py, which is used to test the correct tupling with FunTuple, is a good example to make your own tupling algorithm.
What do you think is the best solution to fix this annoyance? Should we split the tests between DaVinciTests and DaVinciExamples packages or move all the tests in DaVinciTests, duplicating the option files in the two packages?

added needs computing input label

Duplication is to be avoided by all means. Which options files are shared among tests?

Maybe @clemenci has an idea how to best address this.

I believe the best way to share Python files is by using PYTHONPATH. See, for example, https://gitlab.cern.ch/lhcb/LHCb/-/blob/master/Det/DetDescChecks/options/LoadDDDB.py

My main argument is that if you need to share stuff you should share it and not relying on the fact you can locate the sources of another project/packages.

Hi @clemenci, thanks for the prompt reply. Yes, this is indeed a good suggestion for the python files, but we have to deal also with .json/.yaml files for setting the values of the DaVinci properties. For example, we used the database in Phys/DaVinci/options/DaVinciDB-Example.yaml for defining the data properties of all the tests/examples in DaVinciTests and DaVinciExamples.

Just an idea, but could one use Path to get the location of files upon import of the relevant package? Something along what I do at https://github.com/scikit-hep/decaylanguage/blob/master/tests/dec/test_dec.py but of course with a few adaptations given that here all is in the same package. Dunno, did not think it through but it might work neatly since upon import you can know from Path the location of the files hence also the other non-Python files.

That's one very valid option, if we do not zip python directories (it's not done with the new CMake configuration, but I'd like to resurrect the feature at some point).

Another option is to generate the input files as part of the test fixture.

Yet another is to use a data package like https://gitlab.cern.ch/lhcb-datapkg/QMTestFiles

That's one very valid option, if we do not zip python directories (it's not done with the new CMake configuration, but I'd like to resurrect the feature at some point).

Ah, I confess I did not think of that. Also, I do not have a feeling for how much one gains zipping the files as compared to not doing it. And what happens when you import a file from a zipped directory?

Another option is to generate the input files as part of the test fixture.

This is probably viable for small examples but I would be scared of opening a Pandora box.

Yet another is to use a data package like https://gitlab.cern.ch/lhcb-datapkg/QMTestFiles

IMO this is no good for this case since the goal is to minimise dependencies.

In short, my suggestion may still be the easiest, and we deal with broken tests if at some point Python directories get zipped :D. Dunno. I have no strong feelings and did not look at all the files involved.

I am not sure if this solution can work since the file paths are written in the .qmt test when the different arguments are passed to the run_mc function. I will try to find a way to solve the problem.

mentioned in merge request !580 (merged)

mentioned in issue #20 (closed)

Let's discuss this at #20 (closed). Regardless of how to specify other packages, the user should have all at hand when lb-checkout-ing a package.

closed