Idea: Use pip to manage the *installed* JAR packages
Currently, cmmnbuild-dep-manager is responsible for resolving and managing JAR installation state. This means we are writing custom tooling to handle the fact that we have a special case for Java. In the past we have discussed using pip
for this purpose, but it always meant that we would have to cease using the Gradle resolver, and rely on the pip
one instead - this left me uncomfortable, and opens the risk that the resolution is inconsistent or even incorrect.
Instead, we could continue to use Gradle for resolving JARs, but instead of downloading/managing the JARs ourselves, use pseudo Python packages which contain a single wheel in them. With this approach we are able to use things like pip list
to see both Python and Java dependencies in the same place.
In order to do this we would need a Python package index which can serve a wheel containing a JAR stored in {prefix}/share/java/{groupid}/{jar_name}.jar
. The name of the Wheel would be cmmnbuild_JAR_GID_{groupid}_PID_{productid}-{version}-any.whl
(or equivalent, but MUST include groupID, productID and version in machine-readable form).
These wheels could be generated up-front, or on demand (and cached) - I would opt for the latter if practical.
When cmmnbuild-dep-manager
has resolved the JARs it needs, it would then issue a {sys.executable} -m pip install --no-deps {all_the_jars}
.
As well as this, cmmnbuild-dep-manager
requires the ability to know if the current environment is reflective of the declared requirements. We would therefore require an extra package cmmnbuild_resolved_{requirements_hash}.whl
which will allow cmmnbuild-dep-manager
to avoid needing to re-resolve the dependencies. The details of this package need to be fleshed out (e.g. should it include the resolved dependencies as dependencies).
With this in place, we can begin to use standard package management tools more readily (e.g. pip list
). It is possible that we can do this in an entirely backwards compatible way (if the current jars directory is considered implementation detail). We would want to make sure that we can continue to support custom insertion of JARs etc.
There have been similar ideas in the past (but perhaps were relying on replacing the Gradle resolve step with pip
?).