Uniform treatment of Algorithm properties in PyConf
In PyConf.components
we inspect all properties an Algorithm
is configured with. If we determine the property value is DataHandle
-like, we assume the property defines input, and if the property value is Tool
-like the property defines tool handles. With this information, we can then instantiate the set of underlying Configurable
instances to ensure that the algorithm's data dependencies will be met (by creating the producing algorithms) and the tools will be available (by creating the tools).
There are many workarounds in PyConf.components
for the cases where a property value is a list of DataHandle
or Tool
. We would need to add similar workarounds to support other containing structures, such as dict
.
In Moore#51 (closed), the issue of 'input-providing properties' came up. These are properties which are not DataHandle
directly, but somehow 'contain' data which is implicit input to an algorithm. In a sense, these properties are just like any other container (list
, dict
), in that the Algorithm
must make sure to extract any 'special' object from within them. These special objects are:
-
DataHandle
objects, in which case we need to configure their producers. -
Tool
objects, in which case we need to configure the tool.- More generally this means 'anything with an equivalent
Configurable
', such as services.
- More generally this means 'anything with an equivalent
We should unify how we treat property values. Each property value should have all data and configurable objects extracted from it, and the Algorithm
should then store these in a uniform way (e.g. a flattened list).
One way to think about this is to not consider Tool
as something special. It can:
- Inject data dependencies to an
Algorithm
. - Require an underlying
Configurable
to be instantiated.
We can see how these requirement affect the behaviour of Algorithm
, and use that to define the interfaces all other things can adopt (e.g. functors). Point 1 could mean moving property 'inputs' into direct algorithm inputs, and point 2 could mean defining a .configuration()
interface which returns a dataflow_config
object.
The downside of unification is that sometimes one does want to treat things specially, e.g. to retrieve the list of all Tool
objects held by an Algorithm
.
All that said, this issue represents a low-priority refactoring task. It has the potential to make PyConf more understandable with more clearly defined boundaries, but we can always add a few more hacks to support additional property types.
/cc @rmatev