This adds tables to store register values in the configuration database, and "plumbing" commands to import and export whole configurations. Imported values can optionally be deduplicated. The file format is expected to be compatible with the one used by
When importing, each "component" is attributed a name based on the file name. The names need to be unique within a configuration.
This MR builds on top of !224 (merged) (database migrations in particular need to be applied in order, I don't want to deal with merging them).
I guess #121
How Has This Been Tested?
As usual, start by migrating your database:
gem-configdb alembic upgrade head
I'm sure you'll want to test this MR with actual files for
read_configuration_file that I don't have access to, but first start with the data I used so you have an idea what to expect if it works. So generate some:
gem-configdb generate-test-data test_data_0 gem-configdb generate-test-data test_data_1 gem-configdb generate-test-data test_data_2
Each folder will contain one endcap worth of VFAT data, plus some additional random values. You can inspect the files or generate more yourself, the loader doesn't know what it loads (apart that it loads all
.cfg files in a folder).
Importing is easy, if a bit slow (about 20s per set, most of it spent reading files and mashing values in Python):
$ gem-configdb import-data test_data_0 42 $ gem-configdb import-data test_data_1 43 $ gem-configdb import-data test_data_2 44
The numbers are the ids of the newly created configurations.
To export the data, do:
$ gem-configdb export-data 42 empty-folder
Of course, you'll get an error if
empty-folder isn't actually empty. You'll also get an error if it exists. So make sure this isn't the case. Oh, and by the way,
42 comes from the loading of
test_data_0, you'll get a different number.
I didn't write a tool to compare
test_data_0 because it's a pain to do with all those text files. Instead, we can leverage the database by importing
empty-folder back, enabling deduplication:
$ gem-configdb import-data empty-folder -r 42 WARNING:root:No difference with respect to the reference configuration 42
As you can see, the loader detected that it could reuse the good old configuration 42 instead of creating a new one, so it did just that. But what if we try to load something else?
$ gem-configdb import-data test_data_1 -r 42 45
You can't see it, but this time most of the data from configuration 42 was reused (about 80%, because each set of generated data contains 10% of random configurations -- this can be configured at generation time with
Let's finally check that the dedup'ed configuration is identical to the non-dedup'ed one:
$ gem-configdb export-data test_data_1 -r 45 WARNING:root:No difference with respect to the reference configuration 45
To revert the schema to the tip of !224 (merged), do:
gem-configdb alembic downgrade f4adeca56f06
Types of changes
Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to change)
My code follows the code style of this project.
My change requires a change to the documentation.
I have updated the documentation accordingly.
I have read the CONTRIBUTING document.
I have added tests to cover my changes.
All new and existing tests passed.