feat: download component and set to next stage

The following features or changes were made:

• download component always moves it to the next stage in localDB
• mqdbt is updated to handle setting stages both in localDB and in prodDB (this is swapped out everywhere)
• alternative stages are skipped when transitioning stages automatically, both in signing-off (prodDB) and in pulling a new component (localDB)
• switching stage from alternative to another stage will come with a checkbox to indicate whether to mark the alternative stage as ready-for-upload (default) or not

Pulling a component in MODULE/INITIAL-WARM gives us:

Here's some screenshots of the flow now.

• when component is not in an alternative stage, one can only switch to an alternative stage:
• when switching from a non-alternative stage, one can only switch to an alternative stage:
• after switching, a message is displayed to let the user know what happened/changed:
• when component is in an alternative stage, one can switch to any other stage:
• when switching from an alternative stage, one can switch to any other stage, and indicate if current alternative stage should be considered for upload or not:

This implements the options from an email sent to itk pixel modules mailing list (see below).

Put simply, when a component is downloaded from the PDB -- we will adjust its stage to the next stage with respect to the production database.

Email from Giordon

Dear all,

I have had multiple persons with "both sides" interpretations of what the interactions between PDB and localDB (LDB) should be when it comes to uploading (signing-off) and downloading (pulling). I personally get confused quite often and can see it either way... so we just need to document how to interpret this once and for all.

Recall that for module assembly/testing,we have for example these two stages in order:

MODULE/WIREBONDING MODULE/INITIAL_WARM

PULLING

PDB: module current stage is MODULE/WIREBONDING LDB pulls the module and marks the current stage as: (pick one of these two scenarios) a) MODULE/WIREBONDING b) MODULE/INITIAL_WARM and marks the current stage as needing sign off from you (e.g. you are running QC at this current stage.

If you pick (a), you say you received a module that needs to be wirebonded (that is its current state, "in-progress"). [LDB = PDB]

If you pick (b), you say that the module has been wirebonded, and you are working on running QC for the initial warm. [LDB = PDB + 1]

For reporting, scenario (a) interprets the current stage in PDB as the to-be-signed-off stage. scenario (b) interprets the current stage in PDB as the most-recently-signed-off stage.

UPLOADING

LDB: module current stage is MODULE/WIREBONDING. You have signed off in LDB of the current stage. You upload to PDB. In localDB, your module is now at MODULE/INITIAL_WARM. In PDB, the current stage moves from (pick one of these two scenarios): c) MODULE/ASSEMBLY -> MODULE/WIREBONDING d) MODULE/WIREBONDING -> MODULE/INITIAL_WARM

If you pick (c), you say that you've finished wirebonding, and so you mark the module as "wirebonding" for the current stage in PDB (uploading the tests for it). Your LDB switches to initial warm reflecting that you are currently working on this stage, while your PDB is at wirebonding. [LDB = PDB + 1]

If you pick (d), you say that you've finished wirebonding, so you mark the module as "initial warm" for the current stage in PDB. Your LDB matches this initial warm, reflecting that you are currently working on this stage. [LDB = PDB]

For reporting, scenario (c) interprets the current stage in PDB as the most-recently-signed-off stage. scenario (d) interprets the current stage in PDB as the to-be-signed-off stage.

Current LDB Behavior

Currently, LDB is using (a) and (c). The reason for this is to show that the only way to progress a module through QC is via signing off and uploading to have the stage switch in LDB to the next stage, while having the stage in PDB reflect the most-recently-completed-stage.

This behavior has been ok until we added the QC_CROSSCHECK (which occurs at another site) in which case, the following procedure happens:

• Site A: INITIAL_WARM sign-off in LDB.
• PDB: ASSEMBLY -> INITIAL_WARM
• Site A: INITIAL_WARM -> QC_CROSSCHECK
• Site B: downloads and has module in INITIAL_WARM

Site B is now stuck. they cannot just sign-off and switch to the next stage, because they were not the institute that ran the tests at INITIAL_WARM. Their localDB does not work. Hence, I need to fix this mismatch which only occurs when going between sites. So the big question is, which scenario pairs do you want? Pick from the following:

(a) and (d). interprets the current stage in PDB as the to-be-signed-off stage.
(b) and (c). interprets the current stage in PDB as the most-recently-signed-off stage.

Which works best for everyone?

@hoide responded with

Hide's response

Hi Giordon, all,

Thanks for raising this point. The situation seems unfortunately a little bit more complicated, I would say. I think the behavior

2. (b) and (c). interprets the current stage in PDB as the most-recently-signed-off stage.

should look like a smooth solution, as the interpretation of PDB record state is invariant before/after this change, and we just need to change the behavior of the LocalDB to "internally" increment the stage after pulling the past stages info. But as you discussed, at this point the choice seems to be arbitrary between two options.

I think complication comes with alternative stages. They are optional, and one should explicitly switch the stage to an alternative stage when needed. When and how this manual trnasition should take place, and how it should be recorded on PDB? The QC_CROSSCHECK is defined as an alternative stage, so by default such a stage should be skipped in the natural sequence of the stage flow. So in the scheme-#2 above (b and c), after pulling a module which completed INITIAL_WARM should transit to INITIAL_COLD in the LocalDB's behavior. This is still a logical and deterministic behavior. In order for the user to get to QC_CROSSCHECK, in the scheme-#2, the user will need to explicitly change stage from INITIAL_COLD to QC_CROSSCHECK by hand (and this manual transition to an alternative stage is actually supported in LocalDB).

I think it is better that this manual transition from INITIAL_COLD to QC_CROSSCHECK is not subject of PDB stage transition recording, as it is an expedient stage transition and not reflecting the actual decision.

In the scheme-#1 (a and d), on the contrary, the stage explicitly switched from INITIAL_WARM to INITIAL_COLD on PDB (because this is the default transition), and then the stage transits from INITIAL_COLD to QC_CROSSCHECK when needed. At least to me, this does not look like cleanest recording on PDB. Similar story for another alternative stage "wirebond protection". In the scheme-#2, the PDB stage transition record will be as clean as what we will naturally expect, i.e. "unreal" stage transition involving alternative stages will not be recorded and that is only managed within LocalDB.

After above considerations I tend to think the scheme-#2 is preferable.

Best, Hide

@msaimper confirmed/concurred with

Matthias' email

Hello Giordon,

thank you for raising this point.

~~

As discussed privately already before this thread was started, I tend to agree with Hide and think that scheme-#2 is preferable because easier to implement and involves minimal stage changes in PDB.

The 'to-be-signed-off stage' is sometimes not obvious due to the alternative stages, and also if sites are performing reworks.

~~

If I understood correctly, to finish implementing scheme-#2 one needs to update localdb only, and force the stage of a freshly pulled module to go to the next mandatory stage w.r.t the one in PDB. Even during production localdb will allow manual stage switching to 'alternative stages' so this should work.

If this mechanism does not work because the PDB stage is incorrect in the first place, the user 'only' has to fix the stage of the component in PDB and repull, which is a few clicks on the unicorn interface.

In this context, I believe it would be very useful to have in localdb a functionality allowing to synchronize the stage of all the child components to the current PDB stage for a given module component.

This will help fixing the PDB stage of the module if needed: the user will only have the fix the PDB stage of the module component e.g. via the unicorn interface, and then click this button in localdb, and then repull the component to localdb.