Skip to content

Interactive viewer for model FSM

Design of v2+ is built based on a very fundamental understanding of minimization procedure with following basic concepts:

  • having an input data sample as any kind of input data towards which we minimize the model where
  • a model is the set of numerical parameters with some sort of (polymorphic) relation to the input data. This set of parameters is getting altered by
  • a fitting procedure which defines a rule, numerical algorithm or any other kind of alteration procedure.

Thus, we made a framework using finite state-machine (FSM) representation where

  • model is the state and
  • fitting procedure is the transition.

As the example, the simplest FSM is just numerical fitting procedure that adjusts function of N parameters with given initials ("model") to the given data. This is trivial FSM with two states and one transition.

More complex scenarios like the one currently in use for SADC detectors (MSADC and WaveBoard).

FSM can be embedded. Currently, following generic procedures containing embedded FSMs are defined:

  1. Breakdown fit procedure (BreakdownFitProcedure class). This procedure is used when model can be splitted onto set of independent models and input data set can be split on sub-samples. Current use case for this model is the time domains -- having sets of pulses which do not overlap in time we has to apply fitting within each time domain independently. Managed FSM will be evaluated in the sub-domains.
  2. Competing fit procedure (CompetingFitProcedure class). This procedure expects that its subject FSM can yield multiple fitting results. Among these results CompetingFitProcedure will take the "best" one (according to least square error or any other metric).
  3. Fallback fit procedure (FallbackFitProcedure class). This procedure maintains an ordered list of fitting procedures which it will apply sequentially until one of them will eventually succeed or all of them fail.

The FSM gets defined using runtime configuration (JSON currently). Then we feed it with input data and evaluate the transitions till we get some final state (model) satisfying certain criteria. All the transitions are logged and model states are cached during FSM evaluation.

Now, regarding the complexity of the resulting FSM evaluation, it becomes quite tedious to manage and debug the FSMs. Even for Moyal model MSADC we have:

  1. Pulse/peak finding procedure
  2. Top-level competing procedure comparing results within the entire time domain
  3. Breakdown procedure performing fits within sub-domains
  4. In-domain competing procedure performing competing fit within sub-domains
  5. Few initial conditions to compare
  6. Numerical fit procedure (finally) doing the fit as non-linear constrained minimization (with GSL fitter)
  7. Merging procedure, optionally combining pulses which seem to be "close" in time, capable to re-run the competing FSM

It was expected, that the resulting FSM is complex and hard to inspect. To cope with its complexity we anticipated JSON dump of the models after each transition. Resulting JSON is huge.

Most of the plots I quoted so far were obtained by querying certain model state with Jupyter.

image

However, typically path looks like data['_subprocedures']['fullDomainNPoleFit:FitProc']['modelsCollection']['nodes'][1]['model'] and this is just for some sort of top-level model. Combining it with the initial conditions or switching between Moyal and N-pole models with different number of samples, sampling interval, parameters, etc becomes cumbersome.

For instance

  1. recent reports with SRD were referring to some particular events appearing with 1e-4 frequency among the events (and 1e-6 among the input samples assuming that every event brings, like 3e2 waveforms)
  2. slowdown reported recently by Martina and Laura is probably caused by merging procedure re-launcing the fit. However I am not sure since disentangling the resulting FSM graph became quite complicated.

So, in order to significantly simplify issue-tracing activities and provide better understanding of what is going on, a FSM inspection tool (browser) is proposed:

  1. It will render the FSM as (possibly cyclic) directed graph with model states as the nodes and fitting procedures as transitions, labeled appropriately
  2. By clicking on the graph node one has to be able to render the available information of the model as a certain kind of plot (model-specific)
  3. It must be possible to go inside the composite node to see the FSM beneath competing, breakdown, fallback procedure

Regarding complexity of this task we should implement it by high-level language --- Python or JavaScript. There are options:

  1. Write everything on JavaScript using D3 and DAGRE. This monolithic single page app (SPA) is the most straightforward way, yet due to features of JS ecosystem it is hard to extend and customize.
  2. Write everything on JavaScript using Vue or React with Router to have persistant states, history, etc. This SPA will have better state-management system than 1st option due to reactive programming paradigm, although maintenance is quite harsch (these frameworks are quite volatile).
  3. Write server part on Python using Flask, build FSM with Graphviz and plot the stuff using seaborn/pyplot/D3, etc. This heterogeneous solution is simpler since Python has much nicer tools to manage the hierarchical data, Graphviz is more stable than DagreJS and the extensibility is much easier.