Test suite: Automate testing of 'algo test' FSM
Goal: Ability to fully verify the 'algo test' FSM on any Serenity with a single command.
Should use a standard test framework, like pytest
, and implement as a test case that does the following:
- Runs the algo test setup transition once, and then runs the 'playback' transition a few times (with a different input data file each time).
- Checks the monitoring status of the device after running each transition
- Checks that only the expected set of monitorable objects are enabled after each transition, i.e. TTC interface; all other components should be disabled.
- Check I/O data after each capture transition:
- Compare captured RX buffers with injected data
- Compare captured TX buffers with injected data - should be offset by 7 clock cycles.
Command line flags:
-
--algo-test-config
: Path to config file containing values of all parameters for algo test FSM, except for the firmware package and input file -
--board
: Hostname (and port) of board -
--processor
: IDs of processors to use (e.g.x0
,x1
) -
--tolerate-monitoring-warnings
: Flag warnings in monitoring as pytest warnings (without this flag, assert that monitoring status is 'good') -
--tolerate-command-warnings
: Flag warnings in transitions as pytest warnings (without this flag, assert that transition end state is just 'done') -
--fw-package
: Path to firmware package
Parameters for test case function:
-
settings_pool
:SettingsPool
object created from--algo-test-config
config file, with--fw-package
used in replacements) -
processor
:Processor
object (specified by combination of--board
and--processors
) -
tolerate_monitoring_warnings
: Corresponding to--tolerate-monitoring-warnings
-
tolerate_command_warnings
: Corresponding to--tolerate-command-warnings
Note: Where-ever possible, should try to make as much of this this test code as possible board-/plugin-independent. That board-/plugin-independent test code will then be migrated to the Python library repo soon.
Edited by Tom Williams