Test suite: Automate testing of 'algo test' FSM
Goal: Ability to fully verify the 'algo test' FSM on any Serenity with a single command.
Should use a standard test framework, like pytest, and implement as a test case that does the following:
- Runs the algo test setup transition once, and then runs the 'playback' transition a few times (with a different input data file each time).
- Checks the monitoring status of the device after running each transition
- Checks that only the expected set of monitorable objects are enabled after each transition, i.e. TTC interface; all other components should be disabled.
- Check I/O data after each capture transition:
- Compare captured RX buffers with injected data
- Compare captured TX buffers with injected data - should be offset by 7 clock cycles.
Command line flags:
-
--algo-test-config: Path to config file containing values of all parameters for algo test FSM, except for the firmware package and input file -
--board: Hostname (and port) of board -
--processor: IDs of processors to use (e.g.x0,x1) -
--tolerate-monitoring-warnings: Flag warnings in monitoring as pytest warnings (without this flag, assert that monitoring status is 'good') -
--tolerate-command-warnings: Flag warnings in transitions as pytest warnings (without this flag, assert that transition end state is just 'done') -
--fw-package: Path to firmware package
Parameters for test case function:
-
settings_pool:SettingsPoolobject created from--algo-test-configconfig file, with--fw-packageused in replacements) -
processor:Processorobject (specified by combination of--boardand--processors) -
tolerate_monitoring_warnings: Corresponding to--tolerate-monitoring-warnings -
tolerate_command_warnings: Corresponding to--tolerate-command-warnings
Note: Where-ever possible, should try to make as much of this this test code as possible board-/plugin-independent. That board-/plugin-independent test code will then be migrated to the Python library repo soon.
Edited by Tom Williams