add more regression tests in a smarter way

Instead of running shell code that has to be compatible with Gitlabs terrible yaml syntax, replace it with a simple bash script that is invoked in the regression-tests CI job.

The script can also be run locally, so you don't have to wait on and use the resources of CI runners.

Optimize the CI pipeline a bit by declaring dependencies and caches. Can definitely still be improved further.