Remove more unnecessary stuff from gitlab-ci

32 jobs for pythonport in 9 minutes and 56 seconds (queued for 10 seconds)