Migration to Gymnasium v0.29

Summary

With the release of stable-baselines3 2.0, the old Gym API is finally being replaced in favor of Gymnasium. Our jobs fail with the latest SB3 version unless the Shimmy package is installed.

This means it's finally time to think about migrating to Gymnasium.

Steps:

• Collect the concrete API changes between Gym and Gymnasium.
• Collect our own changes we want to apply to SingleOptimizable and FunctionOptimizable
• Design a migration story for downstream users to minimize breakage.

Upstream Changes in Gymnasium

Summarized in their migration guide.

• Env.np_random: numpy.random.Generator new attribute
• `Env.reset(self, *, seed: int=None, options: dict[str, Any]=None) new parameters
• Env.reset() -> tuple[Obs, dict] new return value info
• Env.seed() removed
• Env.step() new return values terminated and truncated instead of done
• Env.__init__(self, render_mode: str=None) new parameter
• Env.render() parameter render_mode removed
• Env.render() must be called inside step() if render_mode == "human"