Migration to Gymnasium v1.0

While we've been busy migrating to Gymnasium at all, it has had a new series of breaking changes and finally released the (hopefully final) version 1.0.

Gymnasium v1.0 Release Notes

This is a low-priority item because Stable Baselines doesn't support v1.0 (and doesn't seem to plan to just yet). However, this kind of change has snuck up on us in the past, so it's best to think about it now.

Big Changes

• Vector envs. They've been stabilized and their support in SB3 is probably a matter of time. We have to consider: 1. Do we want to support running them in the GUI? 2. Do we want a similar vectorization feature for simulation-only optimization problems?
• Functional Environments. They're similar to SeparableEnv but even more fine-grained. An example follows. We have to consider whether we want to phase out SeparableEnv and GoalEnv in their favor.

from typing import Any
import gymnasium as gym
from gymnasium.functional import StateType, ObsType, ActType, RewardType, TerminalType, Params

class ExampleFuncEnv(gym.functional.FuncEnv):
  def initial(self, rng: Any, params: Params | None = None) -> StateType:
    ...
  def transition(self, state: StateType, action: ActType, rng: Any, params: Params | None = None) -> StateType:
    ...
  def observation(self, state: StateType, rng: Any, params: Params | None = None) -> ObsType:
    ...
  def reward(
      self, state: StateType, action: ActType, next_state: StateType, rng: Any, params: Params | None = None
  ) -> RewardType:
    ...
  def terminal(self, state: StateType, rng: Any, params: Params | None = None) -> TerminalType:
    ...

Small Changes

• Gymnasium no longer loads registrations from entry points; it now only knows about envs that have either been imported, or whose package is specified with the syntax gymnasium.make("package:namespace/env-v0"). This shouldn't affect us because we now basically have our own implementation of make().
• gym.make() has removed kwargs auto_reset and step_api_compatibility
• All wrappers have been renamed, e.g. AutoresetWrapper → AutoReset.
• Wrapper StepAPICompatibility has been removed; if we want to provide support for Gym-style old environments, we must use Gymnasium v0.29 or Shimmy.
• Return value of Space.seed() has been changed and it's been affirmed that this method is still supported. This will require docs changes on our end.