Adding continue_training option
Summary
This MR introduces the following changes
- Adding
continue_training
option for all models. With that, you can continue a broken training. For example, if your Job was killed due to time constrains, set this option toTrue
and the latest model is loaded with weights, optimiser state and epoch number. This is then used for continuing the training.