Speed up training with advanced pytorch features
-
initial compilation support !187 (merged) -
full compilation support -
nested tensors !197 -
meff sdp !199 (merged) -
fp8 precision https://lightning.ai/docs/fabric/stable/api/generated/lightning.fabric.plugins.precision.TransformerEnginePrecision.html
A lot of these are bundled with https://github.com/Lightning-AI/lightning-thunder
Edited by Samuel Van Stroud