Add truncated backpropagation through time (TBPTT) example

[svnv-svsv-jm]

📚 Documentation

It seems like there is not example anymore about how to deal with Truncated Backpropagation Through Time in the latest versions. Am I wrong?

cc @Borda

[awaelchli]

Special support for TBPTT was removed in Lightning 2.0. The release notes here provide an example how you could implement it normally, see the section Truncated backpropagation through time (TBPTT).

Adding a TBTT example to the docs would be possible too. Contributions towards this would be welcomed.

[svnv-svsv-jm]

Yes, that would be good.

However, removing this means that I still have to load a huge time-window in memory, then loop over it in the training_step, which kind of defeats the purpose of TBPTT.

One would have to pass DataLoader's to the training_step in the batch argument for it to be less memory-intensive.