The ‘magic’ underlying PyTorch https://towardsdatascience.com/pytorch-autograd-understanding-the-heart-of-pytorchs-magic-2686cd94ec95
:That is true. As I wrote earlier, PyTorch is a jacobian-vector product engine. In the process it never explicitly constructs the whole Jacobian. It’s usually simpler and more efficient to compute the JVP directly.:
Source: https://www.cs.toronto.edu/~rgrosse/courses/csc321_2018/slides/lec10.pdf
jacobian-vector products?