Frequently Asked Questions

Q: Failed to load the pruned model.
A: When using TP, your model's architecture will be altered, rendering the original structure defined in your .py file incompatible. Consequently, it becomes necessary to store the entire model object using torch.save(model, PATH) rather than solely utilizing torch.save(model.state_dict(), PATH). This modification ensures that the model can be successfully reloaded through model = torch.load(PATH).

Q: The size of .pth file increases after pruning.
A: It's important to note that we save the complete model object in the .pth file, resulting in supplementary information being stored within the saved model. However, the actual model size has been reduced after pruning. To validate this, you can either load the model into memory or export the pruned model to ONNX format.

Q: Error raised by optimizer.step after pruning.
A: After pruning, it's necessary to establish a fresh optimizer. This is because the parameters within the pruned model have been substituted with new ones. Failing to create a new optimizer can lead to an erroneous binding to the outdated parameters prior to pruning. Additionally, take note that TP cannot prune the momentum values defined within optimizers.

Q: KeyError during pruning.
A: TP relies on AutoGrad for tracing the computational graph. Therefore, please ensure that all layers are executed in the forward pass, and that all parameters have .requires_grad=True.

Q: The model remains unaltered after pruning
A: Please verify that your model is not being executed within a torch.no_grad() context. Additionally, ensure that all parameters have .requires_grad=True in order to facilitate network tracing.