Optimizing PyTorch Models
Most AI models never leave a Jupyter notebook. This workshop shows how to turn real-world PyTorch codebases into deployable, efficient binaries—covering practical techniques for optimizing and exporting complex models to production at low cost.
Other than building and training deep neural networks, PyTorch offers a host of
other tools to help optimize and export models, especially to lower-level
frameworks and runtimes for deployment on edge devices. This workshop focuses on
techniques which ensure that torch models (instances of torch.nn.Module) work
well with frameworks like CoreML, TensorRT, and ONNX Runtime.
For most neural network architectures that are already present in the torch ecosystem (torch, torchvision, torchaudio, etc), optimization, compiling and export is usually not a significant problem. They're well supported by the community and are thus amenable to optimization. However, when it comes to SoTA models that typically beat an ML benchmark and are inspired by research papers, there is no guarantee that the code accompanying the papers will be suitable for deployment or even optimization. This is not all that surprising, since most code that is released alongside papers is meant primarily to reproduce the benchmarks that are in the paper. Portability and consumption is typically not a priority.
In this workshop, we will introduce a collection of recipes which we have developed by working with various SoTA models and making them work on edge devices, especially with torch. We will interactively cover a few SoTA architectures for computer vision tasks and develop a guidebook for how best to optimize and export torch models.
This is meant to be a fully hands-on workshop, meaning that the speaker's talking will be intersperesed with both the speaker and the audience coding. Participants will have to bring their own computers, or share them with others. A detailed Github repository will be made available to the participants closer to the workshop dates so that they can come prepared with the necessary tools and dependencies installed.
Jaidev Deshpande
- Aftershoot
- IIT Madras
jaidevd
Men's M
Jaidev currently does MLOps at Aftershoot. He has a decade of experience in machine learning and software development. You are likely to run into him at various tech events.