🤗 Model Memory Calculator

This tool provides a static memory estimate for the vRAM needed to load and train Hub models.

The minimum recommended vRAM needed to load a model is denoted as the size of the "largest layer", and training of a model is roughly 4x its size (for Adam).

These calculations are accurate within a few percent at most, such as bert-base-cased being 413.68 MB and the calculator estimating 413.18 MB.

When performing inference, expect to add up to an additional 20% to this as found by EleutherAI.

More tests will be performed in the future to get a more accurate benchmark for each model.

Currently this tool supports all models hosted that use transformers and timm.

To use this tool pass in the URL or model name of the model you want to calculate the memory usage for, select which framework it originates from (auto will try and detect it from the model metadata), and what precisions you want to use.