🌚 Discover
Benchmarks

Benchmarks

It's hard to pinpoint reliable and consistent benchmarks, but hopefully this page gives you an idea on some of the current hardware requirements for running AI in its many forms.

4-bit System Requirements

ModelMinimum Total VRAMCard ExamplesRAM/Swap to Load*
7B6GBGTX 1660, 2060, AMD 5700 XT, RTX 3050, 30606 GB
13B10GBAMD 6900 XT, RTX 2060 12GB, 3060 12GB, 3080, A200012 GB
30B20GBRTX 3080 20GB, A4500, A5000, 3090, 4090, 6000, Tesla V10032 GB
65B40GBA100 40GB, 2x3090, 2x4090, A40, RTX A6000, 800064 GB

8-bit System Requirements

ModelVRAM UsedMinimum Total VRAMCard ExamplesRAM/Swap to Load*
7B9.2GB10GB3060 12GB, 3080 10GB24 GB
13B16.3GB20GB3090, 3090 Ti, 409032 GB
30B36GB40GBA6000 48GB, A100 40GB64 GB
65B74GB80GBA100 80GB128 GB

LLM Leaderboards

LLM Search Tools

LLM Eval & Benchmark Resources