At 1byte/param that's 1.6GB (f8), at 2 bytes (f16) that's 2.3GB -- but there's o... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		mjburgess 6 days ago \| parent \| context \| favorite \| on: How large are large language models? At 1byte/param that's 1.6GB (f8), at 2 bytes (f16) that's 2.3GB -- but there's other space costs beyond loading the parameters for the GPU. So a rule of thumb is ~4x parameter count. So round up, 2B -> 2*4 = 8GB VRAM

1718627440 5 days ago [–]

That sounds about the size of a modern browser (aka. any Electron et al. application)

Consider applying for YC's Fall 2025 batch! Applications are open till Aug 4
Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact