NelsaHost
search Login
smart_toy
Llama Large Language Models

Llama 3.1 70B (70.00B)

Parameters
70.00B
VRAM (FP16)
140.0GB
VRAM (INT4)
35.0GB
Context
128000
check_circle View Compatible GPUs open_in_new View on HuggingFace

tune Quantization Options

Quantization VRAM Required Min GPU
FP16 (Half Precision) 140.0GB A100 / H100
INT8 (8-bit Integer) 70.0GB A100 / H100
Q4_K_M (GGUF 4-bit) 35.0GB A6000 / 2x 4090
q3_k_m 28.0GB A6000 / 2x 4090

memory Compatible GPUs

check_circle

A100 40GB

40.0GB VRAM

Perfect
check_circle

A100 40GB

40.0GB VRAM

Perfect
check_circle

H100 SXM

80.0GB VRAM

Perfect
check_circle

H100 SXM

80.0GB VRAM

Perfect
check_circle

H100 SXM

80.0GB VRAM

Perfect
check_circle

H100 PCIe

80.0GB VRAM

Perfect
View All Compatible GPUs

Model Details

Family Llama
Category Large Language Models
Parameters 70.00B
Context Length 128000

Similar Models

Llama 3 70B 70.00B Llama 3 8B 8.00B Llama 3.1 405B 405.00B Llama 3.1 8B 8.00B