NelsaHost

Hardware AI Models Compatibility Compare

search Login

Can I run Mistral Large 2 on NVIDIA RTX 4060?

cancel

Fail/OOM

This GPU doesn't have enough VRAM

GPU VRAM

8.0GB

Required

246.0GB

Headroom

-238.0GB

VRAM Usage

0GB 100% used 8.0GB

info Technical Analysis

NVIDIA RTX 4060 cannot run Mistral Large 2 in this configuration. The model requires 246.0GB but only 8.0GB is available, leaving you 238.0GB short.

lightbulb Recommendation

Consider using a more aggressive quantization (Q4_K_M, Q3_K_M) to reduce VRAM requirements, or upgrade to a GPU with more VRAM. Cloud GPU services like RunPod or Vast.ai offer affordable options.

tune Recommended Settings

Batch_Size

None

Context_Length

None

Inference_Framework

llama.cpp or vLLM

help Frequently Asked Questions

Can I run Mistral Large 2 on NVIDIA RTX 4060? expand_more

NVIDIA RTX 4060 (8.0GB VRAM) cannot run Mistral Large 2 (123.00B) which requires 246.0GB. You are 238.0GB short. Consider using a more aggressive quantization (like Q4_K_M or Q3_K_M) or upgrading to a GPU with more VRAM.

How much VRAM does Mistral Large 2 need? expand_more

Mistral Large 2 requires approximately 246.0GB of VRAM.

What performance can I expect? expand_more

Estimated None tokens per second.