NelsaHost
search Login

Can I run Whisper Small on NVIDIA H100 PCIe?

check_circle
Perfect
Yes, you can run this model!
GPU VRAM
80.0GB
Required
0.5GB
Headroom
+79.5GB

VRAM Usage

0GB 1% used 80.0GB

Performance Estimate

Tokens/sec ~117.0
Batch size 32

info Technical Analysis

NVIDIA H100 PCIe provides excellent compatibility with Whisper Small. With 80.0GB of VRAM and only 0.5GB required, you have 79.5GB of headroom for comfortable inference. This allows for extended context lengths, batch processing, and smooth operation.

lightbulb Recommendation

You can run Whisper Small on NVIDIA H100 PCIe without any compromises. Consider using full context length and larger batch sizes for optimal throughput.

tune Recommended Settings

Batch_Size
32
Context_Length
448
Inference_Framework
llama.cpp or vLLM

help Frequently Asked Questions

Can I run Whisper Small on NVIDIA H100 PCIe? expand_more
NVIDIA H100 PCIe has 80.0GB VRAM, which provides 79.5GB of headroom beyond the 0.5GB required by Whisper Small (0.24B). This is plenty of room for comfortable inference with room for KV cache, batching, and extended context lengths.
How much VRAM does Whisper Small need? expand_more
Whisper Small requires approximately 0.5GB of VRAM.
What performance can I expect? expand_more
Estimated 117 tokens per second.

GPU

memory
NVIDIA H100 PCIe
80.0GB VRAM

AI Model

smart_toy
Whisper Small (0.24B)
0.24B params

More with H100 PCIe

Whisper Medium Perfect Whisper Base Perfect SDXL 1.0 Perfect SD 3 Medium Perfect