NVIDIA releases Nemotron 3 Nano Omni, an open-weights multimodal model handling text, images, video, and audio with 9x throughput gains over alternatives.
NVIDIA released Nemotron 3 Nano Omni, an open-weights omni-modal model combining a hybrid Mamba-Transformer MoE backbone with dedicated vision (C-RADIOv4-H) and audio (Parakeet-TDT-0.6B-v2) encoders. The model tops leaderboards including MMlongbench-Doc, OCRBenchV2, WorldSense, DailyOmni, and VoiceBench while delivering 7.4x–9.2x system efficiency over comparable open omni models like Qwen3-Omni. BF16, FP8, and NVFP4 checkpoints are available on HuggingFace now. Training used staged multimodal alignment, preference optimization, and multimodal reinforcement learning.
Nemotron 3 Nano Omni collapses what used to be 3-4 specialized pipelines (OCR model, ASR model, video captioner, LLM) into a single open-weights model you can self-host. The Mamba-Transformer MoE backbone enables long-context multimodal reasoning without the quadratic attention cost that kills most vision-language models on dense documents. FP8 and NVFP4 quantization options mean you can run this on a single A100 or H100 at meaningful throughput — 9.2x faster than Qwen3-Omni on video workloads per NVIDIA's benchmarks.
Pull the FP8 checkpoint from HuggingFace this week and run your hardest document-intelligence test case — a 20+ page contract or technical report — against your current OCR+LLM pipeline. If accuracy matches or exceeds at lower latency, you have a migration path that eliminates at least one API dependency.
Run: pip install transformers accelerate and then pull the model with: from transformers import AutoProcessor, AutoModelForCausalLM; model = AutoModelForCausalLM.from_pretrained('nvidia/Nemotron-3-Nano-Omni', torch_dtype='auto', device_map='auto')
Tags