Nemotron 3 Diarization: NVIDIA's Open Speaker Diarization Model for Up to Eight Speakers

A 14.72% diarization error rate, roughly 24% lower than the next system, put NVIDIA's new speaker diarization model at the top of VoiceArena's initial Diarization-Bench, out of 12 systems. Nemotron 3 Diarization was released on 23 September 2026 as a 100M-parameter, open-weight model on Hugging Face. Its one job is to work out who spoke when in a recording, including the moments when people talk over each other, for up to eight speakers.

That is a narrow job, but it is the one that breaks most meeting, podcast and call transcripts. A transcript can get every word right and still be useless if you cannot tell who agreed to send the report by Friday.

What Nemotron 3 Diarization actually does

Diarization is not transcription. The model produces no words. It takes 16 kHz mono audio and outputs, for every 10 ms slice, the probability that each of eight speaker channels is talking. Postprocessing turns that into plain segments like:

0.400 2.100 speaker_0
1.800 3.250 speaker_1
3.600 4.700 speaker_0

In that example, both speakers overlap between 1.8 and 2.1 seconds. Because each speaker has its own channel, two can be active in the same frame, which is how the model handles crosstalk rather than forcing every moment onto one person.

To get a speaker-labelled transcript, you pair it with a speech-to-text model that returns word timestamps, then assign each word to whichever speaker is active at that moment. NVIDIA's own blog shows this with its Parakeet ASR model in about 20 lines of Python.

Two design choices are worth knowing:

• Speakers are numbered by arrival. The first voice heard becomes speaker_0, the next speaker_1, and so on. This keeps labels stable across chunks when streaming, so the same person does not flip between labels mid-meeting.
• Labels are anonymous. The model can tell you speaker_2 talked from 41s to 58s. It cannot tell you that speaker_2 is Priya. Mapping labels to names is your application's job, using meeting metadata or a separate speaker verification step.

Streaming or offline, from one checkpoint

The same model runs in four recommended configurations, trading latency for accuracy:

• Offline style, 30.4 s input-buffer latency: recorded meetings, podcasts, batch jobs
• Low latency, 1.04 s: live captions with a short delay
• Very low latency, 0.64 s: voice agents
• Ultra-low latency, 0.32 s: the lowest recommended setting

Those numbers are only the audio the model waits for before running. Compute, network and your ASR step add to real end-to-end delay. With chunked processing there is no fixed maximum recording length.

How much better is it, really

The benchmark numbers come mostly from NVIDIA and should be read that way, but they are detailed and include the unflattering results. Against NVIDIA's own previous four-speaker model, error rates drop on every evaluation set. On NOTSOFAR1 meeting audio in offline mode, diarization error rate falls from 21.77% to 6.77%. On two-speaker phone calls, though, the new model is slightly worse (5.98% versus 5.68%). The gains show up most as speaker counts rise, which is what the model was built for.

Two outside checks back the direction. VoiceArena's initial Diarization-Bench, across 139 English conversations (about 22 hours), ranked it first; VoiceArena notes those initial results may change once its version 1 evaluation is complete. Argmax, which added the model to its Pro SDK 3 for on-device transcription, reported the lowest error rate of six systems it tested across 11 datasets.

Speed is not the constraint. On an RTX PRO 5000, NVIDIA reports processing audio thousands of times faster than real time in offline batch mode.

Using the speaker diarization model

There are three realistic routes in:

1. Try the hosted demo first. NVIDIA's Hugging Face Space accepts a live microphone or an uploaded file, so you can test it on your own messy audio before installing anything.
2. NeMo-Speech.cpp. NVIDIA's native runtime offers a one-line CLI: nemo-speech diarize meeting.wav, or nemo-speech transcribe meeting.wav --diarize --json for a speaker-tagged transcript.
3. Python. Load it through NVIDIA NeMo (SortformerEncLabelModel.from_pretrained("nvidia/Nemotron-3-Diarization")) or through Hugging Face Transformers, which supports it natively from source.

For teams that do not want to run GPUs, NVIDIA lists Baseten and DigitalOcean as hosted deployment partners.

One practical trap: search results still surface an earlier Nemotron-3-Diarization-preview repo. That one sits under an evaluation-only licence that forbids production use. The general release at nvidia/Nemotron-3-Diarization is the one to use.

Licence and limits

The model is released under the OpenMDW License 1.1, and the model card states it is "ready for commercial or non-commercial use." It was trained on about 10,000 hours of real conversations plus roughly 82,600 hours of simulated multi-speaker mixtures across English, Mandarin, several Indian languages and other multilingual sources.

The limits are clear from NVIDIA's own documentation:

• Hardware. The supported path is Linux with an NVIDIA GPU (Ampere, Ada, Hopper or Blackwell, including consumer RTX 30, 40 and 50 cards). A community C++ port called audio.cpp claims Metal and CPU support, but that is a third-party project and has not been checked here.
• Eight speakers maximum. A larger panel or town hall will see missed or misassigned speech.
• Hard audio still hurts. Noise, heavy reverb, far-field microphones and very long recordings raise error rates.
• It is a component, not an app. There is no interface for uploading a Zoom file and getting a clean document back. Someone has to wire it to an ASR model and handle the output.

Who it's for

It suits teams building their own transcription or call analysis pipelines, voice agent developers who need to know which caller is speaking, and anyone producing podcasts or panels who already has a GPU and a bit of Python. It is also a strong upgrade for anyone currently running an older open diarization model.

It is not for someone who just wants better meeting notes tomorrow. For that, a finished tool is the faster route, and the options are sorted by recording type in Best AI transcription tools. What Nemotron 3 Diarization changes is the ceiling for people who build their own: accurate, overlap-aware speaker labels are now free, open and commercially usable.

Sources: Nemotron 3 Diarization model card, NVIDIA's Hugging Face blog post (23 Sep 2026), NVIDIA AI on X, VoiceArena Diarization-Bench, OpenMDW License 1.1.