What is speaker diarization? How transcripts work out who said what

Speaker diarization is the part of speech recognition that works out "who spoke when." It splits a recording wherever the speaker changes and groups the pieces that belong to the same voice under one label, such as Speaker 1 and Speaker 2, so a transcript can show who said what. It does not know names: Google Cloud describes it as labeling "by number the individual voices detected in the audio," and Amazon Transcribe says it can tell apart up to 30 unique speakers (as of October 2026).

For interviews this is not a technical footnote. If a transcript gives the interviewer's words to the candidate, a summary or scorecard built from it will credit the candidate with things they never said. This page explains how diarization works, how it is measured, where it fails and how to check a tool before you rely on it.

How does speaker diarization work?

Systems differ, but most follow the same stages. The open-source pyannote.audio project describes its building blocks as "speech activity detection, speaker change detection, overlapped speech detection, speaker embedding" (pyannote.audio on GitHub):

1. Find the speech. Separate talking from silence, music and background noise.
2. Find the changes. Mark the points where one voice stops and another starts.
3. Describe each voice. Turn each stretch of speech into a numerical "embedding" that captures what the voice sounds like.
4. Group similar voices. Cluster the embeddings so stretches from the same person share a label.
5. Handle overlap. Detect where two people talk at once, which is the hardest part.
6. Line up with the words. Attach a speaker label to each word or sentence of the transcript.

Commercial speech services expose the result in similar ways. Deepgram's documentation says its diarization "recognizes speaker changes and assigns a speaker to each word in the transcript" (Deepgram). Google Cloud's Speech-to-Text tags each word with a speaker number and lets you set a minimum and maximum speaker count (Google Cloud). Microsoft's Azure Speech service offers real-time diarization and returns generic identifiers such as "Guest-1" (Microsoft Learn). Amazon Transcribe labels speakers spk_0, spk_1 and so on (AWS). All as of October 2026.

How is diarization different from speaker identification and channel separation?

Three techniques get confused, and transcription tools often combine them:

Channel separation avoids most diarization errors when each person is on their own channel. Amazon Transcribe's channel identification labels the two channels of a stereo file ch_0 and ch_1 and does not support more than two channels (AWS); Deepgram's multichannel option returns a separate transcript per channel for up to 20 channels (Deepgram).

Names usually come from somewhere else: the calendar invite, the meeting's participant list, or a person correcting the labels. Metaview's help center, for example, tells desktop users to keep the meeting window and participant tiles visible because a hidden meeting "can make speaker identification less accurate," and warns that ad hoc recordings may need speakers assigned after processing (Metaview, as of October 2026).

Does diarization happen live or after the call?

Both, and they are not always the same quality. After the call, a system can compare every stretch of speech against the whole recording before it decides how many voices there were. Live, it has to decide as the words arrive and may relabel earlier lines later. Vendors often run different models for each: Deepgram documents a batch diarizer (v2) and a streaming diarizer (v1), and says the v2 model is not available for streaming (Deepgram, as of October 2026). Amazon Transcribe and Azure both document diarization for streaming as well as recorded audio.

The practical consequence: speaker labels you see in a live transcript during an interview are a draft. Judge a tool on the final transcript it produces after the call, and if it offers both, check whether the summary is written from the live labels or the final ones.

What should you do when the labels are wrong?

• Fix them in the tool before sharing. Most note takers let you rename or reassign speakers; Metaview's help center, for example, describes assigning or correcting speakers after processing.
• Regenerate the summary after fixing labels, since the first summary was written from the wrong ones.
• Never paste an unchecked quote into a scorecard or a client document. Find it in the recording first.
• Note the fix. If a scorecard changed because a quote turned out to be the interviewer's, record that, in case the decision is questioned later.

How is speaker diarization measured?

The standard measure is diarization error rate (DER). The pyannote.metrics documentation calls it "the de facto standard metric" and defines it as the sum of three kinds of error divided by the total duration of reference speech (pyannote.metrics):

• False alarm: non-speech, such as a cough or keyboard noise, counted as speech.
• Missed detection: speech the system did not detect at all.
• Confusion: speech given to the wrong speaker.

The same documentation notes that overlapping speech counts toward the error, so a system without overlap detection will show more missed speech. For interviews, confusion is the error that hurts: a candidate's answer filed under the interviewer, or the interviewer's leading question filed under the candidate. A low overall DER can still hide a few confusion errors in exactly the sentences you want to quote.

Where does speaker diarization go wrong?

• Crosstalk. Two people talking at once, or a candidate starting an answer before the question ends.
• Short turns. "Yes," "right" and "mm-hm" are too short to fingerprint, so they get attached to whoever spoke nearby.
• Similar voices. Two panelists of similar age and accent can merge into one speaker.
• Unknown speaker count. The system guesses how many people there are; guessing two when there were three merges a person away.
• Echo. Laptop speakers play the other side back into your microphone, so one voice appears twice.
• Phone and room audio. Narrow phone audio and a distant room microphone carry less of what makes a voice distinctive.

Most fixes are cheap: headphones, one speaker at a time, telling the tool how many speakers to expect where it allows, and correcting labels before anything is shared.

Why does speaker diarization matter in interviews?

An invented example. The interviewer asks, "So you led the whole migration?" and the candidate says, "I was on the team, yes." If diarization assigns the question to the candidate, the AI summary can come out as "Candidate led the migration." That sentence then travels into a scorecard, a debrief and possibly a client submittal.

• Evidence must be the candidate's own words. Evidence-based interview feedback depends on quotes attributed to the right person.
• Summaries amplify errors. A model summarizing a mislabeled transcript has no way to know; see how to use AI to summarize interviews for the checks to run.
• Panels are harder. Three interviewers and a candidate in one room or on one call is a four-way diarization problem; our AI note takers for panel interviews guide covers the setups that hold up.
• The capture method decides the problem. A meeting bot hears one mixed stream, plus the participant names the platform shows; see what is a meeting bot. A desktop app can record your microphone and the call's sound separately. Interview Signal, which we make, does this: the microphone is labeled as the interviewer and the meeting's sound as the candidate, so the two sides are never confused with each other, though several people on the far side of the call share one label.

How do you check a tool's diarization before you rely on it?

1. Record a 15-minute mock interview that sounds like your real ones: same platform, same headset, and a colleague who interrupts once or twice.
2. Include short replies, one stretch of crosstalk, and a third voice for two minutes.
3. Read the transcript and mark every line with the wrong speaker. Pay most attention to the candidate's claims.
4. Repeat on a phone call and, if you interview in person, in a meeting room.
5. Check what happens to labels in the summary and scorecard, not only in the transcript.
6. Make "quotes checked against the recording" a rule before anything leaves the hiring team.