How the scorecard with verified quotes works in Interview Signal
On this page
The Interview Signal scorecard scores each area of the role's rubric from 1 to 10 while the interview runs, and every score carries the candidate's own words as evidence. Each quote is checked against what the candidate actually said: a quote that is not in the transcript word for word is deleted, and an area left with no quote is capped at 5 and marked "Probe further". This page describes version 0.9.10 of the desktop app, the current release as of October 2026.
Where do the scorecard areas come from?
From the role's kit. When you paste a job description and press "Generate the kit", Interview Signal writes the questions and a scorecard for the role, usually four to seven areas. Each area has a weight of 1, 2 or 3 (how much the role turns on it) and a rubric that says what a 1–3, a 5–6 and a 9–10 answer looks like. See how the kit is built from a job description.
Everything on the scorecard is editable on the kit screen, under "Scorecard": rename an area, change its "Weight", rewrite the three rubric lines ("1-3 looks like…", "5-6 looks like…", "9-10 looks like…"), or press "Add a scorecard area". A kit needs at least one area. The kit is saved with every interview, so an old interview still shows the scorecard it was scored against even after the role has been edited.
How does scoring work during the call?
- Leave "Score as we go" on. The switch sits above the transcript. With it on, the app scores again each time the candidate has said about ninety new words. The same setting is in Settings as "Score the candidate as the interview runs".
- Press "Evaluate" to score right now. The button reads "Scoring…" while it works. Use it after an answer you care about, because ending the interview does not run one more scoring pass.
- Pin the question if you want the score tied to it. Click a question in the guide and press "Pin". If nothing is pinned, the scorer uses the last thing you asked that it heard, and if your microphone caught nothing usable it works out which guide question the answer belongs to and scores anyway.
- Read each area. An area shows its score, a one-line assessment and the quotes behind it. Click a quote and the transcript scrolls to the moment it was said and highlights it.
Only the areas an answer actually speaks to get a score. An answer about a missed quarter says nothing about compensation, so that area stays at "—" instead of getting a guess. When the candidate returns to a theme later, the new quotes are added to the old ones (up to four per area), while the score and the assessment are replaced by a judgment of everything heard so far.
The scorer is told to use the whole scale, anchored to the rubric:
| Score | What it means |
|---|---|
| 9–10 | The high band is plainly met, with first-hand specifics |
| 7–8 | Most of the high band, or the mid band exceeded with specifics |
| 5–6 | The mid band: a real answer, thin on specifics |
| 3–4 | Generic, second-hand or rehearsed claims with nothing checkable in them |
| 1–2 | Avoids the question, deflects, or contradicts itself |
A "specific" is something checkable: a named system, account or person, a number, a date, a decision they made, or a mistake of their own and what changed after it. Fluency does not count as evidence, so a polished answer made of general claims scores 3–4 however complete it sounds. Assessments never infer gender; the candidate is "they" unless the transcript itself uses a pronoun.
How are the quotes checked?
Nothing the model returns is trusted until it has been matched against the transcript:
- Only the candidate's words count. Quotes are checked against the candidate's side of the transcript, not the whole conversation, so a quote of your own question is evidence of nothing and is removed.
- Word for word. The match ignores capital letters, punctuation and curly versus straight quotation marks, because those vary in transcription. Word order and word choice must match, so a paraphrase fails.
- At least four words. Shorter quotes are refused, because "yes, I did" appears in almost any interview. In Japanese or Chinese, two characters count as a word.
- You are told. When you press Evaluate and quotes were removed, a message says how many "could not be matched to the recording" and were dropped.
- No quote, no high score. An area left with no surviving quote is capped at 5 and shows "Score capped — no quotable evidence yet" and "Probe further". Five means "seemed fine", not a strength a client should read about.
The reason is simple: a paraphrase inside quotation marks, sent to a client as evidence, is the worst mistake an interview tool can make on a recruiter's behalf.
How is the fit score calculated?
The fit score at the top of the scorecard is a weighted average of the areas that have a score, on the same 1–10 scale, rounded to one decimal. Unscored areas are left out rather than counted as zero, and the app always says what the number was computed from.
Example, with made-up scores: areas weighted 3, 2, 2 and 1 are scored 8, 6 and 7, and the fourth has no score yet. The fit is (8×3 + 6×2 + 7×2) ÷ 7 = 7.1, shown as "7.1 / 10 · from 3 of 4 areas". The submittal and the exports carry the same qualifier, "(3 of 4 areas scored)", so a partial score is never read as a complete one.
Can I change a score or add my own notes?
Yes. Press the "+" on any area to open "Your score" (it starts on "model's", then 10 down to 1) and a note field, "Your note on this area — stays on this computer". Press "Done", or Enter in the note, to close it. Your score replaces the model's in the fit score, the submittal and Compare, and a small "you" badge marks it; hover over the badge to see what the model read. The model's reading is kept underneath. You can also score an area the model never reached.
The "Your notes" box under the scorecard is for anything else. Your notes and area notes go into the submittal as your observations, never as the candidate's words.
What do Dual AI and the answer check add?
With Dual AI on, a second model's score appears beside each area as "B" and a number, and when the two are three or more points apart the area shows "The two models disagree — probe this area". The scorecard itself follows the first model. The answer check runs in the same scoring call and can flag an answer that reads as AI-written or read out, but it never changes a score and is never written into the submittal.
What happens to the scorecard after the call?
Press "End interview". If a must-ask question has not been covered, a dialog lists it and asks you to type END to finish anyway. The review screen then shows the scorecard beside the client submittal, which is written from the scorecard and the transcript; any area that was never scored is written as not assessed rather than left out.
The scorecard, its quotes and your notes are saved in the interview's folder on your computer. What reaches Interview Signal's servers is counts: how many scorecard areas the kit had, how many were scored, and the fit number. The security page lists every stored field.
Limits
- It reads only the words in the transcript. It does not hear tone and does not record video.
- A verified quote proves the candidate said those words, not that they are true. Check claims that matter.
- If the audio is poor, the transcript and therefore the quotes will be too. Run the audio check before the call.
- Scoring needs a connection. Press Evaluate while offline and the app says the scorecard will catch up when the connection comes back; recording carries on.
Questions people ask
Does Interview Signal make up quotes in the scorecard?
It is built so that an invented quote cannot survive. The model is told to copy the candidate's words exactly, and then every quote is checked against what the candidate actually said in the transcript. A quote that is not there word for word is deleted, and an area left with no quote has its score capped at 5 and is marked Probe further.
Why did a quote disappear from my Interview Signal scorecard?
It was not in the candidate's side of the transcript word for word, so the app removed it. When you press Evaluate, a message tells you how many quotes could not be matched to the recording and were dropped. Quotes shorter than four words are also refused, because a phrase like yes I did appears in almost any interview and proves nothing.
Can I override the AI score in Interview Signal?
Yes. Press the plus button on any scorecard area, pick your own score from 10 to 1 and add a note if you want. Your score replaces the model's in the fit score, the submittal and the Compare screen, a small you badge marks it, and the model's original reading is kept underneath so you can still see what it said.
How does Interview Signal calculate the fit score?
The fit score is a weighted average of the scorecard areas that have a score, on the same 1 to 10 scale, rounded to one decimal. Areas with no score are left out rather than counted as zero, and the app says how many areas the number comes from, for example 7.1 out of 10 from 3 of 4 areas.