How to compare candidates interviewed on video and in person

You can compare them, but do not assume a 3 on video means the same as a 3 in person. The research is mixed: a 2016 meta-analysis by Blacksmith, Willford and Behrend found interviewer ratings were lower in technology-mediated interviews (d = -0.41), while a 2026 meta-analysis by Moon and Park found no reliable difference in interview ratings. The dependable fix is in the design, not a correction factor: interview every finalist at a stage in the same medium, and where that is impossible, use the same structured questions and anchored scale, score only what the candidate said, and check whether the medium explains the gap before you decide.

The general method for comparing finalists is in how to compare candidates after interviews. This page covers the one variable that method assumes away: some finalists were in the room and some were on a screen.

Do candidates score differently on video than in person?

Two meta-analyses have looked at interviewer ratings, and they point in different directions.

What both support is narrower than "video lowers scores." The effect, if any, varies a lot from one setting to the next, which means you cannot add half a point to every video interview and call it fair. Blacksmith and colleagues draw the practical conclusion themselves: organizations should be "especially wary of varying interview mode across applicants," because inconsistent administration could create fairness problems.

Why might the medium change a score?

Blacksmith and colleagues offer several explanations, which they present as hypotheses rather than findings: video and phone remove or distort the cues candidates use to manage the impression they make; eye contact is hard through a camera; lag makes turn-taking awkward; the informal warm-up before an in-person interview, such as the walk from reception, is missing; and candidates may be more anxious. Three more come from how interviews are usually run:

• Unscored time. In-person candidates often get a tour, lunch or a coffee with the team. Those moments shape impressions but are not on anyone's scorecard.
• Different things are visible. In the room you see posture and energy; on video you see a kitchen, a bad connection or a lagging camera. None of these is a competency.
• Halo. A strong overall impression bleeds into specific ratings, and the in-person format gives it more to work with. See the halo effect in interviews.

How do you keep the medium from deciding?

1. Fix the medium per stage. Decide at intake that, for example, screens are by video and finals are in person, for everyone. The U.S. Office of Personnel Management's structured interview guide asks for the same seating arrangements and the same interview time for all candidates; the medium is the larger version of the same rule.
2. When you must mix, standardize everything else. Same questions in the same order, the same time allowed, the same interviewers where possible, and the same anchored scale. Write on the scorecard which medium was used.
3. Score only the structured part. Nothing from the tour, the lunch or the small talk goes on the scorecard, in either medium.
4. Take presence out of the anchors unless the job requires it. If "presence" or "energy" is a criterion, define what it looks like in observable terms that both media can show, such as structuring an answer or handling an interruption.
5. Do not penalize the connection. When video or audio drops, repeat the question and give the time back. How to run a video interview has a script for this.
6. Re-read close calls blind to the medium. For finalists within a point of each other, have someone who did not interview them score the written answers or transcripts against the anchors, without knowing who was in the room. Get consent before recording anyone.
7. Give a same-medium follow-up when it matters. If the decision hinges on a competency where the medium might have played a part, give both finalists a short follow-up in the same format, with the same questions.

Sometimes a candidate needs a different format, for example video instead of travel, as a reasonable accommodation. That request is not a reason to compare them less carefully; it is a reason to make the comparison rest on the anchors. See ADA accommodations in interviews.

How can you check whether the medium is affecting your own scores?

Once you have enough interviews, compare average scores by medium for the same stage and the same interviewers. An example, with invented numbers: across 40 final interviews for one job family, candidates interviewed in person average 3.1 out of 4 and candidates interviewed on video average 2.7. Before you conclude anything, check three things:

• Who chose the medium. If local candidates came in person and remote ones were on video, the groups may differ for reasons that have nothing to do with the medium, such as experience or role level.
• Which competencies differ. A gap concentrated in "communication" or "presence" points at the medium. A gap spread evenly across technical competencies points at the candidates.
• Which interviewers. If one interviewer does all the video interviews, you may be measuring that interviewer, not the medium. Comparing candidates who had different interviewers covers that case.

With small numbers, treat the result as a prompt to look at the scorecards, not as a finding. The broader drift check is in how to score interviews consistently.

A worked example: two finalists, two formats

Invented candidates, scored on a 1 to 4 anchored scale:

On totals, Sam leads by one point, all of it from communication. The evidence under that score describes the medium, not the anchor. The fix is to go back to the anchor ("explains a complex point so a non-specialist can act on it") and look for answers that show it. If the notes cannot settle it, both finalists get the same 20-minute follow-up on video, with one explanation exercise scored on that anchor. The decision then rests on evidence gathered the same way.

One tool-specific caution. In Interview Signal, which we make, an in-person interview puts both voices on the interviewer's microphone, so quotes cannot be verified against the candidate: areas come back as "Probe further" and scores above 5 are capped. Its in-person scorecards are not comparable like for like with its video ones; for in-person rounds, score with your own written anchors.

A checklist before you compare across media

[ ] Medium fixed per stage at intake, and recorded on each scorecard
[ ] Same questions, order, time and anchored scale in both media
[ ] Nothing scored from tours, meals or small talk
[ ] No anchor that only one medium can show
[ ] Connection problems repeated, not penalized
[ ] Close calls re-read blind to the medium (with consent)
[ ] Same-medium follow-up offered to every finalist still in the running
[ ] Score gaps by medium checked against who chose the medium

Order effects do not stop at the medium: a candidate seen right after a strong or weak one can be rated against that person instead of the anchors. See the contrast effect in interviews.