When interview and assessment results disagree: which do you trust?
On this page
Trust neither by default. First check what each result measured against the job's must-haves, and how dependable each was: a structured interview scored against anchors is stronger evidence than an unstructured chat, and a work sample scored with a written rubric by two reviewers is stronger than one person's impression. Then apply the combination rule you set before the interviews. If the conflict sits on a must-have, run one short follow-up aimed at that competency, offered on the same terms to any candidate in the same position, rather than breaking the tie by instinct.
The hire or no-hire decision framework covers interviewers who disagree with each other. This page is about a different conflict: two kinds of evidence about the same candidate, an interview and an assessment, pointing in opposite directions.
Which is more predictive, the interview or the assessment?
The most recent large review is Sackett, Zhang, Berry and Lievens (2022), Journal of Applied Psychology, which re-estimated the validity of common selection procedures after arguing that earlier corrections had overstated them. Its mean operational validity estimates, as correlations with job performance:
| Selection procedure | Mean validity (Sackett et al., 2022) |
|---|---|
| Structured interviews | .42 |
| Job knowledge tests | .40 |
| Work sample tests | .33 |
| Cognitive ability tests | .31 |
| Assessment centers | .29 |
| Situational judgment tests (knowledge format) | .26 |
| Unstructured interviews | .19 |
Three points from the paper matter more for a single candidate than the ranking does:
- The averages vary by setting. The structured interview has the highest mean (.42) but also a large residual standard deviation across studies (.19), so in a given organization its validity can be well below the average. The authors say plainly that an employer "cannot count on the mean value" applying to them.
- Job-specific measures do best. The strongest predictors in their re-analysis were structured interviews, job knowledge tests, empirically keyed biodata and work samples, all of them specific to the job. An assessment or an interview that does not match the actual job loses that advantage.
- The tools differ in adverse impact. Work samples, job knowledge tests and cognitive ability tests show substantial group mean differences, while structured interviews show much smaller ones. Leaning on the assessment every time the two disagree can change who gets hired, not only how well you predict.
So a well-run structured interview is not weaker evidence than a test by default, and an unstructured interview is weak evidence against almost anything. The question is how well each was done in your process.
Did they measure the same thing?
Many "disagreements" are two tools measuring two different competencies. Map each result to your job analysis before comparing them.
| Pattern | Common explanations to check |
|---|---|
| Strong interview, weak work sample | Talks well about work the team did ("we") rather than work they did; the task was unclear or too long for the time given; low effort on an unpaid take-home; the task tested a skill the interview never asked about; the interview rewarded fluency, not the anchor |
| Weak interview, strong work sample | Interview anxiety; questions never reached the competency the task tested; unstructured interview; a take-home completed with help that the role would not have (check in a walkthrough) |
| Strong interview, low test score | The test measures a general ability the interview did not assess; test conditions (timing, device, accommodation not provided); a cut score set without reference to the job |
| Weak interview, high test score | Ability is there, the job-specific experience or behavior is not; or the interviewer scored delivery rather than content |
If the two measured different must-haves, there is no conflict to resolve: the candidate met one and missed the other, and your rule says what that means. If they measured the same competency and still disagree, look at how dependable each result was.
How dependable was each result?
| Check | Interview | Assessment |
|---|---|---|
| Built for this job? | Questions written from the job analysis | Task or test chosen for this job's must-haves, not a generic screen |
| Scored against something written? | Anchored scale per competency | Rubric or scoring key written before anyone took it |
| Scored independently? | Scorecard submitted before any discussion | Reviewers blind to the interview results; two reviewers for borderline cases |
| Same conditions for everyone? | Same questions, time, medium | Same time limit, instructions, tools allowed, accommodations offered |
| Evidence you can reread? | Candidate's words under each score | The submission or answer sheet, with the rubric applied line by line |
The result with more "no" answers carries less weight in this case. If both are weak, the fix is the process, not this candidate. How to evaluate a take-home assignment covers rubrics and blind review; what is a structured interview covers the interview side.
What rule should decide?
Write the rule before the first interview. In their meta-analysis, Kuncel, Klieger, Connelly and Ones (2013, Journal of Applied Psychology) found a consistent and substantial loss of validity when selection data were combined by holistic judgment, even by experts who knew the jobs; for job performance, mechanical combination improved prediction by more than 50%. A simple written rule is the mechanical method most teams can use. Give each must-have an owner:
EVIDENCE OWNERSHIP (example rule, written at intake)
Competency Primary evidence Secondary evidence
Data modelling (must) Work sample Interview Q4
Stakeholder mgmt (must) Interview Q2, Q3 none
Written communication Work sample write-up Interview Q6
Rules
- Primary evidence decides the score for that competency.
- If secondary evidence disagrees by 2+ points on a must-have:
one targeted follow-up on that competency, then decide.
- Assessment results are not retaken unless the retake policy
(same for everyone) allows it.
- The rule changes only between candidates, never mid-decision.Some teams prefer a weighted total across tools, others a minimum per tool. Either can work; what fails is choosing after you know which candidate each choice favors.
How do you run a targeted follow-up?
- Aim at one competency. "Can she model this data under realistic constraints?" not "let's get another feel for her."
- Use a different method from the one in doubt. If the take-home is suspect, walk through it live and ask the candidate to change one part; if the interview is suspect, give a short supervised exercise. The take-home guide's walkthrough section has questions.
- Score it on the same anchors, written before the follow-up.
- Offer the same step to any other finalist with a disputed result on that competency, so the evidence stays comparable.
- Decide straight afterwards. A follow-up that leads to another follow-up is postponing the decision.
References can fill the gap too, if you ask a former manager about the specific competency, with an example, rather than for general impressions.
A worked example
Invented case. A data analyst finalist scores 4 out of 4 on "analytical problem solving" in the interview, with a clear account of how she rebuilt a churn model, and 2 out of 4 on the same competency in a 90-minute take-home, where the rubric shows she skipped the data quality step and reported an inflated accuracy figure.
Same thing measured? Yes, both target analytical problem solving. Dependability: the interview was structured and anchored, but the "we" in her account was never probed; the take-home was rubric-scored by two blind reviewers who agreed.
Rule: the work sample is primary evidence for this competency; a 2-point gap on a must-have triggers one follow-up.
Follow-up: a 30-minute live walkthrough of her submission, asking what she would check before trusting the accuracy figure. She identifies the leakage problem within minutes and explains how she would fix it, and says she ran out of time. Scored 3 on the same anchor. The other finalist with a split result on this competency is offered the same walkthrough.
Decision record: both scores kept, the follow-up and its score added, the rule cited. Nobody's instinct was needed to break the tie.
What should you not do?
- Do not discard the result you dislike. Ignoring a test because the interview was charming, or ignoring the interview because "the numbers don't lie," are the same mistake.
- Do not invent a retake for one candidate. Retakes follow a policy that applies to everyone.
- Do not move a cut score mid-search. If the assessment's cut score was wrong, change it for every candidate and record why.
- Do not ignore adverse impact. Under the federal Uniform Guidelines, a selection procedure that has an adverse impact on a race, sex or ethnic group "will be considered to be discriminatory" unless it has been validated (29 CFR 1607.3). If an assessment regularly overrides interviews for one group, check its pass rates. This is not legal advice.
Once each finalist's evidence is reconciled, compare them side by side with the matrix in how to compare candidates after interviews, and see types of pre-employment assessments if you are choosing an assessment for the next search.
Questions people ask
A candidate interviewed well but did badly on the work sample. Which do I trust?
Neither by default. First check whether the two measured the same competency, and how dependable each result was: a structured, anchored interview versus a chat, a rubric-scored work sample versus a gut read. Then apply the rule you set before the interviews. If the conflict sits on a must-have, run one short follow-up aimed at that competency instead of breaking the tie by instinct.
Are structured interviews more predictive than work samples?
On average, in the 2022 meta-analytic review by Sackett, Zhang, Berry and Lievens in the Journal of Applied Psychology, yes: structured interviews had the highest mean operational validity at .42, against .33 for work sample tests. But the authors stress that validity varies across settings and that an employer cannot count on the average applying to them, so the averages do not settle a single candidate's case.
Should I average the interview score and the assessment score?
Only if you decided to before seeing any results, and only for competencies both actually measured. Research on combining selection data, such as Kuncel and colleagues' 2013 meta-analysis, found that applying a consistent rule predicted job performance better than holistic judgment. The rule can be an average, a weighted sum or a gate per competency; what matters is that it is written down and applied to every candidate.
Can I let a candidate retake an assessment they failed?
Only under a retake policy that applies to everyone, written before the search. Letting one candidate you like retake a test, or quietly ignoring a result, makes the comparison unequal. If you think the assessment itself was flawed, fix it for every candidate and say so in the decision record.