Behaviorally anchored rating scales (BARS) for interviews: how to build one, step by step

A behaviorally anchored rating scale (BARS) is a rating scale in which every point is described by an example of real job behavior that people who know the job agree belongs at that level, so an interviewer rates an answer by matching it to examples rather than to adjectives. The method is traced to Smith and Kendall (1963, Journal of Applied Psychology). You build one in five moves: collect critical incidents from subject matter experts, have a second group sort them back into competencies, have that group rate how effective each one is, keep the incidents they agree on, and place them on the scale as anchors.

If you need anchors today, the behavioral anchors library has ready-made ones, and interview rating scale covers choosing the number of points. This page is the full build: the steps, the agreement arithmetic, a worked example and the cost.

What is a behaviorally anchored rating scale, and how is it different?

In interviews the anchors describe what a candidate reports having done (behavioral questions) or would do (situational questions), not behavior an interviewer watched on the job. The US Office of Personnel Management's Structured Interviews: A Practical Guide (September 2008) builds interview rating scales from the same parts: one proficiency range for every competency, at least three levels and ideally five to seven, and behavioral examples written by experts for each level.

Where does the method come from?

Smith and Kendall's 1963 paper, "Retranslation of expectations: An approach to the construction of unambiguous anchors for rating scales", gave the method its key check. "Retranslation" works like checking a translation by translating it back: a second group, who did not write the examples, assigns each one to the dimension they think it shows. Examples that do not land back where they started are ambiguous and are dropped. The examples themselves are usually gathered with the critical incident technique described by Flanagan (1954, Psychological Bulletin): specific accounts of effective and ineffective behavior, each with the situation, the action and the outcome. OPM's guide cites Flanagan and uses both steps, telling developers to "eliminate critical incidents not clearly linked to a competency and those associated with multiple competencies."

How do you build a BARS for an interview, step by step?

1. Pick the competencies. Start from a job analysis or a recent intake. OPM notes a structured interview typically assesses four to six competencies. Define each in one sentence.
2. Choose two groups of experts. People who do or supervise the job well, at or above the level you are hiring. OPM suggests about six or seven subject matter experts for writing interview questions; use a second, separate group for steps 4 and 5.
3. Collect incidents. Each expert in the first group writes effective and ineffective incidents for each competency: what led up to it, what the person did, what happened. OPM's Appendix C has effective and ineffective incident forms you can copy. Aim for more incidents than you need, since some will be dropped.
4. Retranslate. Edit the incidents into one-sentence behaviors, shuffle them, and ask each expert in the second group, working alone, to assign each one to a competency. Keep an incident only if most of them place it where you intended.
5. Scale. The same group, still working alone and often in the same sitting, rates how effective each incident is on your scale. Compute the mean and the standard deviation for each incident. A wide spread means the experts disagree about how good the behavior is, and the incident goes.
6. Place the anchors. Put each surviving incident at the scale point nearest its mean. Choose one or two per point. If a point has no incident, collect more for that level rather than inventing one in the room.
7. Pilot. Run the questions with colleagues, as OPM's step 6 suggests, and check that answers spread across the scale. Then calibrate interviewers on recorded or written answers; see interview calibration.
8. Document it. Keep who took part, the incidents, the agreement figures and the decisions. OPM's step 8 asks for the same, and you will want it when someone asks why an anchor says what it says.

Set the keep-or-drop thresholds before the second workshop, so nobody bends them to save a favorite example. The thresholds in the example below are ours, for illustration; there is no single standard.

Worked example: scaling six incidents for one competency

Invented data. Competency: handling an escalated customer complaint, for a support team lead. Eight experts in the second group sorted and rated six incidents on a 1 to 5 scale. Example thresholds: keep an incident if at least 70% of experts assigned it to this competency and the standard deviation of its effectiveness ratings is 1.0 or less.

The arithmetic for incident F: the ratings sum to 25, so the mean is 25 ÷ 8 = 3.13. The standard deviation is the square root of the average squared distance from the mean (dividing by 7, one fewer than the number of raters), which comes to 0.35: the experts nearly all said 3. Incident C has the same average as a 3 but a spread of 1.31, because some experts saw a refund as good service and others as giving money away before diagnosing anything. An anchor that splits the experts will split your interviewers too.

What does the finished scale look like?

COMPETENCY: Handling an escalated customer complaint
Definition: Takes ownership of a complaint, finds out what happened,
gets the right people involved and keeps the customer informed to
resolution.
Question: "Tell me about a complaint that reached you after the
customer was already angry. What did you do, and what happened?"

5  Gathers facts from the customer and the records, escalates with a
   written summary, and updates the customer on a schedule until it
   is resolved.
4  Responds quickly, explains what went wrong, and commits to a time
   for the next update.
3  Apologizes, routes the issue to the right team, and tells the
   customer when to expect an answer.
2  Acknowledges the complaint but leaves no date or next step.
1  Passes the complaint on and closes it without ownership.

Rate the point whose anchor the answer most resembles. Anchors are
examples, not a checklist; the candidate's situation will differ.

The last line follows OPM's instruction that behavioral examples are a general guide rather than an absolute, because candidates' experiences differ.

Is a BARS worth the effort for interviews?

It costs real time. An example with invented but plausible numbers: seven experts spend 90 minutes writing incidents (10.5 hours), eight more spend 75 minutes sorting and rating them (10 hours), and a facilitator spends four hours editing and compiling. That is about 24.5 hours of expert time for one role's competency set, before the pilot.

• Worth it for roles you hire again and again, roles with many interviewers who need to score the same way, and panels where disagreements about anchors keep coming back.
• Probably not worth it for a one-off hire. Write anchors with the hiring manager, borrow from the anchor library, and calibrate once.

The broader evidence is about structure in general rather than BARS alone. Campion, Palmer and Campion (1997, Personnel Psychology) note that virtually every previous review had concluded that structuring the selection interview improves its psychometric properties, and they identify fifteen components of structure. A carefully built anchored scale is one of the cheaper ways to add it.

How do you keep a BARS working after launch?

• Collect new incidents from real interviews. When an answer does not fit any anchor, write it down; that is next year's retranslation material.
• Check the score spread. If nobody ever scores a 1 or a 5, the end anchors may be unrealistic. The 1–5 rating scale page covers the decision rules that keep the middle from becoming the default.
• Re-run steps 4 and 5 when the job changes, for example after new tools, a new customer segment or a reorganization.
• Keep anchors about job behavior only. Nothing about age, accent, family or other personal characteristics belongs on any scale; see illegal interview questions for the topics interviewers must avoid.

Common mistakes