Interview scorecard for DevOps engineers, with anchors for each competency
On this page
- The six competencies this scorecard covers
- The scorecard at a glance: weights and anchors
- The scorecard template
- Anchors for each competency
- Two work samples to use in the interview
- A filled example
- Adjusting for role and level
- Mistakes specific to scoring DevOps engineers
- Where this fits in the hiring process
- Questions people ask
A DevOps engineer interview scorecard should score six things: troubleshooting method under pressure, delivery pipelines and release safety, infrastructure as code and change review, observability and alerting judgment, security judgment, and incident communication with blameless follow-up. Below is a copy-ready scorecard with example weights, what a 1, 3 and 5 look like on a 1–5 scale, two work samples and a filled example. Cloud and Kubernetes certifications are pass/fail checks when the job requires them, never points, and the on-call schedule is a must-have asked the same way of everyone.
The five points follow the 1–5 interview rating scale: 1 is no evidence or evidence of the opposite, 3 is a clear example that meets the bar for the level, 5 is rare. The layout comes from the interview scorecard template. The full loop, with a live incident simulation, a pipeline design session and an infrastructure code review, is in the interview guide for DevOps roles; this page adds the anchors each row is scored against and two different exercises, an alert review and a postmortem rewrite. For application developers, use the interview scorecard for software engineers.
The six competencies this scorecard covers
- Troubleshooting method under pressure. Do they mitigate first, form a hypothesis and test it, rather than change three things at once?
- Delivery pipelines and release safety. Can they get code from merge to production quickly with a way back: tests at the right stages, progressive rollout, automatic rollback?
- Infrastructure as code and change review. Is infrastructure defined in code, reviewed like code, and changed in small steps that can be planned and reversed?
- Observability and alerting judgment. Do they page on what users feel, measured against an objective, rather than on every resource threshold?
- Security judgment. Secrets, least-privilege access, patching and audit trails as part of the design, not a later ticket.
- Incident communication and blameless follow-up. Do they keep people informed during an incident and turn it into fixes to the system, not blame?
The scorecard at a glance: weights and anchors
Weights are an example for a mid-level DevOps engineer at a software company. Agree yours with the engineering manager at intake and lock them before the first interview.
| Competency | Weight | 1 looks like | 3 looks like | 5 looks like |
|---|---|---|---|---|
| Troubleshooting method under pressure | 20% | Changes things at random, or keeps investigating while users are down with no thought of rollback. | Mitigates first where possible, states a hypothesis, checks it with one query or log, and narrows down step by step. | All of a 3, plus spots a second contributing change others missed and explains why the fix addresses the cause, not the symptom. |
| Delivery pipelines and release safety | 20% | Has only pushed through pipelines others built, or describes manual deploys with no rollback plan. | Built or changed a pipeline: build once and promote the same artifact, tests at the right stages, a rollback path that was used. | Introduced progressive delivery (canary or blue-green with health-checked rollback) and can show how delivery measures changed afterward. |
| Infrastructure as code and change review | 15% | Makes infrastructure changes by hand in a console with no record. | Writes and reviews infrastructure code, reads a plan before applying it, and keeps changes small and reversible. | Designed modules or guardrails others use (policy checks, drift detection) and reviews others' changes for risk, not just syntax. |
| Observability and alerting judgment | 15% | Pages on every threshold; cannot say what users experienced during an incident. | Pages on user-facing symptoms such as error rate and latency, demotes noisy alerts, and links alerts to runbooks. | Set service level objectives with the product owner and used the error budget to decide between features and reliability work. |
| Security judgment | 10% | Hard-codes secrets or grants wildcard permissions and sees no problem. | Uses a secrets manager, scopes access to what a service needs, and flags public exposure in review. | Built a control others rely on (short-lived credentials, automated access review) and explains the trade-off with developer speed. |
| Incident communication and blameless follow-up | 20% | Describes incidents in terms of who was at fault, or went silent while fixing. | Sent regular status updates during an incident and wrote a postmortem with contributing causes and owned, dated actions. | Led incidents across teams and changed the postmortem process itself so actions get done, with an example. |
A 1 on security judgment is a no whatever the total: people in this seat usually hold production access. How to set and lock weights is in weighted interview scorecard.
The scorecard template
Candidate: [name] Role: [DevOps / platform / cloud engineer], [level]
Interviewer: [name] Stage: [screen / incident / alert review / postmortem / manager]
Pass/fail (checked before scoring)
On-call rotation as stated in the job description: [ ] meets [ ] does not
Required certification, verified with the issuer (if any): [ ] yes [ ] n/a
Troubleshooting method under pressure (20%) Score 1-5: [ ]
Evidence: [mitigation first?, hypothesis, how it was checked]
Delivery pipelines and release safety (20%) Score 1-5: [ ]
Evidence: [pipeline built or changed, rollback used]
Infrastructure as code and change review (15%) Score 1-5: [ ]
Evidence: [tooling, review practice, change size]
Observability and alerting judgment (15%) Score 1-5: [ ]
Evidence: [alert review: what pages, what is demoted, SLO proposed]
Security judgment (10%) - a 1 is a no Score 1-5: [ ]
Evidence: [secrets, access scope, issues flagged]
Incident communication and blameless follow-up (20%) Score 1-5: [ ]
Evidence: [postmortem rewrite: causes found, actions owned and dated]
Weighted score: sum of (weight x score) / sum of weights scored = [ ]
Recommendation: [Strong yes / Yes / No / Strong no]
Anchors for each competency
The table gives the 1, 3 and 5. Here is the question that tests each row and what a 2 and a 4 sound like.
Troubleshooting method under pressure
Ask: "Tell me about the last production problem you diagnosed. What did you check first, and why?" More in interview questions for composure under pressure.
- 2: Found the problem, but by trying fixes until one worked.
- 4: Mitigated, then found the root cause with a clear chain of evidence.
Delivery pipelines and release safety
Ask: "Walk me through how code reached production in your last team, and what you changed about it."
- 2: Describes the pipeline accurately but changed only small parts of it.
- 4: Owned a meaningful change, such as cutting deploy time or adding automatic rollback, and can say what it fixed.
Infrastructure as code and change review
Ask: "Tell me about an infrastructure change that went wrong, or nearly did. What did the review or the plan show?"
- 2: Writes infrastructure code but applies changes without reading the plan, or reviews for style only.
- 4: Caught a risky change in review, theirs or someone else's, and explains the risk.
Observability and alerting judgment
Ask: the alert review below.
- 2: Removes the noisy alert but replaces it with nothing that would catch the real outages.
- 4: Proposes symptom-based paging and a first service level objective, and asks who should agree the target.
Security judgment
Ask: "How did a service in your last environment get its database credentials, and who could read them?"
- 2: Knows the right practice but describes exceptions they left in place.
- 4: Removed a standing risk (shared admin keys, broad roles) and explains how they did it without breaking developers.
Incident communication and blameless follow-up
Ask: the postmortem rewrite below, plus "Tell me about an incident you caused." More in interview questions for ownership.
- 2: Owns the incident but the follow-up stopped at "be more careful".
- 4: Owns it and describes a system change that made the same mistake harder to repeat.
Two work samples to use in the interview
Both are invented and the same for every candidate. They complement the incident simulation in the interview guide rather than replacing it. Tell candidates in advance and how to ask for an adjustment; see ADA accommodations in interviews.
Alert review (20 minutes). Share an invented paging summary for one service for one week: 41 pages, 35 of them from a "CPU above 80%" alert that cleared on its own within minutes, and two real customer-facing outages that were first reported by customers, not by an alert. The service has no service level objectives. Brief: "You join this team on Monday. What do you change about alerting, in what order?"
What to listen for: paging on what users feel (error rate, latency, availability) rather than a resource threshold; moving the CPU alert to a dashboard or ticket; and proposing a first objective to agree with the product owner. Google's SRE book defines a service level indicator as "a carefully defined quantitative measure of some aspect of the level of service that is provided" and an objective as "a target value or range of values for a service level that is measured by an SLI" (checked November 2026). A candidate does not need the vocabulary to score a 3, only the idea. This scores observability and alerting.
Postmortem rewrite (15 minutes). Hand over a short invented postmortem: a database migration locked a table for 40 minutes; "Root cause: the engineer ran the migration at peak time. Action item: the engineer will be more careful." Brief: "Rewrite the root cause and action items so the team learns something."
What to listen for: contributing causes in the system (no check for long-running locks, no rule on migration timing, no staging run on production-sized data); actions with an owner and a date; and no individual blamed. The SRE book's postmortem chapter says a blameless postmortem "must focus on identifying the contributing causes of the incident without indicting any individual or team" (checked November 2026). This scores incident follow-up.
A filled example
A made-up candidate for a mid-level DevOps engineer at a software company running containers on one cloud provider:
Troubleshooting method under pressure — 4. Described a memory leak after a release: rolled back first, then compared heap profiles between versions to find the cause.
Delivery pipelines and release safety — 3. Moved the team to building one container image and promoting it through environments; rollback is a redeploy of the previous tag.
Infrastructure as code and change review — 3. Writes and reviews infrastructure code daily and reads every plan before applying.
Observability and alerting judgment — 2. In the alert review, deleted the CPU alert but proposed no alert that would have caught the two outages until prompted.
Security judgment — 3. Uses the cloud provider's secrets service and scoped roles; flagged a public storage bucket in a past review.
Incident communication and blameless follow-up — 4. Rewrote the postmortem with three system causes and dated actions, and described an outage they caused themselves.
Weighted score: 3.25 ((4×20 + 3×20 + 3×15 + 2×15 + 3×10 + 4×20) ÷ 100). Recommendation: Yes, with alerting design probed with the hiring manager, since the team is about to set its first service level objectives.
Adjusting for role and level
| Role or level | Adjustment |
|---|---|
| Site reliability engineer | Raise observability to 25% and lower delivery; see site reliability engineer screening questions. |
| Platform engineer | Add a row for developer experience: an internal tool or template that other teams adopted, and how they knew it helped. |
| Cloud or infrastructure engineer | Raise infrastructure as code to 25% and add cost awareness to its anchors. |
| Junior or first DevOps role | Accept examples from personal projects, coursework or support work; score the two work samples at full weight; do not require on-call history. |
| Senior or staff | Add a row for sequencing large changes (a migration the team could absorb in steps); a 3 on delivery needs a change they led across teams. |
Mistakes specific to scoring DevOps engineers
| Mistake | Why it fails | Fix |
|---|---|---|
| Tool-name bingo | Rewards a candidate who has touched many tools over one who understands why a release fails | Score method and judgment; tools are context |
| Rewarding heroics | A story of fixing everything overnight alone can mean nothing was learned | Score what changed afterward so it would not happen again |
| Scoring "I'd roll back first" as a lack of curiosity | Mitigation before diagnosis is usually the right call when users are affected | Score it up, then ask how they would find the cause |
| Asking about family to judge on-call fit | The EEOC lists child care arrangements among questions that may be regarded as evidence of intent to discriminate | State the rotation and ask every candidate whether they can meet it |
| Treating delivery metrics as targets | DORA's metrics guide (last updated January 5, 2026) lists setting metrics as a goal as a common pitfall | Score a candidate who used measures to choose an improvement above one who chased a number |
| Scoring a certification | Double-counts a pass/fail fact | Verify it: AWS or Kubernetes |
DORA's guide, checked November 2026, names five software delivery measures: change lead time, deployment frequency and failed deployment recovery time for throughput, and change fail rate and deployment rework rate for instability. A candidate who can describe how their team measured any of them, and what they changed as a result, has strong evidence for the delivery row.
Where this fits in the hiring process
The DevOps engineer phone screen and the deeper screening questions place the candidate (DevOps, SRE, platform or cloud) and check the on-call fit. The incident simulation from the interview guide, the two work samples above and the manager conversation fill in the six rows. Keep the rows consistent with the must-haves in the DevOps engineer job description template, and carry the weakest scored row into the first month of the 30-60-90 day plan for DevOps engineers.
If you interview with Interview Signal, the desktop app we make, these rows can become the scorecard areas in its kit, each weighted 1, 2 or 3 with rubric lines on a 1–10 scale; each score carries the candidate's own quotes, checked word for word against the transcript, and an area with no surviving quote is capped at 5 and marked "Probe further" (see how the scorecard with verified quotes works). Whatever you use, have each interviewer score before reading the others, then compare in a calibration session.
Questions people ask
What should a DevOps engineer interview scorecard measure?
Six things for most DevOps seats: troubleshooting method under pressure, delivery pipelines and release safety, infrastructure as code and change review, observability and alerting judgment, security judgment, and incident communication with blameless follow-up. Weight them for the seat and write what a 1, 3 and 5 sound like for each before the first interview.
Should cloud or Kubernetes certifications add points on a DevOps scorecard?
No. If the job requires a certification such as an AWS or Kubernetes credential, verify it with the issuer as a pass/fail line. The scored rows should measure how the candidate debugs, ships and recovers, which a certification exam does not show on its own.
How do you ask a DevOps candidate about on-call without asking about their family?
State the rotation and response expectations from the job description and ask every candidate the same question: can you meet this on-call schedule? Do not ask about children, child care or a partner's job; the EEOC lists child care arrangements among pre-employment questions that may be regarded as evidence of intent to discriminate.
How is a DevOps scorecard different from a software engineer scorecard?
A software engineer scorecard weights problem decomposition, coding and system design. A DevOps scorecard weights what happens after the code is written: how it reaches production safely, how failures are detected and paged, how incidents are run and learned from, and how infrastructure and access are kept secure.