Interview guide for QA engineers: a loop with test design, a bug report and an automation review

A QA engineer's job is to find out what is wrong before customers do, and to make that cheap enough that the team keeps doing it. That takes three different skills that most loops blur together: deciding what to test when there is never time to test everything, finding and describing bugs so developers can fix them quickly, and building automation that the team trusts rather than reruns until it goes green. A generic developer loop misses the first two entirely. This guide gives engineering managers a loop for a mid-level QA automation engineer: five stages, owners, a test design session, a live bug hunt, a flaky test review, weights and a decision rule. Adjustments for manual testers, SDETs and performance testers are at the end.

The first call, including how to tell who built a framework from who ran one, is in QA engineer phone screen questions. The general engineering loop this one borrows from is the interview guide for software engineers.

The loop at a glance

Stage 1: the screen (30 minutes)

• "How does code get from a pull request to production on your team, and where do your tests run?" Listen for: a clear pipeline and their own part in it.
• "Roughly how many automated tests do you own, how long does the suite take, and how often does it fail for reasons that are not bugs?" Candidates who own a suite know the flake rate.
• "What bug that you found are you proudest of?" Listen for: how they found it, not just that it was serious.

Stage 2: the test design session (45 minutes)

Give a one-page spec for a feature the candidate can understand without your domain knowledge. Example: "Customers can apply one discount code at checkout. Codes can be percentage or fixed amount, may have a minimum order value and an expiry date, and cannot reduce the total below zero." Ask: "You have two days before release. What do you test, in what order, and what do you automate?"

• Strong answers start with risk: money calculations, expiry at time-zone boundaries, stacking codes despite the rule, and what happens when the cart changes after a code is applied.
• Technique without jargon: boundary values around the minimum order and zero total, equivalence classes for code types, and a few negative cases.
• Questions back: a good candidate finds gaps in the spec, such as "What happens to the code if an item is returned?" Credit those.
• Automation split: calculation rules at the unit or API level, a small number of end-to-end checkout tests, and exploratory time for the rest.

Stage 3: the live bug hunt and report (45 minutes)

Give the candidate 25 minutes with a small web app you control, such as a test build of a sign-up and checkout flow, with five planted bugs of different severity. Write the bug list and expected severity once and use the same build for every candidate.

Example planted bugs

• Critical: the total goes negative when a fixed discount exceeds the order value.
• High: the sign-up form accepts an email that already exists and creates a duplicate account.
• Medium: an error message shows a raw server exception.
• Low: a field label is misaligned at narrow widths.
• Hidden: a date picker accepts February 30.

Then the candidate writes one bug report in 15 minutes for the most important issue they found. Score the report on reproducible steps, expected versus actual results, environment details, severity with a reason, and evidence. A developer on the panel should be able to reproduce it without asking a question. Missing the critical bug entirely is a gate failure for this role.

Stage 4: the automation review (45 minutes)

Share a short invented test file in your framework and language: an end-to-end test that uses fixed sleeps, depends on data created by another test, asserts on a full page of text, and has a hard-coded password. Tell the candidate it fails about one run in ten.

• Diagnosis: identifies the sleeps and shared test data as likely flake sources and explains how to confirm which one it is.
• Design: proposes explicit waits, isolated test data, narrower assertions, and moving the secret into configuration.
• Judgment: "Should this check be an end-to-end test at all?" Strong candidates push some of it down to an API test.

If you prefer a hands-on version, have the candidate fix the test in a prepared repository with the interviewer watching. For take-home versions, read how to evaluate a take-home assignment first.

Stage 5: the release-risk conversation (45 minutes)

1. "It is release day. Two medium bugs are open and the product manager wants to ship. What do you do?" Listen for: describing the risk in customer terms and letting the right person decide, rather than blocking or rubber-stamping.
2. "Tell me about a time a bug you should have caught reached production." Listen for: ownership and a change to the process, not blame on the developers.
3. From the developer: "How would you want to work with me on a feature from the start?" Listen for: involvement in design and acceptance criteria, not just testing at the end.
4. "How do you decide a test is no longer worth keeping?" Listen for: maintenance cost versus the risk it covers.

Scorecard competencies and weights

Example weights for a mid-level QA automation engineer. Lock yours before the first candidate.

Build the sheets in the scorecard builder, which checks the weights add up to 100.

Legal points for QA hiring

Checked against the linked primary sources as of October 2026. Not legal advice.

• Sponsorship questions. In a September 9, 2022 technical assistance letter, the Justice Department's Immigrant and Employee Rights Section restated two questions it has identified as appropriate: whether the applicant is legally authorized to work in the United States, and whether they will now or in the future require sponsorship for employment visa status. It cautioned employers against adding further pre-screening questions.
• Accommodations for timed exercises. Under 29 CFR 1630.11, tests must be selected and administered so that, for an applicant whose disability impairs sensory, manual or speaking skills, results reflect the skill being measured rather than the impairment, unless that skill is what the test measures. Ask every candidate in advance whether they need an adjustment, such as extra time or a screen reader.
• Exercises as selection procedures. Under the Uniform Guidelines at 29 CFR 1607.3, a selection procedure with adverse impact is treated as discriminatory unless it is validated. Keep exercises tied to the real job and identical for every candidate.

The decision rule

1. Scorecards and the written bug report go in before the debrief.
2. Gate: the candidate found the critical planted bug.
3. Floor: test design at 3 or above.
4. Weighted total: in this example, 2.8 or higher on a 1–4 scale is an offer.
5. Tie-breaker: when two candidates are close, prefer the one whose bug report a developer could act on fastest.

Adjusting the loop

Mistakes that weaken QA loops

• "How would you test a pen?" It is a vocabulary test. Use a real spec.
• No bug report. Writing a clear report is half the job.
• Algorithm rounds. They filter for something QA engineers rarely do.
• Treating QA as a gate, not a partner. Score how the candidate works with developers.

After the hire, adapt the 30-60-90 day plan for QA engineers.