Skip to content
    ← Back to blog

    Behavioral Interview Scoring: How to Rate Answers Fairly

    Behavioral interview scoring means rating every candidate’s answers against the same job-related competencies, on the same scale, with written descriptions of what each score looks like. You choose four to six competencies, ask each candidate the same questions, score each answer against behavioral anchors right after it is given, and combine the ratings with a fixed rule. This guide shows how to do each step, where scoring tends to go wrong, and how to keep it manageable when you have hundreds of applicants.

    Resyme
    Make Behavioral Interviews Easier to Compare
    Resyme uses behaviorally anchored interviews and objective candidate comparisons to help recruiters assess answers more consistently.

    TL;DR

    • Score 4 to 6 competencies drawn from a job analysis, and use one rating scale (usually 1 to 5) for every candidate.
    • Write anchors as observable behavior, with examples of poor, acceptable and excellent answers, not labels like “good communicator.”
    • Take notes under Situation, Task, Action and Result, then score right away and before talking to anyone else.
    • Weight competencies equally unless you can document a reason not to, and have at least two people rate independently.
    • At high volume, standardize the first round and check the strongest claims against other evidence before the final decision.

    Table of Contents

    What behavioral interview scoring is

    A behavioral interview rests on a simple premise: past performance is the best predictor of future performance. The University of Washington’s hiring guidance says interviewers should aim to draw out a STAR answer (Situation or Task, Action, Result) or the similar EAR format. Scoring is what turns that conversation into data you can compare.

    Without scoring, interviewers fall back on impressions. With it, each answer gets a number that is tied to evidence from the candidate’s own words.

    The core parts are the same everywhere:

    • The same questions. A structured behavioral interview, as one Missouri state leadership playbook puts it, asks every candidate the same questions in the same order.
    • A common scale. The U.S. Office of Personnel Management (OPM) calls a common rating scale for all candidates a key component of the structured interview procedure.
    • A standard to measure against. The same playbook says candidates are compared with an ideal hire, not with one another. Comparison between candidates comes after the scoring.
    • Trained, calibrated raters. Google’s re:Work guide lists four aspects of structured interviewing: vetted, role-relevant questions, comprehensive feedback, standardized rubrics, and interviewer training and calibration.

    If you are still deciding between formats, our guide to structured vs. unstructured interviews covers the trade-offs.

    Step 1: Choose the competencies you will score

    Scoring is only as good as what you score. Start with the job, not with a list of generic soft skills.

    OPM says questions should reflect competencies derived from a job analysis, and that a structured interview typically assesses four to six of them unless the job is unique or senior. That limit matters. Ten competencies in forty-five minutes means shallow answers and noisy scores.

    A workable way to pick them:

    1. Ask the hiring manager and two strong performers in the role what separates good from average in the first year.
    2. List the behaviors behind those answers, such as “escalates a problem before it reaches the customer.”
    3. Group them into four to six competencies, for example ownership, problem-solving, collaboration, communication and a role-specific skill.
    4. Drop anything you cannot observe in an interview.

    The London School of Economics suggests a mix in which around 70% of questions are behavioral and 30% technical. That is a sensible default for roles where craft skills matter alongside conduct.

    The University of Washington frames competencies as more than what someone knows: they are how a person applies knowledge to meet job expectations. That is why a behavioral question about a real situation tells you more than a quiz. For a broader view of how employers define the skills that matter, this skills-based hiring guide is a helpful companion when you are drafting your list.

    For deeper detail, see our posts on competency-based interviewing and defensible interview question design.

    Step 2: Build a rating scale with anchors

    The scale is the ruler. The anchors are the markings on it.

    Pick the range

    OPM’s guide says to decide on one proficiency range for all competencies, for instance 1 to 5 with 5 the most proficient. It asks for at least three levels, aims for five to seven, and labels at least three of them. RIT’s guidance agrees that behavioral scales should be somewhat complex, such as 1 to 5, because that allows more variance and better comparison between candidates. Its example runs from “Far Exceeds Requirements” to “Significant Gap.”

    Write anchors as behavior

    An anchor describes what a score looks like in practice. SHRM quotes recruiter Victoria Gates saying the scale should be anchored to observable, job-related behaviors rather than vague labels, and that interviewers need examples of poor, acceptable and excellent responses for each competency. Google’s guide describes the same practice: it documents what a poor, borderline, solid and outstanding answer would cover for each attribute a question tests.

    Here is a sample for “ownership,” using the question “Tell me about a time you found a problem nobody had asked you to fix.”

    Score Label What the answer shows
    1 Significant gap No real example, or describes noticing the problem and leaving it to someone else
    2 Below requirement Acted only after being asked; vague about what they personally did
    3 Meets requirement Raised or fixed the problem on their own; clear action, modest or unmeasured result
    4 Exceeds requirement Took initiative, involved the right people, and can state a concrete result
    5 Far exceeds requirement Fixed the immediate issue, found the root cause, changed the process, and can show lasting results

    AIEH recommends drafting anchors at the extremes first and filling in the midpoints afterward. It also suggests piloting the rubric with 5 to 10 candidates and multiple raters, then documenting it with version control. Extremes are easier to describe, and the midpoints are where raters disagree most.

    What a complete rubric contains

    AIEH lists the parts of a rubric that meets selection-research standards: 2 to 5 dimensions per interview, a 5-point ordinal scale, behavioral anchors for each point, probes and follow-up guidance, independent-rating instructions, and decision-aggregation rules. Our walkthrough of behaviorally anchored rating scales goes through the build in six steps, and the structured interview templates give you a starting point.

    Step 3: Score each answer on the evidence

    The most common scoring failure is scoring the person instead of the answer. Three habits prevent it.

    Take notes under STAR

    TLDV recommends factual notes under each STAR element, then comparing that evidence with the written anchors. It adds that the action is the key evidence, while situation and task mostly give context for judging how difficult the problem was. A candidate who handled a hard situation and a candidate who handled an easy one can describe similar actions, and the context is what separates them.

    Write what was said, not what you concluded. “Rebuilt the onboarding checklist after three missed deadlines; onboarding time fell from 10 days to 6” is a note. “Seems proactive” is not.

    Score straight away

    OPM’s guide says that right after the candidate leaves, the interviewer should review notes and rate the candidate. Ratings should be defensible and supported by the notes, with examples of actual answers and an explanation of how they apply to the competency. TLDV adds that you should score before conferring with anybody else, so one loud opinion does not set the room.

    Write a one-line rationale

    Every score gets a reason tied to something the candidate said. This takes thirty seconds per answer and makes the score reviewable later. It also exposes the moments when you are scoring confidence or likability instead of evidence.

    A worked example

    Suppose a candidate for a customer success role answers the ownership question like this: a client’s reports stopped syncing, nobody on support owned the integration, so she traced the issue to a changed field name, patched it, and wrote a checklist that support now uses. She says tickets on that issue stopped.

    Situation and task: clear, moderately hard. Action: independent, specific, with a root cause. Result: process change, but the effect is described, not quantified. Against the sample table, that is a 4. A 5 would need a measured, lasting result. Your rationale reads: “Found root cause, wrote checklist, said tickets stopped; no figures given.” Another rater can now agree or disagree with a specific point.

    Step 4: Combine scores across questions and interviewers

    Weight competencies equally by default

    OPM recommends equal weights for each competency in a structured interview, calling it the most effective and defensible course when there is no clear, documented rationale for doing otherwise. If you weight differently, for instance counting a technical skill double, write down why before the first interview.

    Rate independently, then compare

    The International Test Commission’s guideline says raters should work independently of one another, that at least two raters should be used per assessment where cost and availability allow, and that agreement between raters should be measured statistically. It also notes that scoring open-ended items such as interviews usually involves some subjectivity and tends to be less reliable than multiple-choice scoring. Trained assessors, clear rating instructions and practice on example answers reduce that gap.

    Expect disagreement, and close it with process

    A study of 112 candidates interviewed by 28 interviewers found inter-rater reliability of 0.55. Follow-up discussions pointed to unclear roles, different readings of the job specification and inconsistent use of the rating system. Those are process problems, and each has a fix: a one-page role brief, a shared rubric, and a calibration session.

    AIEH suggests calibrating on shared example answers until raters converge, pairing new interviewers with experienced ones, recalibrating quarterly, and auditing agreement across a sample of candidates rated by several people.

    A simple aggregation rule for a small team:

    1. Each interviewer scores every competency alone.
    2. Scores are collected before any discussion.
    3. Where two raters differ by two or more points, they compare notes against the anchor, not against each other’s impressions.
    4. The final score per competency is the agreed score, and the total is the equal-weight average.

    Our posts on candidate scorecards and candidate comparison methods show how to lay these out.

    How scoring reduces bias, and where it still needs care

    RIT’s guidance says a rating scale gives a quantitative basis for comparison between interviewers, lets you check your view against colleagues, and helps remove distortions caused by interviewer bias, differing questions and interpersonal factors.

    Scoring does not remove every source of variation. A 2008 study using a many-faceted Rasch model on panels of seven raters found that rankings based on raw scores and rankings adjusted for rater severity can differ substantially, with one interviewee moving 15 places. Some raters are simply harsher than others. If different candidates are seen by different raters, compare scores with that in mind, and review each rater’s average from time to time.

    There is also a compliance angle in the U.S. One industry summary notes that the EEOC’s 1978 Uniform Guidelines treat interviews as selection procedures, so unstructured processes that produce disparate outcomes carry legal exposure, and structured interviewing is a main way to lower it. For your specific hiring process, check with employment counsel.

    Scoring at high volume

    When you have hundreds of applicants for a role, full panel scoring for each is not realistic. A practical pattern:

    • Round one: the same short set of behavioral questions for everyone, scored on the same anchors, so you can cut the list without relying on resume polish.
    • Round two: a smaller group meets the hiring manager with deeper probes and a second independent rater.
    • Before the offer: check the strongest claims, such as the result in the best STAR story, against references, work samples or a skills test.

    The third step matters because a polished STAR answer is still self-reported. Scoring tells you how well the evidence fits the anchor. It does not tell you the evidence is true. Probing questions help in the room (“Who else was involved? What did you personally write or decide? What were the numbers?”), and independent checks do the rest.

    Our perspective

    Most scoring problems we see are not about the scale. They are about volume. A team agrees on a rubric, then the applicant count triples, and the rubric quietly turns into a gut feeling by Thursday.

    We built Resyme around that gap. Our interviews are designed to use questions tailored to the role and anchored in behavior, so every candidate is asked about the same competencies and answers are captured in a consistent format. The reports are designed to let recruiters compare candidates objectively, and our honesty and integrity validation is built to flag where an answer may not hold up, so the time you save on pre-screening goes to the people who deserve a real conversation.

    We think the best use of automation here is not replacing judgment. It is making sure every candidate gets a structured, fair first conversation, so human judgment starts from comparable evidence.

    Raul

    Run structured behavioral interviews without the manual work

    If you are a recruiter, agency or small business without an ATS, Resyme runs automated, role-tailored behavioral interviews and produces comparable reports for each candidate. You can pay per interview at €4.50 to €7.99, or choose a subscription: Company at €599 per month or Agency at €4,500 per month. There is also a Free plan, and Enterprise pricing is available from us on request. See the options on the Resyme site.

    Resyme

    For a hands-on walkthrough of the format, read our guide to running behavioral AI interviews, and see role-based interview questions for question ideas by function. Our piece on objective candidate comparison explains how to rank finalists once the scores are in.

    FAQ

    What is a good score on a behavioral interview?

    It depends on your scale and the role’s requirement. On a 1 to 5 scale, 3 usually means the candidate meets the requirement, and 4 or 5 means they exceed it. Set the threshold for each competency before interviewing, based on what the job needs, so you are not adjusting the bar after meeting people you like.

    How many competencies should I score?

    Four to six is the typical range for a structured interview, according to OPM, unless the job is unusual or senior. If a role needs more, split the assessment across interview stages instead of cramming everything into one conversation.

    Should every competency carry the same weight?

    By default, yes. OPM calls equal weights the most defensible approach unless you have a clear, documented reason for different ones. If you do weight a competency more heavily, record the reason before scoring starts.

    When should interviewers write their scores?

    Immediately after the answer or the interview, and before discussing with other interviewers. Fresh notes keep the score tied to what the candidate actually said, and independent scoring stops one opinion from steering the group.

    Sources