Skip to content
    ← Back to blog

    Text Interview Analysis: How It Works and Where It Fails

    Text interview analysis is the automated reading of what a candidate said or wrote, usually from a transcript of a recorded or written interview, so you can score answers against the same job-related criteria. Done well, it cuts pre-screening time and makes candidates comparable. Done badly, it hides bias behind a score. This guide covers how the process works, what NLP can and cannot tell you, and the checks that keep a human accountable for the hire.

    Resyme
    Make Interview Insights More Useful
    Resyme uses tailored, behaviorally anchored AI interviews to help recruiters compare candidates and reduce time spent on pre-screening.
    Visit Resyme

    TL;DR

    • Text interview analysis looks at the words in an answer (content, structure, how fully a question was addressed), not at facial expressions or tone.
    • The reliable workflow is: clean transcript, consistent questions, scoring against predefined job criteria, then human review of the evidence.
    • Research shows language models can score like experts on average but can be biased by dialect, culture and hedging, so outliers need a person.
    • In the US, tell candidates when AI is used and get consent. Some states, such as Illinois, require separate consent for AI analysis of video interviews.
    • Use the output to compare candidates and to decide whom to talk to next, never as an automatic reject.

    Table of Contents

    What text interview analysis means in recruitment

    In hiring, text interview analysis examines the content of candidate answers. The source is often a transcript: an asynchronous video or voice interview is recorded, automatically transcribed, and then the text is analyzed. In other setups the candidate types answers in a chat or form, and the analysis starts directly from text.

    The Recruitment and Employment Confederation’s guidance on asynchronous video interview assessment draws a useful line here. Analyzing the candidate’s words is one thing. Analyzing tone, intonation, facial expressions or body language is another, and the REC advises against buying tools that claim to recognize emotions or expressions because they rest on controversial, pseudo-scientific assumptions. That distinction should guide any vendor conversation: ask what the tool reads, and whether those signals are relevant to the job.

    The same guidance tells recruiters to establish how a tool informs decisions, what it analyzes, and whether the criteria and the competencies assessed are job-relevant. Those three questions are a good test for any product in this space, including ours.

    Text analysis versus transcript reading

    A recruiter reading a transcript is doing text analysis by hand. Software does the same at volume: it extracts the evidence in each answer, maps it to a competency or requirement, and presents it in a form you can compare across fifty or five hundred candidates. The value is consistency. Every candidate gets the same questions and the same scoring lens, which is the core of structured interviewing.

    How the workflow runs, step by step

    Whether you use software or a spreadsheet, the same sequence produces defensible results.

    1. Define the criteria before any interview. Utah State University’s hiring toolkit advises committees to align on evaluation criteria and how feedback will be documented before interviews begin, and to use a hiring matrix with predetermined criteria. Consistent questions and criteria help reduce the influence of bias.
    2. Ask the same job-related questions of every candidate. If you need a starting point, our guides on defensible interview question design and structured interview templates show how to keep the set short and tied to the role.
    3. Produce an accurate transcript. Transcript errors flow straight into the analysis. Indeed’s career advice recommends replaying the whole interview after drafting a transcript to proofread it and fill in missing details.
    4. Code the answers against your criteria. Tag each passage to a requirement, such as a specific skill, a behavior example or a certification, and note where the candidate gave no evidence at all.
    5. Separate evidence from judgment. A recruiting-focused transcription guide from Transcrio suggests keeping transcript quotes distinct from your own assessment and rereading the passages around each scorecard entry to confirm context. It also recommends reviewing your own question phrasing for bias.
    6. Compare, then decide. Line candidates up on the same criteria. The comparison produces a shortlist for human conversation, not a verdict.

    A worked example

    Say you are hiring a customer success manager and your criteria are renewal ownership, handling an upset client, and reporting discipline. Each candidate answers four behavioral questions. Candidate A describes a specific account, the action taken and the result, with numbers. Candidate B talks about “always putting the client first” without an example. Text analysis flags that A gave concrete evidence on renewals and B gave none, and lays both next to each other under the same headings. You still decide whether B deserves a follow-up call, but you have saved the half hour you would have spent finding that out.

    What NLP can identify in candidate answers

    Natural language processing can do several useful things with interview text: extract skills and tools mentioned, check whether an answer addresses the question, identify whether an example follows a situation, action and result pattern, and summarize long answers so reviewers do not have to read every word.

    It can also gauge how fully a question was answered. A 2022 study of mock interviews with 24 military veterans tested models that detect under-explained and over-explained answers. Tokenizer-based features did best at spotting under-explanation (F1 of 0.29), while lexicon-based methods did best at over-explanation (F1 of 0.51). Those are modest numbers. They show the problem is tractable, and also that automated judgment of answer quality is far from solved. Treat such signals as prompts for a reviewer, not as scores that settle anything.

    For teams who want to go further into behavioral evidence, we cover how to analyze soft skills in a way recruiters can audit and what AI interview analysis looks like in practice.

    What it cannot do

    It cannot read character. Vendors that promise personality, emotion or “honesty” from tone or expression deserve skepticism. A 2024 study reported in IEEE Transactions on Affective Computing found that large language models can match or beat task-specific AI at zero-shot personality prediction from video interviews, but with uneven performance across traits and insufficient test-retest reliability. A tool that gives a different answer on a second run is a poor basis for a hiring decision. When you evaluate vendor claims about emotional or personality scoring, an emotional intelligence leadership assessment shows how much careful, validated method sits behind that kind of measure, which is a useful yardstick for what a text-only tool should have to prove.

    Bias and accuracy risks you need to plan for

    The research on automated interview scoring is mixed, and an honest buyer reads all of it.

    Transcription itself can be biased. A 2024 study in the Journal of Applied Psychology found significant demographic bias in commercial speech recognition services across nearly all comparisons. The reassuring part: that transcription bias did not translate into meaningful measurement bias in the machine learning interview scores the researchers examined. The practical lesson is still to check transcripts for candidates with accents or non-standard speech.

    Language models can penalize cultural style. A 2025 AAAI/ACM study used 100 UK and 100 Indian interview transcripts and found consistently lower LLM scores for the Indian transcripts, even after anonymization, with differences linked to sentence complexity and lexical diversity. Text-only assessment is not neutral just because it ignores faces.

    Hedging reads as weakness. A 2025 benchmark found that hedged responses were rated 25.6% lower on average than equivalent confident responses in LLM hiring evaluations. Candidates who say “I believe” or “in my experience” may be penalized for style, not substance.

    Scores look expert-like but miss errors. Research by Maity and colleagues found that GPT-4 Turbo and GPT-3.5 Turbo can score comparably to human experts but often fail to identify errors or to give specific, actionable improvement advice. The authors advocate a human-in-the-loop approach, since these models are not fully suited to automatic deployment in HR assessment.

    What to do about it

    • Score against job-related criteria only, and drop any signal you cannot tie to the role.
    • Review outliers, high and low, before they affect anyone’s progress. The REC lists outlier scores, transcription errors, relevant competencies and possible discriminatory impact as the things human review should address.
    • Ask vendors how they test for differences across groups and how often they re-test.
    • Keep a person responsible for every decision that affects a candidate. This is the point of the discussion on why AI should support rather than replace hiring judgment, which carries beyond onboarding into screening.

    There is no single federal rulebook for analyzing interview text, so build your process around the strictest reasonable standard and check your own state and city rules with counsel.

    George Mason University’s guidance on AI in recruitment is a practical model. It says to inform candidates when AI is used for substantive purposes, obtain recording consent, document AI-related actions, review AI outputs with humans, and vet and monitor tools for bias. It limits AI screening criteria to objective, job-related minimum qualifications and requires human review of AI-generated summaries or scores before deliberations.

    State law can add duties. Corporate candidate privacy notices, such as SharkNinja’s 2026 notice, point out that Illinois requires specific, separate consent before an employer can run AI-based analysis of a video interview. If you hire across states, treat that as the baseline rather than the exception.

    A short checklist you can put in your process document:

    • A plain-language notice before the interview saying what is recorded, what is analyzed and who sees it.
    • A separate consent step where state law requires it.
    • A named reviewer for every AI-generated summary or score.
    • A log of what the tool produced and what the human decided.
    • A route for candidates to ask for a person to review their result or to use another format.

    Buro Happold’s published approach to AI in recruitment is a useful example of tone: it treats AI as a co-pilot for transcription and summarization that reduces real-time note-taking, while people make the hiring decisions.

    Using text analysis to compare candidates and spot inconsistencies

    The strongest use of text analysis is comparison. When every candidate answers the same questions and you score them against the same predetermined criteria, differences in evidence become visible. Transcript evidence can be isolated and reviewed on its own, which supports a more objective look than a gut impression after a long day of calls.

    It can also surface possible inconsistencies: a candidate who describes leading a team of ten in one answer and a team of three in another, or who claims a tool in the resume but cannot describe a single task done with it. These are flags for a follow-up question, not proof of dishonesty. People misremember, and nerves distort. Ask the clarifying question, then judge.

    For high-volume roles, this is where the time savings sit. Rather than phone screening three hundred applicants, you can have each one complete the same structured interview and review the ranked evidence. For specialized roles, it means the questions follow the domain. Our walkthrough on running behavioral AI interviews and on turning job descriptions into interview questions shows how we set that up.

    Setting up a lean workflow for a small team

    You do not need an applicant tracking system to start. A small business or startup can run this in a week:

    1. Write three to six must-have criteria for the role from the job description.
    2. Draft four to six questions, each tied to one criterion.
    3. Invite candidates to answer by voice or text on their own time.
    4. Review the transcript-based report for each, reading the quoted evidence, not just the score.
    5. Shortlist for live interviews and note why each person advanced or did not.

    Keep the notes. If a decision is ever questioned, a record showing consistent criteria and human sign-off is your best protection.

    Our perspective

    We built Resyme around one belief: a screening interview should produce evidence a recruiter can check, not a mystery number. That is why we start from the role, generate domain-specific, behaviorally anchored questions, and present each answer next to the criterion it addresses. We also compare candidates on identical questions, because that is where fairness and speed come from at the same time.

    We are cautious about anything that claims to read character from a face or a voice. Words are checkable, and a reviewer can disagree with them. That is the standard we hold ourselves to, and it is the standard we suggest you hold any vendor to, us included. Ask to see a real report before you buy.

    — Raul

    See it on real interview data

    If you are screening more candidates than your team can interview, Resyme runs the structured interview for you and returns comparable reports with the evidence quoted. Pricing is published: the Company plan is €599 per month, the Agency plan is €4,500 per month, and pay-as-you-go interviews cost €4.50 to €7.99 each. A Free plan and an Enterprise plan are also available. Review the options on the Resyme pricing and plans page, or look at a sample AI interview report first.

    Resyme

    For more on how recruiters are choosing tools, see our overview of why interviewing needs its own AI tool and our notes on resume screening governance.

    FAQ

    Is text interview analysis the same as analyzing facial expressions or tone?

    No. Text analysis works on the words of an answer, usually from a transcript. Facial expression and tone analysis is a separate category, and the REC advises against tools that claim to recognize emotions because they rely on pseudo-scientific assumptions. Choose tools that can show you the exact words behind each score.

    Can AI text analysis tell whether a candidate is lying?

    Not reliably. It can flag inconsistencies between answers, or between an answer and a resume, so you can ask a follow-up question. It cannot prove dishonesty, and research shows language models can be uneven and sensitive to style, so a person should make the call.

    Best practice is to tell candidates when AI is used and to obtain recording consent. Some states go further: Illinois requires specific, separate consent before AI analysis of a video interview. Check the rules where you hire and ask your counsel.

    Is it biased against non-native speakers?

    It can be. Studies have found lower LLM scores for Indian interview transcripts than UK ones, even after anonymization, and demographic bias in commercial speech recognition. Review transcripts for accuracy, score on job-related content, and have a human look at outlier results.

    What is the minimum I need to start?

    A set of job-related criteria, the same four to six questions for every candidate, an accurate transcript, and a named human reviewer. Software speeds up the steps, but those four elements make the process defensible.

    Sources