All work

AI Interviewer · voice AI · TypeScript · realtime

An interviewer that scores you while you are still talking

AI Interviewer runs a live, spoken job interview. It asks five questions about the role, listens, and scores each answer before asking the next. When you hang up, a full report is written in the background and emailed to you, with the recording attached to your results.

Role

Sole author

Frontend

Next.js, WebRTC

Backend

Express, Drizzle, BullMQ

AI

OpenAI Realtime, GPT-4o mini

Source

A mock interview that feels like a call

Create a job, paste its description, and start. The interviewer greets you and asks the first question straight away. You answer out loud, on camera, the way you would in the real thing.

Job templates

Each interview is built from a job title and description, so the questions are about that role.

A real conversation

Natural voice, quick turn-taking, and an interviewer that knows when you have finished speaking.

Live scores

After every answer: communication, technical depth, clarity, problem solving and confidence, each out of 100, updating on screen.

Video and sound controls

Camera, microphone and speaker settings in the interview room.

The full report

An overall score, a short summary, three strengths, three things to work on, and feedback on every question.

Recording and email

Watch your interview back next to the report, and get an email the moment it is ready.

Dashboard

Every past interview and its results in one place.

Voice is unforgiving about latency

A chat app can take a second to answer. A voice interviewer cannot. A pause of more than a beat after you stop talking and the whole thing feels broken. That rules out anything that waits for the full answer, sends it somewhere, waits again and then speaks.

And interviews are heavy. Two-way audio and a video recording for every session is a lot of bytes, and none of it needs to pass through my server to do its job.

How I got there

Three passes, in the order they happened.

Explore

Run everything through my own server

Relay the audio
Score at the end
Upload through the API

Realise

Every hop adds delay and bytes

Relay the audio
Score at the end
Upload through the API

Insight

Let each piece go where it is going

Browser to model
Score every answer
Straight to S3

Hand out a key, then get out of the way.

What happens when you press start

The server's job is to set things up and tidy up after. During the interview itself, it is not in the audio path at all.

  1. Mint a short-lived key

    The backend creates a realtime session with the interview's instructions, voice, tools and turn detection, and gets back a key that only works for that one session. It also creates a presigned upload link for the recording.

    backend/src/routes/interview.routes.ts

  2. Browser to model, directly

    The browser opens a WebRTC connection straight to OpenAI with that key and streams audio both ways. My API key never reaches the browser, and the audio never reaches my server.

    frontend/hooks/use-interview.ts

  3. Know when you have finished

    Server-side voice detection with a 300ms silence window and a low threshold, so the interviewer replies quickly without talking over a pause mid-sentence.

    backend/src/lib/openai.ts

  4. Score through a tool call

    After each answer, and before speaking again, the model must call submit_feedback with five scores. The browser catches the call and updates the panel. After the fifth answer it calls end_interview, which ends the session cleanly.

  5. Upload, then queue

    The recording goes straight to S3 with the presigned link. The transcript goes to the API, which marks the interview as processing and puts an evaluation job on a BullMQ queue.

  6. Write the report, send the email

    A worker asks GPT-4o mini for a structured JSON report, saves it, marks the interview complete and emails you. The results page picks it up as soon as it lands.

    backend/src/workers/evaluation.worker.ts

Scoring is a tool, not a guess

Asking a talking model for numbers in its speech would be fragile. A tool call gives structured scores at a known moment, every time.

tools: [
  {
    type: "function",
    name: "submit_feedback",
    description: "Submit live feedback scores after the candidate answers a question.",
    parameters: {
      type: "object",
      properties: {
        communication: { type: "number", description: "Score 0-100 for fluency, confidence, and pacing" },
        technical: { type: "number", description: "Score 0-100 for correctness, depth of knowledge, and terminology" },
        clarity: { type: "number", description: "Score 0-100 for conciseness and avoiding rambling" },
        problemSolving: { type: "number", description: "Score 0-100 for analytical thinking and approach" },
        confidence: { type: "number", description: "Score 0-100 for self-assurance and conviction in answers" },
      },
      required: ["communication", "technical", "clarity", "problemSolving", "confidence"],
    },
  },
  {
    type: "function",
    name: "end_interview",
    description: "Call this when the 5th question has been answered and the interview is over.",
    parameters: { type: "object", properties: {} },
  },
],
backend/src/lib/openai.ts

What AI Interviewer came down to:

01

Out of the audio path

A short-lived key and WebRTC put the browser and the model in direct contact, so the conversation is as fast as it can be.

02

Scores as structure

Tool calls deliver five scores after every answer, and end the session when the interview is done.

03

Heavy work off the request

Recordings go straight to S3, and the full report is written by a queued worker and emailed when it is ready.

Back to all work