The second round of the screen plays out inside this section, the closing gate before any interview hitting the table. A recruiter genuinely slows the pace here, and even so, your current chair drives around 95% of the result.
That tracks: nothing demonstrates what you can ship in production today better than the seat you sit in this quarter. To earn a "yes", the section has to hit every entry on the AI Engineer role profile, one bullet per area listed under Domain Expertise. And every bullet has to come off work you genuinely owned in production, never a Jira card that drifted past your queue.
1
LLM Application Development
You turn an LLM into a feature people use every day. Demos are easy and production is not, so hiring
managers want proof of shipped features with real users, not a weekend prototype. Point out how you used
tool calling and structured outputs, with the OpenAI or Anthropic APIs behind FastAPI, to ship a feature
that lifted adoption.
Techniques
Chat & copilot UX
Structured outputs
Streaming responses
Tool / function calling
Tools
OpenAI, Anthropic, Bedrock
LangChain, LlamaIndex
FastAPI, Next.js
Metrics
Features in production
Active users on the feature
Adoption / retention lift
2
RAG & Retrieval Systems
You ground the model in your own data so it stops guessing. Hallucinated answers burn user trust fast,
so hiring managers want retrieval that measurably improves groundedness, not a vector database bolted
on. Walk them through how you used hybrid search and reranking, on pgvector with Cohere Rerank, to lift
retrieval recall and cut the hallucination rate.
Techniques
Chunking & embedding
Hybrid & semantic search
Reranking
Context window management
Tools
Pinecone, pgvector, Weaviate
text-embedding-3, Cohere Rerank
LlamaIndex, Haystack
Metrics
Retrieval recall / nDCG
Answer groundedness
Hallucination rate cut
3
Agents & Tool Use
You let a model take real actions, not just answer. Hiring managers look here to see whether your agent
finishes the task on its own, or whether it loops until a human rescues it. Talk about how you used
function calling and multi-step planning, in LangGraph, to raise the task success rate and cut manual
handoffs.
Techniques
Function calling
Multi-step planning
ReAct & tool routing
Memory & state management
Tools
LangGraph, CrewAI
OpenAI Assistants
Anthropic Tool Use
Metrics
Task success rate
Steps to completion
Manual handoffs cut
4
Prompt Engineering & Evals
You prove the model got better, not just different. An eval pass rate is a number a hiring manager can
check; "the answers feel better" is not. Lay out how you used golden datasets and LLM-as-judge
evals, in Braintrust or LangSmith, to catch regressions before release.
Techniques
Few-shot & chain-of-thought
LLM-as-judge
Golden datasets
Regression eval suites
Tools
LangSmith, Braintrust
Phoenix, Helicone
Promptfoo, Ragas
Metrics
Eval pass rate
Regression catches before release
Prompt iterations tracked
5
LLM Deployment & LLMOps
You run the live feature: latency, fallbacks, token cost. Two things ride on it for a hiring manager:
time-to-first-token users will tolerate, and cost that doesn't scale faster than usage. Show them
how you used streaming and semantic caching, with vLLM and Helicone telemetry, to cut
time-to-first-token and cost per request.
Techniques
Streaming & SSE
Caching & semantic cache
Rate limiting & fallback
Cost & token telemetry
Tools
AWS Bedrock, Azure OpenAI
LangSmith, Helicone
FastAPI, vLLM
Metrics
Time-to-first-token
Cost per request
Error budget hit rate
6
Safety, Guardrails & Compliance
You stop the model from being talked into something harmful. Hiring managers read working guardrails as
production readiness, not red tape, because one leaked record or jailbreak can end the feature. Spell
out how you used prompt-injection defense and PII redaction, with Llama Guard and Presidio, to block
unsafe outputs and hold the leak rate at zero.
Techniques
Input / output moderation
Prompt injection defense
PII detection & redaction
Toxicity & bias filtering
Tools
NeMo Guardrails, Guardrails AI
OpenAI Moderation, Llama Guard
Presidio, Lakera
Metrics
Unsafe outputs blocked
Jailbreak attempts caught
PII leak rate
7
Cross-Functional Collaboration
You ship an AI feature with Product and Design. Companies keep the AI engineers who ship with the squad,
not the ones producing solo prototypes, so hiring managers look for it. Mention how you used joint
use-case scoping and shared launch criteria, run through Figma and Linear, to take features from idea to
launch faster.
Techniques
Use-case scoping with Product
Prompt review with Design
API contracts with Backend
Joint launch criteria
Tools
Notion, Figma, Linear
Slack, GitHub
Loom, Miro
Metrics
Features shipped jointly
Squads supported
Time from idea to launch
8
Tooling & Workflow
You hold a prompt-driven product steady with versioned prompts and CI evals. That discipline lets the
whole team change prompts without fear, so it tells a hiring manager you run LLMs like software. Tell
them how you used prompt version control and CI evals, wired into GitHub Actions with LangSmith traces,
to keep every change tested before it ships.
Techniques
Prompt version control
CI evals on every PR
Trace logging
Reproducible LLM envs
Tools
Git, GitHub Actions
LangSmith, Braintrust, Helicone
Docker, Poetry, uv
Metrics
Prompts under version control
Evals running in CI
Onboarding ramp time cut