You can build an AI application that works and still struggle to explain it in an interview. Hiring managers often care less about impressive terminology and more about whether you understand the decisions behind your project.
If you are moving from Salesforce, QA, data engineering, testing, or support into AI engineering, prepare for these six questions. Each tests a practical skill you will need beyond the interview.
1. Explain RAG Simply
What the interviewer is testing: Can you explain how a model uses external information without hiding behind jargon?
Weak answer: “RAG uses embeddings and a vector database to improve LLM responses.” This names components but does not explain what happens or why.
Strong answer: “Retrieval-Augmented Generation searches a trusted knowledge source for relevant information and gives those passages to the language model before it answers. For example, a support assistant can retrieve the latest product manual from PostgreSQL with pgvector rather than relying only on what the model learned during training.”
Be ready to explain document chunking, embeddings, similarity search, and what happens when retrieval returns irrelevant information. You should also know that RAG can reduce unsupported answers but does not eliminate hallucinations.
2. How Do You Evaluate an LLM Application?
What the interviewer is testing: Do you measure quality systematically, or do you judge an application by trying a few questions?
Weak answer: “I test different prompts until the output looks good.” That may help during development, but it is not a repeatable evaluation process.
Strong answer: “I create a fixed evaluation set of representative questions and expected results. For a RAG application, I measure whether the correct evidence appears in the top five retrieved chunks, whether the answer is supported by that evidence, and how much each request costs.”
For a starter project, 50–100 test cases can help expose common failures. Tools such as Ragas, DeepEval, or LangSmith can support evaluation and tracing, but the metrics must match the application and its risks.
3. How Do You Handle Hallucinations?
What the interviewer is testing: Can you design safeguards against confident but incorrect answers?
Weak answer: “I tell the model not to hallucinate.” Instructions alone cannot guarantee factual accuracy.
Strong answer: “I retrieve relevant evidence, instruct the model to answer from that evidence, and allow it to say when the information is insufficient. I test unsupported claims, check citations against the retrieved sources, and route uncertain or high-risk cases to human review.”
Explain how you would handle missing documents, conflicting policies, stale information, and irrelevant retrieval results. A reliable system should be able to stop rather than invent an answer when the evidence is inadequate.
4. When Would You Fine-Tune a Model?
What the interviewer is testing: Can you select the simplest technique that solves the actual problem?
Weak answer: “Fine-tuning makes the model smarter and more accurate.” The outcome depends on the training data, task, and evaluation method.
Strong answer: “I would first establish a baseline with prompting. If the model consistently fails to follow a specific format or perform a stable task, and I have representative labelled examples, I would evaluate fine-tuning against the baseline. If the problem is missing or changing knowledge, I would consider RAG instead.”
Be prepared to discuss training-data quality, held-out test data, maintenance, and cost. Fine-tuning is not automatically the best solution just because a project uses an LLM.
5. How Do You Control AI Application Costs?
What the interviewer is testing: Do you understand the trade-offs between quality, latency, and operating cost?
Weak answer: “I choose a cheaper model.” Model choice matters, but unnecessary tokens, repeated calls, and inefficient retrieval can also increase the bill.
Strong answer: “I track input and output tokens, cost per request, latency, and retries. I reduce unnecessary context, cache suitable repeated requests, use smaller models for routine tasks, and reserve more capable models for requests that need them. I then compare answer quality before and after each change.”
For example, if an application sends 10,000 input tokens for every question when 2,000 relevant tokens would work, reducing context could substantially lower input-token usage. You must still verify that retrieval quality and answer accuracy remain acceptable.
6. Walk Me Through a Project You Built
What the interviewer is testing: Can you explain a real engineering problem, defend your decisions, and identify what went wrong?
Weak answer: “I followed a tutorial to build an AI chatbot using Python and LangChain.” That explains what you copied, not what you understand or contributed.
Strong answer: “I built a document-question-answering prototype using Python, an LLM API, and PostgreSQL with pgvector. I implemented document chunking and retrieval, created a test set, measured retrieval failures, and improved the context supplied to the model. It remains a prototype, so I have not claimed production-scale reliability.”
Only use that example if it matches your actual work. Replace the details with your own tools, measurements, decisions, and limitations.
Prepare a 90-Second Project Story
One mistake can end an interview early: describing a tutorial project as if it were production software. Experienced interviewers will ask about real users, failure handling, security, deployment, monitoring, and how you know the system works.
Prepare a 90-second explanation using five parts:
- Problem — 15 seconds: What user problem were you solving?
- Architecture — 20 seconds: Which model, database, APIs, and frameworks did you use?
- Your contribution — 20 seconds: What did you implement yourself?
- Evidence — 20 seconds: What did you test or measure, and what improved?
- Limitations — 15 seconds: What remains incomplete, and what would you improve next?
Practise answering follow-up questions about why you selected each tool, how you handled errors, and what you would change if usage increased from 10 to 1,000 users. Honest limits supported by clear technical reasoning are stronger than claims you cannot defend.
Which of the six would sink you if someone asked it today?
Join the KKRGENAI community discussion to share your answers, practise project explanations, and prepare for your next AI engineering interview.
