Picture a candidate named Rahul. Six years at a Bengaluru SaaS company, a clean record of launches, and forty minutes into a loop at an AI search startup. The interviewer shares a scenario:
→ “the company’s support bot gives wrong answers, even though the right help article exists in the knowledge base. What would you do?”
He doesn’t hesitate: “I’d fine-tune the model on our help center so it learns the content and how humans would’ve handled a customer query”
The interviewer’s pen stops moving and interview ends abruptly.
Fine-tuning doesn’t reliably teach a model facts, and any fact it did absorb would go stale the first time a policy changed. But the bigger miss is quieter. Rahul never asked whether the right article was being found at all. He never asked what a wrong answer costs. He named a technique, skipped the diagnosis, and handed the interviewer nothing to push back on except the technique itself.
He probably knew more than he showed. Most candidates do.
The gap isn’t knowledge, it’s structure: a repeatable way to turn “here’s an AI problem” into “here’s the decision, here’s why, and here’s what would change my mind.”
This guide gives you that structure.
The framework is called STACK-AI, it has seven steps, and it handles the most common AI technical questions - choosing between approaches, deciding when to use a technique, and diagnosing a product that’s behaving badly.
For “system design” questions, you’ll use S.P.E.C.T.S. instead, and you’ll see how to tell the two apart in the first thirty seconds.
Example of Agentic System Design question: Design an Agentic AI System That Autonomously Adapts to New Tasks.
Table of Contents
What these questions are really about
Nobody is going to ask you to write code. Nobody is going to ask you to derive a loss function or explain what happens inside a transformer layer.
The interviewer wants to know if you could collaborate with an ML engineer and trust the decision that comes out. Specifically, whether you can:
explain how the pieces of an AI system work and fail, in plain language
make a defensible call between real options
judge what’s cheap, what’s expensive, and what’s a two-week experiment versus a two-quarter program
Here’s the idea this whole article rests on. A PM doesn’t need to build the architecture. A PM needs to know which architectural levers exist, what each one costs, and which one fits the problem in front of them.
Candidates fail these questions in two opposite ways.
Some go too shallow: “we’d use RAG because it’s better.” That names a technique with no mechanism and no reason.
Others go too deep, reciting how vector databases index data while never saying what the product needs.
The strong answer sits in the middle. It shows mechanism-level understanding, ties it to a product constraint, and ends in a decision.
Understanding the types of AI technical questions
Before you pick a framework, you need to know which kind of question you’re facing. They arrive in three shapes.
1. Choice questions
The interviewer hands you two or three options and asks which one fits.
“RAG or fine-tuning for this product?”
“Hosted model API or an open-weights model you run yourself?”
What they’re testing: whether you can lay out a real option space and pick one using a constraint, not a preference.
2. When-to-use questions
The interviewer names a technique and asks when it earns its place.
“When would you use embeddings?”
“When would you build an agent instead of a workflow?”
What they’re testing: whether you understand the mechanism well enough to know where a technique helps and where it breaks.
3. AI System design questions
The interviewer asks you to design the whole thing.
“Design an AI coding assistant that understands a company’s private monorepo.”
What they’re testing: whether you can structure an open-ended problem, make architectural choices, and show how you’d evolve the system.
Concepts to know before answering any AI technical questions
Below ideas show up in almost every answer.
Prompt, System Instructions and Knowledge Base
Context Window and Memory
RAG
Embeddings
AI Workflow and AI Agent
The quality, latency, and cost triangle
Read these concepts in detail here.
Now, Let’s dive deeper now.
How to Answer AI Technical Questions?
You don’t need to memorize every database type or cloud service. What you need is a repeatable framework that works for any technical question thrown at you.
For AI system design, use the S.P.E.C.T.S. framework - read the full guide here.
For the other two types, Choice and When-to-use, Use the below STACK-AI Framework:
STACK-AI has seven steps with noun-based names. The shape is a funnel: generate options, filter them, then decide.
Step 1 - Scope: What decision is on the table, and what does success mean?
Step 2 - Task: What exactly must the AI do, and is this a knowledge problem or a behavior problem?
Step 3 - Alternatives: What are three or four real options, simplest first?
Step 4 - Constraints: What numbers eliminate options, and which one is binding?
Step 5 - Knowledge: What does the system need to know, where does it live, and who may see it?
Step 6 - Assessment: Which option wins, what do I accept, and what would change my mind?
Step 7 - Implementation: How do I ship it, evaluate it, protect it, and evolve it?
Now, let’s dive into each step in detail with an example of support bot.
Step 1: Scope
One example runs through every step. The question:
“A support team has 50,000 help articles that change weekly. How would you architect an AI answer bot?”
Goal: settle what decision is on the table before saying anything technical.
What to cover:
Restate the question in your own words and name its shape
Ask about stakes, scale, users, data, team, timeline, and how much autonomy is acceptable
State assumptions and non-goals out loud so the interviewer can redirect you
Define success in one sentence
A candidate who asks “what happens if the answer is wrong?” in the first minute signals that they think about AI failure before architecture.
Example Response: Applied to the support bot.
Restated: “We’re building a bot that answers customer support questions from a 50,000-article help center that changes every week, and I’m treating this as a choice about how the bot gets its knowledge.”
Questions I’d ask, with the assumptions I’d state if no answer came:
What does a wrong answer cost? Assume wrong how-to answers cost a repeat contact, and wrong billing or refund answers cost real money, chargebacks, and angry escalations.
How many questions a day? Assume 20,000.
Who are the users? Assume logged-in customers of a B2B SaaS product, where some articles apply only to certain plan tiers and some are internal-only.
How much churn in the content? Assume a couple of thousand articles get edited in a typical week.
Who builds it? Assume one PM, three engineers, and one ML engineer, with a pilot due in six weeks.
How much autonomy? Assume the bot answers questions and hands off to a human. It can’t issue refunds or change accounts in version one.
Non-goals: no autonomous actions, no voice, no languages beyond the support team’s current set.
Success: “Customers get a correct, cited answer in a few seconds for questions the articles cover, and a clean handoff to a human for the ones they don’t.”
Step 2: Task
Goal: define precisely what the AI has to do. This is the most underrated step in the framework.
What to cover: