Most AI Projects Don't Fail at the Model. They Fail at the Product Layer.

Every few weeks a product manager messages me on LinkedIn with some version of the same question: "How do I transition into an AI product role? What's the roadmap?"

I've started answering more bluntly than I used to, because most of the roadmaps floating around are written by people who've never shipped a production AI system. They tell PMs to "learn prompt engineering," "take a course on LLMs," "play with ChatGPT." That advice isn't wrong, exactly. It's just shallow. It doesn't address what we actually need from a PM when we're building one of these systems for a client.

So this piece is the answer I'd give over coffee, written from where we sit as the engineering partner building AI systems for companies in fintech, healthcare, insurance, and beyond. We've shipped these things, watched them work, and watched them fail. The failure mode is the part nobody writes about.

The uncomfortable truth: most AI projects fail at the PM layer

I'll start with the contrarian claim, because the rest of the article hinges on it.

In our experience, the engineering side of AI projects is mostly solved. The models work. Bedrock works. RAG works. Guardrails work. Agent frameworks work. None of it is trivial, but it's tractable. A competent team can stand up a working AI pipeline in weeks.

What kills these projects isn't the engineering. It's everything around the engineering:

• Nobody defined "good enough." The team ships, the stakeholder eyeballs ten outputs, and rejects it. There's no eval set to argue against.
• Nobody scoped the failure mode. The happy path was designed; the 30% of traffic where the model is unsure, wrong, or blocked by a guardrail has no UX.
• Nobody owned the cost/latency tradeoff. The system works in the demo, costs $40 per user-session in production, and gets killed.
• Nobody audited the data. The model can only ground in what you give it. The "knowledge base" turns out to be three SharePoint folders of conflicting PDFs from 2019.
• Nobody said no to the wrong AI features. The roadmap is full of "AI-powered" things nobody asked for, while the genuinely high-leverage AI work sits in someone's drafts folder.

Every item on that list is a PM responsibility. Not an engineering one.

This is good news for traditional PMs, by the way. The skills that matter most in the AI era are PM skills, sharpened, updated, and aimed at a probabilistic system instead of a deterministic one. You're not starting over. You're updating a craft.

What changes when the system is probabilistic

The single biggest shift is this: the thing you're shipping no longer has a fixed output for a given input. The same question can produce different answers. The same prompt can succeed in testing and fail in production. The same model gets quietly updated by the provider and behavior drifts.

That breaks a lot of the muscle memory of traditional product management:

• "Acceptance criteria" stops being a checklist. It becomes a distribution. "The agent answers correctly 92% of the time, with no incorrect answers in the safety-critical 8% of cases."
• "Spec the feature" stops being enough. You're now specifying behavior across success, ambiguity, refusal, and failure. All four are part of the product.
• "A/B test it" stops being clean. Variance in model output means small effect sizes get drowned. You need new measurement tooling.
• "Edge case" stops being a useful word. What you used to call edge cases are now your second-largest user segment.

If you internalize one thing from this article, internalize that. Everything that follows is consequences.

The four layers of the roadmap

We think about the transition in four layers. They build on each other. You can't really skip ahead. (This roadmap is the PM counterpart to the AI Systems Architect roadmap we published; same shape, different seat.)

Layer 1: Stack fluency (not stack mastery)

You don't need to write the code. You do need to know enough about how the system works to scope it, sequence it, and call BS when an engineer is over- or under-engineering a piece of it.

The minimum surface area, from where we sit:

• How LLMs actually work, at the API level. What a prompt is, what a system prompt is, what the context window is, what tokens cost, what temperature does, what tool use is.
• RAG vs. fine-tuning vs. prompt engineering. What each one is good for, what each one costs (in engineering time, latency, and dollars), and how to tell which one a problem actually needs. Most "we need to fine-tune" requests don't.
• Agents and orchestration. The difference between a single agent with tools, a fixed multi-step pipeline, and a dynamic multi-agent system. The right choice depends on whether the workflow is predictable.
• Guardrails and grounding. How the system stops itself from doing the wrong thing, and how it knows whether its answer is supported by source data. (We wrote up how contextual grounding actually works in production if you want one concrete example.)
• Model selection. Haiku vs. Sonnet vs. Opus, when each is the right call, why "use the biggest model" is a bad default. Cost and latency change by an order of magnitude across tiers.

You're not learning this to build it. You're learning it to scope it. A PM who can't tell whether a feature needs RAG or fine-tuning can't sequence the work, can't size the team, and can't push back when engineering says "this'll take six weeks."

Where to start: build one thing end-to-end with no help. A trivial RAG app over five PDFs, deployed somewhere real. Two weekends. The point is to feel where the friction is.

Layer 2: Eval design, the new spec writing

This is the layer we'd bet on most. It's the highest-leverage skill for an AI PM, and almost nobody talks about it.

In a deterministic world, a spec said "when the user clicks Submit, validate the form and POST to /orders." In a probabilistic world, that's no longer enforceable. There's no clean assertion. So instead of writing assertions, you write evals.

An eval is a labeled dataset of inputs paired with judgments. Sometimes the judgment is a known correct answer ("for this input, the right output is X"). More often it's a rubric ("for this input, a good output meets criteria A, B, and C"). You run the system across the dataset and measure how often it meets the bar.

Eval design is the new spec writing. The skills transfer almost cleanly. Both are about being concrete about what "done" looks like. But eval design has its own discipline:

• Choosing what to label, in what proportions. Your eval set has to include the boring middle, the edge cases, and the adversarial inputs.
• Picking the right metric per output type. Exact match? Semantic similarity? LLM-as-judge with a rubric? Each has gotchas.
• Versioning evals as the product evolves, so you can detect regressions when a model updates or a prompt changes.
• Knowing when an eval is overfit (the system passes your eval but flunks in production).

We've seen smart teams ship the wrong model into production because their eval set was 50 happy-path examples and zero "the user asked something out of scope" examples. The model looked great on the eval and crumbled on launch day.

If you do nothing else from this roadmap, learn to build good eval sets. It's the single most valuable thing a PM can bring to an AI project.

Where to start: take any AI feature you've worked on or used, and write the eval set you wish someone had built for it. Fifty rows. Inputs in column A, "good output looks like…" in column B. You'll immediately discover how hard it is to be concrete about "good."

Layer 3: Cost, latency, and accuracy as product constraints

In traditional software, cost and latency mostly live with engineering. In AI, they're product decisions, because every choice that affects them also affects the user experience.

A few examples from real projects:

• A support agent that uses Opus answers beautifully and takes 11 seconds. The same agent on Haiku answers adequately in 2 seconds. Which is the product? That's not an engineering call. It's a product call about what the user actually needs in that moment.
• A document-processing pipeline that runs a guardrail check on every output adds 400ms and a few cents per call. On a high-volume use case that math kills the feature. On a low-volume, high-stakes use case it's table stakes. PM has to make the call.
• A RAG system can retrieve 5 chunks or 50. Fifty chunks means better grounding, more cost, slower response, and more noise for the model to wade through. There's no "right" answer, only a product judgment per use case.

The traditional PM instinct is to push these to engineering as "non-functional requirements." In AI work that's a cop-out. You need to own them as first-class product decisions, the same way you'd own information architecture or pricing tiers.

The mental model that helps: think of cost-latency-accuracy as a triangle where you can usually pick two. Cheap and fast → less accurate. Cheap and accurate → slower. Fast and accurate → expensive. Your job is to know which corner your user actually values, by use case.

Where to start: for any AI feature in your product (or a competitor's), guess the per-call cost and the p95 latency. Then go ask an engineer what they actually are. The gap between your guess and reality is the gap you need to close.

Layer 4: Failure-mode UX

The single most underbuilt part of every AI product we've reviewed.

When the model is wrong, unsure, blocked by a guardrail, hitting rate limits, or returning something off-topic, what happens? In most products: nothing graceful. The user sees a hallucinated answer, a generic error, or a chatbot that says "I can't help with that" with no path forward.

That's not the engineer's fault. The fallback IS the product, and product owns it.

What good failure-mode UX looks like:

• Refusals that route forward. When a guardrail blocks an answer, the user gets handed to a person or to a different flow, not dumped at a dead end. (In one auto insurance pilot, the difference between "I can't help with that" and "let me connect you with a representative, here's what you asked, and here's what's in your policy" was the difference between an angry call and a satisfied one. Same block, different UX.)
• Confidence surfaced honestly. When the model is uncertain, the user should know, even if it's just "double-check this number against your statement." Users tolerate uncertainty fine. They don't tolerate confident wrong answers.
• Error states that are recoverable. Rate-limited? Show a queue position. Model down? Show a cached or simpler answer. Don't just throw a spinner forever.
• Eval coverage that includes the fallback paths. If "the fallback fires" is part of your product, it should be part of your eval set. Most aren't.

This is pure PM work. Engineers will build whatever fallback you spec; if you don't spec one, you'll get whatever's quickest.

Where to start: pick any AI product you use regularly. Try to deliberately break it. Ask things out of scope, things it shouldn't answer, things that need a human. Write up what happens. That list is a starter design doc for what your own product's failure-mode UX should look like.

What still matters from classic PM (and what matters less)

A short, opinionated split:

Matters more in the AI era:

• Crisp written specs. With ambiguity everywhere, the PM who writes clearly wins. Imprecise prompts produce imprecise behavior, and an imprecise spec is just an imprecise prompt in another medium.
• Scope discipline. AI features sprawl naturally. "While we're at it, let's also…" is how a $40-per-session bill happens. Killing scope is more valuable, not less.
• Customer empathy for the unsure user. Users encountering a probabilistic system feel less in control. PMs who can design for that anxiety win.
• Stakeholder framing. Executives expect AI to be magic. The PM who can reframe "the model gets it right 92% of the time" as a business outcome (and what to do with the other 8%) earns trust.

Matters less:

• A/B test orthodoxy. Variance in AI outputs and changing model versions make naive A/B tests noisy. You still need experimentation, but the methodology has to evolve.
• Jira ceremony. The work changes too fast for sprints designed around shipping deterministic tickets. Smaller, faster loops with eval-driven definitions of done work better.
• Feature parity instincts. Copying what a competitor's AI feature does isn't a strategy. Their eval set isn't yours, their data isn't yours, their fallback UX isn't yours. The visible feature is the smallest part of what makes it work.
• OKR theater. "Ship the AI chatbot by Q3" is a vanity OKR. "Reduce average support resolution time by 30% with AI assistance, at no degradation to CSAT" is a real one.

A 90-day starting plan

If you're a working PM and want a concrete on-ramp:

• Weeks 1–2: Build the trivial RAG app from Layer 1. Pay your own money for the API calls so you feel the unit economics.
• Weeks 3–4: Write an eval set for an AI feature in your current product (or a competitor's). 50 rows. Get a sense of how hard "good" is to define.
• Weeks 5–6: Shadow one of your engineering teams during a model-selection or prompt-iteration cycle. Don't drive. Just listen. You'll learn where PM has been absent.
• Weeks 7–8: Take one AI feature you own (or want to own) and rewrite its spec across all four states: success, ambiguity, refusal, failure. Notice how much of it you didn't have written down.
• Weeks 9–10: Build a one-page proposal for an AI feature that includes cost-per-call, latency targets, eval methodology, and failure-mode UX. Pitch it internally.
• Weeks 11–12: Volunteer for the next AI feature on your roadmap and own it end-to-end with the playbook above.

In three months you won't be a senior AI PM. You'll be the most useful PM in the room when AI work shows up, which is enough to compound from.

The bottom line

The traditional PM craft isn't obsolete in the AI era. It's the foundation. What changes is the medium: from deterministic features with clean specs to probabilistic systems with evals, fallbacks, and cost-accuracy tradeoffs that you own as a product person.

The PMs who'll lead the next wave of AI products aren't the ones who learned prompt engineering. They're the ones who learned to spec a probabilistic system honestly, build the eval set that defines "good enough," own the failure-mode UX, and tell a stakeholder which 8% of the cases the model gets wrong and why that's okay.

That's the role. The roadmap above is how you get there.

---

Written by Ramesh Nori. If this was useful or you have feedback, reach me at cloudbuckle@gmail.com.