Super Data Science: ML & AI Podcast with Jon Krohn · Jon Krohn

959: Building Agents 101: Design Patterns, Evals and Optimization (with Sinan Ozdemir)

·1 hr 5 min·4 clips

An agent is an LLM with tools; a workflow is a deterministic data and code path.

1. Super Data Science Podcast episode 959 focuses on Building Agentic AI, the new book from Sinan Ozdemir and Jon Krohn. 2. Jon Krohn is the host, and Sinan Ozdemir is an AI entrepreneur, author of 10 books, and the guest who wrote the episode’s featured book. 3. The episode asks how to build, evaluate, and optimize agentic systems without tying the advice to one model release. 4. Sinan Ozdemir says the title shifted from “Building Applied AI” to “Building Agentic AI” because not every system in the book is a pure agent. 5. He defines an agent as “an LLM…with access to tools,” such as Google search or reading a file from a desktop. 6. He defines a workflow as a deterministic data-and-code path where the LLM never chooses the next step. 7. He says most production AI deployments are workflows because the system forces prompts, embeddings, retrieval, and responses in a fixed order. 8. Jon Krohn asks how to decide where a product belongs on the deterministic-to-agent spectrum. 9. Sinan Ozdemir says he starts by asking clients to describe the existing process and then looks for conditionals. 10. He uses recommendation engines and age restrictions in Ireland as examples of branching logic that can push a system toward a hybrid or an agent. 11. Sinan Ozdemir says his book stays model-agnostic on purpose, because model names change faster than the underlying task. 12. He frames this as a machine-learning-engineering choice: the task and domain matter more than the current chatbot name. 13. On model families, he contrasts auto-regressive models like GPT, Claude, and Gemini with auto-encoding models like BERT. 14. He says auto-regressive models predict the next token from context, while auto-encoding models fill in a blank anywhere in a sequence. 15. He notes that BERT can do embeddings and classifications with far fewer parameters than a large GPT model. 16. He gives a practical parameter-count heuristic: under 10 billion is “small,” 10 to 100 billion is “medium,” and 100 billion plus is “large.” 17. He says smaller models can still handle retrieval, RAG, and web search, while medium models are better for longer-horizon agent tasks. 18. Jon Krohn and Sinan Ozdemir discuss context windows, and Sinan explains that long-horizon agents need context windows that the model can actually reason over. 19. He describes a “needle in the haystack” experiment where a birthday hidden in a crowded prompt may still be missed by some models, and he says chapter three on experimentation is a core part of the book. 20. Listeners who build AI products, evaluate LLMs, or compare Claude, Gemini, and OpenAI tools will get the most from this episode, while people wanting a pure product-news roundup may skip it.

As heard by us

A practical guide to agent design, evaluation, and deployment for builders.

Jon Krohn presents Sinan Ozdemir's Building Agentic AI as a practical 101 guide to designing, fine-tuning, optimizing, and deploying agentic systems.

Read the full review in PlayNext →

Why you'd press play

You want a clear starting point for building and deploying agentic AI.

Read the full recommendation in PlayNext →

Listen to the show on

Apple Podcasts Spotify

Play clips from this episode More from Super Data Science: ML & AI Podcast with Jon Krohn