Build an AI Agent (From Scratch): Agents that reason, plan, and act autonomously
$159.99 Original price was: $159.99.$27.99Current price is: $27.99.
🔒 Secure Checkout ⚡ Instant Digital Download
“A practical guide for anyone who wants to move beyond demos and create agentic systems with confidence.”
— Elliot Kim, Coupang Inc.
Build an AI Agent (From Scratch) is an end-to-end, framework-free master training suite comprising a 336-page comprehensive book (PDF/ePub) and a 39-lesson companion video course (4h 7m) published by O’Reilly Online Academy.
Designed for Python engineers who want to move past superficial wrappers and black-box abstractions, this guide walks you through designing, building, and deploying autonomous agentic systems entirely from the ground up. Guided by AI engineers Jungjun Hur and Younghee Song, you will construct a production-grade research agent capable of navigating complex, multi-step challenges from the rigorous GAIA (General AI Assistants) benchmark.
Core Philosophy: Framework-Free Engineering
Most agent tutorials rely on heavy abstractions that obscure what happens under the hood. This program strips away third-party frameworks to teach you the bare mechanics directly in Python:
-
True Agent Loops vs. Static Workflows: Distinguish hardcoded sequential scripts from genuine, LLM-directed autonomous decision cycles.
-
Context Engineering: Master the operational discipline of structuring prompts, messages, and state to permanently prevent context rot and the “Lost in the Middle” degradation curve.
-
Stateless API Control & Structured Outputs: Enforce deterministic schemas using Pydantic, implement error handling, and orchestrate dynamic tool invocation routines over stateless foundation model APIs.
-
Model Context Protocol (MCP) Integration: Standardize tool execution and data fetching by wiring your agent directly to external servers and environments using the open MCP standard.
-
Sandboxed Code-Act Paradigms: Grant agents secure execution boundaries in sandboxed cloud environments to generate Python code, execute CLI commands, and compose tools on the fly.
-
Multi-Agent Coordination (A2A): Scale single-agent workflows into collaborative multi-agent topologies using the Agent-to-Agent protocol to isolate responsibilities and maximize token efficiency.
-
Production Observability & Evaluation: Instrument your architecture with OpenTelemetry and construct an automated LLM-as-a-judge evaluation pipeline to trace trajectories, detect infinite loops, and grade step-by-step reasoning quality.
What You Will Build and Learn
-
Standalone Agent Core: Build a complete, functional ReAct engine from scratch in raw Python without external agent libraries.
-
RAG & Knowledge Bases: Integrate vector-based retrieval augmented generation with local filesystem traversal and dynamic web search.
-
Memory & State Persistence: Implement hierarchical memory models that allow agents to retain long-term state and adapt based on past task failures.
-
Planning & Procedural Reflection: Equip systems with explicit self-correction loops to eliminate hallucinations, circular reasoning, and tool thrashing.
-
Video Code Walkthroughs: Watch all code implementations from the foundational chapters assembled step-by-step on screen across 39 guided lessons.
Technical Specifications
| Attribute | Book Details | Video Course Details |
| Publisher | Independent / Technical Release | O’Reilly Online Academy |
| Format | 336 Pages (PDF, ePub) | 101 Video Lessons (13h 1m) |
| Language & Stack | Python 3, Pydantic, OpenTelemetry | Python 3, Live Coding |
| Target Level | Intermediate Python Developers | Introductory to Intermediate |
| Prerequisites | Working Python knowledge, basic ML/LLM familiarity | Python basics, OpenAI API access |
Curriculum Overview
Part 1: The Core Foundation
-
Chapter 1: What is an AI Agent? — Deconstructing autonomy, perception-action cycles, and the shift from deterministic pipelines to agentic behavior.
-
Chapter 2: The Brain of AI Agents: LLMs — Working with foundation models, token mechanics, stateless API management, and prompt structuring.
-
Chapter 3: Enabling Actions: Tool Use — Dynamic function calling, strict schema enforcement with Pydantic, and secure input/output serialization.
-
Chapter 4: Implementing a Basic ReAct Agent — Synthesizing reasoning and action loops from scratch; hands-on video demonstration.
Part 2: Capabilities, Orchestration, and Production
-
Chapter 5: Building Knowledge Bases with RAG — Ingesting unstructured text, vector search pipelines, and filesystem exploration.
-
Chapter 6: Adding Memory to Your Agent — Short-term scratchpads, long-term state persistence, and hierarchical context pruning.
-
Chapter 7: Planning and Reflection for Complex Tasks — Graph-based planning algorithms, runtime self-reflection, and recovery from failed tool calls.
-
Chapter 8: Empowering Agents with Code Execution — The Code-Act paradigm: executing arbitrary code inside sandboxed runtimes.
-
Chapter 9: Orchestrating Multi-Agent Systems — Agent-to-Agent (A2A) communication protocols, specialized sub-agent delegation, and shared context routing.
-
Chapter 10: Evaluating Agents — Tracing execution with OpenTelemetry, building automated LLM-as-a-judge evaluators, and benchmark scoring against GAIA.
-
Appendix A: OpenAI API Key Management — Environment setup, authentication, and token safety.
Target Audience
This suite is crafted for Python software engineers, machine learning practitioners, and technical architects who want to master how agentic AI works at the architectural level. If you want to build autonomous systems that reliably work in production—without black-box framework lock-in—this material provides the complete technical blueprint.
About the Authors
-
Jungjun Hur is an independent AI and data engineer specializing in production-grade agentic systems and e-commerce workflows. He is also the author of Practical AI Application Development Using LLMs.
-
Younghee Song is an enterprise AI consultant at PricewaterhouseCoopers (PwC), advising organizations on scaling production AI architecture. Both authors are based in South Korea.
13 reviews for Build an AI Agent (From Scratch): Agents that reason, plan, and act autonomously
Related products
-

ChatGPT For Dummies
0 out of 5$45.00Original price was: $45.00.$14.99Current price is: $14.99. Add to cart -

System Design: A guide to building scalable, reliable, and high-performance systems (English Edition)
0 out of 5$49.95Original price was: $49.95.$23.19Current price is: $23.19. Add to cart -

Flutter in Action
0 out of 5$169.99Original price was: $169.99.$24.99Current price is: $24.99. Add to cart -

The Art of Computer Programming (6 books)
4.95 out of 5$499.99Original price was: $499.99.$49.99Current price is: $49.99. Add to cart

jcat (verified owner) –
Every agent framework I’ve used works fine until it doesn’t — and then you’re at your desk past midnight, reading someone else’s source code, trying to find which invisible layer just broke. This book exists so that never happens to you: it makes you build every one of those layers with your own hands, first. I came to it already comfortable calling OpenAI’s API but hazy on everything between “call the model” and “agent that gets things done.”
The premise is simple and it holds up for all ten chapters: skip the frameworks, build the agent yourself, piece by piece, in plain Python. You start with a raw LLM call in chapter 2, add tool calling in chapter 3, and by chapter 4 you have a working ReAct loop, with an ExecutionContext class that becomes the spine every later chapter extends. Nothing gets rewritten as the book goes — it accumulates. By the time you reach multi-agent orchestration in chapter 9, you’re adding one more capability to the same agent you built in week one, not starting over on a new toy example. That structural discipline is the book’s biggest strength, and it’s rarer than it should be in this genre.
What sold me is that the authors test their own claims rather than asserting them — at least in the opening chapters. Chapter 2 runs four different LLMs against twenty GAIA benchmark problems and lays out the accuracy numbers in a table that shows exactly how little a bare model can do without tools. Chapter 4 repeats the test with the finished ReAct agent: adding web search took GPT-5 from 2 correct out of 10 to 7 out of 10 on problems GAIA itself flags as needing web access. That’s real, falsifiable evidence, not a claim you’re asked to take on faith. My one complaint: the discipline doesn’t hold. RAG, memory, planning, code execution, multi-agent — each new capability gets built and demonstrated, but none gets its own before-and-after GAIA score the way tool-calling did in chapter 4. GAIA problems keep surfacing as example material for the rest of the book, but the measured comparison quietly stops right where I most wanted to watch the number move again.
The code supports every chapter with working implementations of the key concepts — chapters 2 through 9, at least. The companion repo is a real, organized Python package, with Jupyter notebooks that track the text almost line for line. Every module compiled clean, and the commit history shows the authors returning months after publication to re-sync notebooks against the book’s exact listings — the kind of upkeep most companion repos never get. One thing worth knowing before you start: the exercises require an OpenAI API key, and it isn’t free — though the authors’ own estimate puts the cost of working through every exercise in the book at under $10.
Chapter 10 is where that coverage thins. It implements the LLM-as-judge rubric scoring — answer relevancy, citation reliability, requirement compliance — as real, runnable code, but nothing in today’s repo covers the chapter’s other half: the OpenTelemetry instrumentation, the traces and spans, the CI/CD evaluation-gated deployment loop.
To summarize, this book actually delivers on its core promise. By the last page you’ll have built, by hand, real (not toy) agent workflows: a ReAct think-act loop from raw API calls; function calling and MCP-based tool integration; a working RAG pipeline with real vector search; context management that survives long conversations through summarization and long-term memory instead of just overflowing; planning and self-reflection so the agent catches its own mistakes instead of looping forever; sandboxed CodeAct-style execution; and multi-agent orchestration, both in-process and across a network via A2A. Every one of those is something you will have written and run with your own hands, chapter by chapter, by the time you close the book.
Dr. Erin Colvin (verified owner) –
Even though I hold a PhD in Computer Science, creating AI Agents is a new field and something I am having to learn and this book, Build an AI Agent (From Scratch), is an excellent introduction to one of the most exciting areas of modern AI development. This book was very easy to read and understand and rather than simply describing what AI agents are, the book focuses on how they actually work and how the pieces fit together to create systems capable of reasoning, planning, making decisions, and taking action. For those people, like me, who like to understand the what and why about a process, this book does just that.
One of the book’s greatest strengths is its practical approach. Complex concepts are broken down into manageable components, making the material approachable without oversimplifying the underlying engineering. The progression from basic concepts to autonomous agent behavior makes it especially useful for software developers who want to move beyond simply calling an LLM and begin building more capable AI-driven applications. This is helpful when our everyday life isn’t filled with complex technology so picking this book up doesn’t require a refresher course or study manual just to catch up.
I particularly appreciated the emphasis on building agents from the ground up. Understanding the mechanics behind reasoning loops, planning, tool use, memory, and autonomous execution provides a much stronger foundation than relying entirely on prebuilt agent frameworks. It gives the reader a better understanding of what is happening behind the scenes and, more importantly, how to troubleshoot or improve an agent when it does not behave as expected. It is also helpful for customizing agents if a prebuilt agent is chosen, those changes can be made in the future.
The book also does a good job of connecting AI concepts with traditional software engineering principles. It reinforces the idea that effective AI agents are not simply clever prompts, they are engineered systems that require thoughtful architecture, clearly defined responsibilities, reliable tools, appropriate safeguards, and well-designed decision-making processes.
Overall, this is a valuable resource for developers, software engineers, and technical professionals who want a practical understanding of agentic AI. It provides enough technical depth to be useful while remaining accessible to someone who is relatively new to building autonomous agents.
I highly recommend this book for anyone who wants to understand not just how to use AI agents, but how to actually build and engineer them.
M Agarwal (verified owner) –
I have used agent frameworks long enough to build things I could not explain. This book fixed that. Instead of teaching yet another framework, Hur and Song have you build the whole thing yourself in plain Python: the ReAct loop, tool calling, MCP, RAG, memory, planning, sandboxed code execution, multi-agent orchestration, and finally evaluation. By chapter 4 you have a working agent you wrote from scratch, and by chapter 10 you know how to prove it actually works, which is rarer in this field than it should be.
Two things make it land. First, everything is measured. The authors keep returning to the GAIA benchmark, so you literally watch your agent evolve from a confidently guessing chatbot into something that can work out how long Eliud Kipchoge would need to run to the Moon. Second, the code is genuinely runnable on an ordinary laptop, organized into one tidy package with notebooks for every chapter. By the end, no framework source code looks mysterious anymore, because you have personally installed every rabbit in the hat.
Honest caveats: the chapters build on each other, so this is a cover-to-cover book, not a reference you dip into. You will need a small pile of API keys, and the experiments cost real money (the authors cheerfully print the dollar amounts, which I respect). And in a field moving this fast, a few sections will age, though the fundamentals will not.
If you want to ship an agent by Friday, grab a framework. If you want to understand agents well enough to debug one at 2 a.m., read this. My agent still occasionally does something baffling, but now I know exactly which of my own lines to blame. Highly recommended.
Madhuri Lokesh (verified owner) –
I really enjoyed reading this book. I have worked with Google ADK, Microsoft Agent Framework, LlamaIndex and LangChain, so I was interested in seeing how much of an agent can be built from the basic pieces without depending on a framework.
What I liked most is that the book does not just explain the concepts, it actually builds them step by step. LLM calls, tool calling, ReAct, RAG, memory, planning, CodeAct, multi-agent and evaluation. When you use frameworks, many of these things are given to you as abstractions. This book helped me understand the building blocks behind those abstractions and also what could be happening behind the APIs of frameworks like LangChain and LangGraph.
Of course, I would not take the code from the book and directly put it into a production system. There are many more things to consider around security, reliability, error handling, observability and scaling. But I don’t think that is really the purpose of the book. It is more about understanding how the pieces work and putting them together yourself.
For me, this was useful even though I already work with agent frameworks. After reading it, I feel I have a better understanding of what those frameworks are trying to abstract for us. I would recommend it to anyone who wants to go beyond just using an agent framework and understand the fundamentals better.
Thomas To (verified owner) –
Every agent framework I have used solves the same problem the same way, abstracting away the loop, the context management, and the tool orchestration before you ever see how any of it works. This book takes the opposite approach and it pays off. Ten chapters spent implementing an agent from the raw LLM call outward through ReAct, RAG, memory, planning, CodeAct, and multi agent coordination, with each chapter extending the same ExecutionContext instead of starting over, built the kind of pattern recognition no framework quickstart has given me.
What sets this book apart is where it spends its attention. It stays on the software engineering and applied ML side of building agents, tool calling, retrieval, context windows, evaluation loops, rather than the underlying transformer architecture that plenty of other books already cover well. The skills that transfer are the ones any framework eventually asks of you anyway, structuring state, deciding what a tool returns and how the agent recovers from a bad call, and measuring whether the system actually works in chapter 10 instead of assuming it does once it runs.
In a field where new frameworks ship every quarter, understanding the machinery underneath them is the one thing that stays useful. After finishing this book I can open the source of any popular agent framework and recognize exactly what it is doing and why, instead of trusting the abstraction on faith. That is the real return here, not a tutorial for one tool but a mental model that outlasts all of them. Five stars, and the clearest case I have read for building an agent yourself before you let a framework build one for you.
Paul Pollock (verified owner) –
Somewhere in chapter 1, I had to admit that most things I’d built and called “agents” were just chatbots with a search bar bolted on. The workflow vs. agent distinction, who decides the next step, the model or my code, rewired how I think about every LLM project I’ve touched since.
What actually helped: the GAIA experiment where accuracy jumps once tools get added. Seeing the number move, not just reading a claim about it, made “agents need tools” click in a way no blog post managed.
Friction point: once ExecutionContext, LlmRequest, and callback layering show up together, I had to slow down and reread. Not bad writing, just dense if async Python and dataclasses aren’t already comfortable territory for you.
If you’ve called an API and shrugged at “agent frameworks” without knowing what’s underneath, this rebuilds that intuition from scratch. If you want copy-paste production code or a framework comparison, look elsewhere.
Ranjit Sahai (verified owner) –
Ever since ChatGPT was released in November 2022, the pace of change in the world we call AI has been nothing short of astronomical. We have gone from personal chatbots that answer questions, to customer-facing chatbots that provide support, to agentic systems that can do things that we had thought to be, oh so far away into the future, instead of a mere four years since then.
Regardless of which type AI agent you use or want to develop, or the modalities (text, image, audio, video) it supports, the one thing that remains unchanged, as Build an AI Agent from Scratch explains, is the brain the agent relies on—the large language model (LLM).
Whether you wish to tap into the AI-powered capabilities being delivered by your mission-critical domain-specific software, or to build such an agent for use internally within your organization, there is no substitute to understanding how agents work, regardless of the type of agent they might be. The key concept you drill down to, throughout the book, is that of context-engineering strategies and the Model Context Protocol (MCP) that enables agent-to-agent communication between your generative chatbot (ChaptGPT, Copilot, Gemini, Claude, etc.) and the domain-specific specialized engine or agentic system.
To help you in your journey to building specialized agent systems, or to even leveraging them, the book walks you through building one from basic principles. It takes you under the hood of what agent-building frameworks do, no matter what the specific framework might be. The book is superbly written. Even as a hobbyist programmer who may never build an AI agent, the book grabbed and held my attention and I walked away with an understanding of what makes agents tick and how I can improve the results I get from them by significantly reducing instances of hallucinations they can be prone to. Allow me to share with you Figure 1.7 from the book that provides an overview of the journey this book takes you through.
Bipin Chadha (verified owner) –
I just finished reading “Build an AI Agent (from scratch)” by Jungjun Hur and Younghee Song. This is a good starter book for someone interested in learning how AI Agents are implemented on top of popular LLMs without using any existing framework. This will help you learn and understand how some of the popular frameworks work. The authors do a great job distinguishing between workflows and agents, perhaps better than any that I have seen. Having said that, I think their description falls short. The author’s definition of agents revolves around LLMs. I believe that LLMs are neither necessary nor sufficient for Agents. Most people will agree (including the authors) that you need other capabilities beyond LLMs to make agents work. The part that I want to argue is that you can have intelligent agents without LLMs. There are many examples of agents in software as well as in biology where a LLM is not present. The opportunity to explore what gives something “agency” got missed.
The authors rightly make the point that building agents adds complexity and cost and one should carefully understand the tradeoffs before building agents. The authors walk through some simple examples on how you might build an agentic application, and offer several tips and lessons in building agents, adding tools to agents, setting up MCP servers, adding simple reasoning and planning capabilities, and giving memory and skills to agents.
Then we come to the topic of multi-agent systems. This topic is covered very superficially and a few examples are provided that involve a handful of agents. True value of agents will come from applications that require a large number of decentralized agents – these are not discussed. To be fair, I do not see most others discussing it either.
Given how risky agentic systems are, I wanted to see better coverage of governance, evaluation and validation of agents. I did like their statement that “how to evaluate an autonomous system whose behavior is nondeterministic and whose failure modes evolve, is arguably the hardest problem in applied AI engineering today.” Perhaps the next version of the book can go into more detail of these topics.
A. Zubarev (verified owner) –
It is a fundamental book in your journey to advancing in the AI field of expertise.
Having mastered and used the regular AI request-response loop (basically riding an agent to user interaction) a problem arises how you can advance beyond this daily and becoming mundane AI use?
There is such a way: basically in building your own, custom agent!
This is the book to make it happen exactly as you want with minimal effort and research on your part.
The benefits are very real, they will lift you off and above wearing your keyboard off to building complete systems, research or projects with some initial effort first and then reaping the benefits all the way long.
The book makes almost every user able to progress as it starts with the very basics of AI and gradually increasing the complexity: from the fundamentals as LLM to MCP and knowledge bases to advance searches, memory and skills.
Reading the book, and soon to experiment I liked the multi-agents chapter the most.
In terms of improvements: I would recommend the author to dedicate more material to deploying the agents and their security. Then more on how to troubleshoot issues and lastly, it would be good to provide exercises.
In closing, I would like to recommend an awesome companion book, also from Manning: “AI Agents in Action”.
I still rated the book as a solid five for both content and ease of implementing the agents. It delivers on the promise.
QuirkyGal (verified owner) –
Many books about building agents focus on the theoretical aspects of agents and rely on frameworks to implement those agents. This approach might work well for building fast, but it does not help develop a deeper understanding of the system or debug it when things go wrong. This is where this book comes in. Building an Agent from Scratch focuses on building agentic systems from scratch to help build the intuition for how all of the components of the system work together. The first few chapters focus on the fundamentals of agents and their building blocks like LLMs, prompt engineering, and differentiating between workflows and tasks where agents would be beneficial. This is very important to cover if you are not too familiar with agents. After setting up the fundamentals, Chapters 3 and 4 move on to using tools through the model context protocol and finally discussing the ReAct framework for how agents can be built to take information and iterate on it. The following chapters focus on improvements that can be made to the agentic system you are building including how to set up RAG, agent memory, and to improve the agent’s results. The final chapters of the book focus on setting up the orchestration of the agents and evals necessary for evaluating the agent’s outputs.
Overall, I was very impressed with the quality of this book. Being able to build the system from scratch while learning the reason each component is needed really helps form the intuition needed to help build resilient agentic systems. Being able to code the system piece by piece while reading this helps build enduring understanding. This is another great title in the “Building from Scratch” series. I highly recommend anyone wanting to learn how to build agent systems read this book cover to cover.
Santosh Shanbhag (verified owner) –
Most online tutorials teach you how to spin up an AI agent in five lines of code using popular frameworks. But when those agents fail or loop endlessly in production, you’re left guessing. That is where Build an AI Agent (From Scratch) by Jungjun Hur and Younghee Song makes a massive difference.
Instead of wrapping your projects in black-box libraries, this book guides you step-by-step to build a fully functional research agent in Python from the ground up. You will implement the core mechanics yourself: the ReAct loop, message tracking, tool execution, and custom memory systems. What truly sets this book apart from easily searchable web content is its focus on Context Engineering and Harness Engineering. You will learn how to design the environment in which your model runs, optimize token efficiency through hierarchical agent skills, coordinate multi-agent teams, and establish rigorous LLM-as-a-judge evaluation pipelines using the GAIA benchmark.
For me, this book is a complete game-changer. It shifted me from a passive framework consumer to an architect with “X-ray vision” into autonomous systems. You’ll gain the precise mental models needed to systematically debug production failures rather than guessing with prompt adjustments. Whether you need to deploy reliable enterprise workflows or read any framework’s source code as if it were your own, this book delivers the engineering depth to build AI systems you can actually trust.
AB (verified owner) –
The content of this book is excellent and I’ve learned a lot from it. In addition to learning a lot about the brick and mortar of building AI agents from scratch, you’ll also learn quite a bit about patching Jupyter notebooks. There are little things that won’t compile when you play the cell, which has been frustrating. In all, it’s an excellent book, but it would be nice if the notebooks worked “out of the box” from the accompanying repo. There are no “show stopper” issues, but if I’m trying to learn a new domain, it would be nice not to fight the tooling for learning it.
Y. Arazi (verified owner) –
This book gives you a real, working understanding of how agent harnesses work, the kind of thing behind tools like Claude Code, Codex, or OpenCode. It is not just theory. There is a lot of Python code throughout, and you build things as you go, starting from a basic OpenAI API call and working your way up to a working agent.
What I liked most is how it lays out the core loop: the LLM tool loop, where a prompt goes out with the system prompt, tools, and user messages, the model responds, a tool gets called, the result comes back, and the loop continues. Once you get that loop, a lot of what looks like magic in agent harnesses stops being magic. The book also spends real time on context engineering, and makes a point that stuck with me: a bigger context window is not automatically better. That is a good instinct to build early if you want your agent to actually be accurate instead of just verbose.
From there it goes deep. There is a full treatment of tool calling, MCP, and the broader agent harness ecosystem, plus a chapter on ReAct (reasoning and acting) that is genuinely one of the better explanations of the pattern I have read. Memory, benchmarking, code execution as part of the agent loop, and multi-agent systems (sub-agents, orchestration, and how to evaluate agent output) are all covered too, and by the end you have gone from beginner material to genuinely advanced territory.
Even if you never plan to build your own agent harness, understanding how one works under the hood makes you a better user of the ones that already exist. That alone makes this worth reading.
My one honest caveat: this space moves fast, and some of the specifics here will likely feel dated sooner rather than later. That is not really a knock on the book, it is just the nature of the field right now. Treat it as a strong foundation rather than a permanent reference.
Comprehensive, hands-on, and a genuinely useful resource if you want to understand agent harnesses from the ground up.