LangGraph

We've seen many people successfully building agentic applications now, with key improvements over the last year. Function calling LLMs have made tool use more predictable and stable. Specific tools like search have been adapted for agentic use. When you query a search engine, it returns multiple links to follow for answers. However, agents need answers they can reference directly and predictable formats for their results, which is what Agentic Search provides.

Agentic Workflow

Example: Writing a paper together:

1. Planning: Outline the paper.
2. Research: Run queries, compile documents.
3. Writing: Draft the paper.
4. Review: Make suggestions, iterate.

This iterative workflow improves the final product compared to a single prompt-based approach. LLMs can be prompted to write outlines, drafts, revisions, and conduct research.

Key Design Patterns of Agentic Workflows

1. Planning & Task Decomposition: Think through steps, create outlines.
2. Self-Reflection: Iteratively improve results with multiple LLMs providing feedback.
3. Memory: Track progress and results across multiple steps.
4. Tool Use: Know available tools and their usage (e.g., search tools).
5. Multi-agent Communication: Assign unique roles to LLMs with specific prompts.

Some other frameworks include:

1. ReAct¹: Early paradigm for reasoning and action in agents.
2. Self-Refine²: Example of iterative refinement.
3. AlphaCodium³: Coding agent using flow engineering.

Why Graphs?

The prototypical LLM agent uses a "reasoning and action" (ReAct) (opens in a new tab) style design, applying an LLM to power a basic loop with the following steps:

• Reason and plan actions to take
• Take actions using tools (regular software functions)
• Observe the effects of the tools and re-plan or react as appropriate

While LLM agents are surprisingly effective at this, the naive agent loop doesn't deliver the reliability users expect at scale (opens in a new tab). They're beautifully stochastic. Well-designed systems take advantage of that randomness and apply it sensibly within a well-designed composite system and make that system tolerant to mistakes in the LLM's outputs, because mistakes will occur. AI design patterns should apply applicable good engineering practices from Software 2.0. Some similarities include:

• AI applications must balance autonomous operations with user control.
• Agent applications resemble distributed systems in their need for error tolerance and correction.
• Multi-agent systems resemble multi-player web apps in their need for parallelism + conflict resolution.
• Everyone loves an undo button and version control.

LangGraph

The above agents are defined by cyclic graphs, supported by LangChain's LangGraph (opens in a new tab). LangGraph is a Python library for building complex, scalable AI agents using graph-based state machines. Single and Multi-agent flows can be represented as cyclic graphs, and this allows for extremely controlled "flows". LangGraph allows you to easily describe and orchestrate these control flows.

While LangChain allows you to define chains of computation (Directed Acyclic Graphs or DAGs), LangGraph specifically introduces the ability to add cycles, enabling more complex, agent-like behaviors where you can call an LLM in a loop, asking it what action to take next.

LangGraph's primary StateGraph (opens in a new tab) abstraction is designed to support these and other needs, providing an API that is lower level than other agent frameworks such as LangChain's AgentExecutor (opens in a new tab) to give you full control of where and how to apply "AI."

It extends Google's Pregel (opens in a new tab) graph processing framework to provide fault tolerance and recovery when running long or error-prone workloads. When developing, you can focus on a local action or task-specific agent, and the system composes these actions to form a more capable and scalable application.

Its parallelism and State reduction functionality let you control what happens if, for example, multiple agents return conflicting information. And finally, its persistent, versioned checkpointing system lets you roll back the agent's state, explore other paths, and maintain full control of what is going on.

Some useful additional capabilities for building agents:

1. Agentic Search: Search tools adapted for agentic use.
2. Human Input: Guide agents at critical points.
3. Persistence: Store and return to the current state of information, useful for debugging and productionalizing agents.
   • Also useful for multiple conversations at the same time, or for Human-in-the-loop (opens in a new tab) features. For example, modifying the state of the graph to control the agent's behavior. This is common in use cases like customer support bots (opens in a new tab), programming assistants (opens in a new tab), and other applications. Before or after executing a given node, you can interrupt the graph's execution and "escalate" control to a user or support person. That person may respond immediately. Or they could respond a month from now. Either way, your workflow can resume at any time as if no time had passed at all.
   • We can also time travel to a previous state of the graph.
   • With streaming (opens in a new tab) we can emit signals for what an Agent is doing at any point in time. Needed for visibility and debugging.
   • Versioning allows us to roll back to a previous state of the graph.

Core Concepts of LangGraph

At its core, LangGraph models agent workflows as state machines. You define the behavior of your agents using three key components:

1. State: LangGraph revolves around the concept of a stateful graph, where each node in the graph represents a step in your computation, and the graph maintains a state that is passed around and updated as the computation progresses. It can be any Python type, but is typically a TypedDict or Pydantic BaseModel.
2. Nodes: Nodes are the building blocks of LangGraph. Each node represents a function or an agent. Tangibly, they are Python functions that encode the logic of your agents. They receive the current State as input, perform some computation or side-effect, and return an updated State. You define nodes to perform specific tasks, such as processing input, making decisions, or interacting with external APIs.
3. Edges: Edges connect the nodes in the graph. They control flow rules that determine which Node to execute next based on the current State. They can be fixed transitions or conditional branches / edges, allowing you to dynamically determine the next node to execute based on the current state of the graph.

By composing Nodes and Edges, you can create complex, looping workflows that evolve the State over time. The real power, though, comes from how LangGraph manages that State.

LangGraph's underlying graph algorithm uses message passing (opens in a new tab) to define a general program. When a Node completes, it sends a message along one or more edges to other node(s). These nodes run their functions, pass the resulting messages to the next set of nodes, and on and on it goes. Inspired by Pregel (opens in a new tab), the program proceeds in discrete "super-steps" that are all executed conceptually in parallel. Whenever the graph is run, all the nodes start in an inactive state. Whenever an incoming edge (or "channel") receives a new message (state), the node becomes active, runs the function, and responds with updates. At the end of each superstep, each node votes to halt by marking itself as inactive if it has no more incoming messages. The graph terminates when all nodes are inactive and when no messages are in transit.

Take this example:

Here we have:

• Agent Node: Represents the LLM (Large Language Model).
• Conditional Edge: Uses the result of the LLM call to determine the next step.
• Action Edge: Calls a function node and automatically loops back to the agent node.
• Entry Point: The starting point of the graph.
• End Node: Another action available after the agent node.
• Agent State: Tracks the state over time.
  • Accessible at all parts of the graph (each node and edge).
  • Local to the graph and can be stored in a persistence layer.
  • Allows resuming with the state at any point in time.

Prompts

There are a bunch of prompts available in the LangChain Prompt Hub (opens in a new tab) from the community. For example, here is one for the ReAct framework (opens in a new tab).

Resources

• LangGraph Conceptual Guides (opens in a new tab)

References