Skip to content

Design Principles

Overview

As we build our agent framework step by step, we'll make key architectural decisions that determine how easy your agents are to develop, debug, and scale. These aren't abstract principles but practical choices that address real problems you'll encounter when building agent applications.

Core Architectural Principles

1. Async-First Architecture

Why Async?

Agent tasks involve multiple slow operations: LLM API calls (500ms-5s), tool executions (variable), and I/O operations. Without async, your agent sits idle during each call, making multi-agent systems painfully slow.

Performance Impact:

  • Synchronous approach: A simple 3-agent workflow could take 30 seconds
  • Async approach: The same workflow takes only 10 seconds with proper concurrency

Implementation Strategy:

We embrace async/await throughout because retrofitting async into synchronous code is much harder than the reverse.

2. Event-Based Streaming

Real-Time Progress Updates

Agent tasks can take 30+ seconds to complete multiple steps. Without streaming, users stare at blank screens wondering if anything is happening.

Benefits:

  • Real-time progress updates: Users see what's happening as it occurs
  • Responsive user interfaces: Better user experience with live feedback
  • Enhanced observability: Supplies the data you need to debug multi-step agent behavior
  • Debugging support: When an agent gets stuck in a tool call loop, streaming events show you exactly where the problem occurs

3. Component Serialization

Configuration as Code

When every component (agents, tools, memory) can serialize itself to JSON, you gain powerful capabilities for configuration management.

Use Cases:

  • Save agent configurations for reuse
  • Share configurations between team members
  • Build visual editors for non-technical users
  • Version control complete agent setups
  • Build configuration UIs
  • Restore agent configuration and state from saved sessions

Implementation:

All components implement serialization to JSON format, enabling programmatic and visual configuration tools.

4. Graceful Cancellation

User Control is Critical

Users will start long-running agent tasks and then need to cancel them - whether due to incorrect prompts, infinite loops, or to provide feedback.

Requirements:

  • Support for cancellation at any point in execution
  • Clean resource cleanup when tasks are cancelled
  • Proper state management during cancellation
  • User feedback on cancellation status

5. Abstract Base Classes with Core Behaviors

Flexibility and Extensibility

When you want to support multiple LLM providers, different tool types, or various memory backends, abstract interfaces prevent vendor lock-in and enable testing with mock implementations.

Benefits:

  • Provider flexibility: Support multiple LLM providers without code changes
  • Tool extensibility: Start with simple functions, later add REST API tools, database tools, or emerging standards like MCP tools
  • Testing support: Use mock implementations for unit testing
  • Future-proofing: Easily adopt new technologies without architectural changes

Example:

The BaseTool interface lets you start with simple functions and later add REST API tools, database tools, or emerging standards like MCP tools without changing your agent code.

References