EDDI Architecture

Version: ≥5.6.x

This document provides a comprehensive overview of EDDI's architecture, design principles, and internal workflow.

Overview
What EDDI Is (and Isn't)
Core Architecture
The Lifecycle Pipeline
Conversation Flow
Bot Composition Model
Key Components
Technology Stack

Overview

E.D.D.I. (Enhanced Dialog Driven Interface) is a multi-agent orchestration middleware for conversational AI systems, not a standalone chatbot or language model. It sits between user-facing applications and multiple AI agents (LLMs like OpenAI, Claude, Gemini, or traditional REST APIs), intelligently routing requests, coordinating responses, and maintaining conversation state across agent interactions.

Core Purpose: Orchestrate multiple AI agents and business systems in complex conversational workflows without writing code.

What EDDI Is (and Isn't)

A Multi-Agent Orchestration Middleware: Coordinates multiple AI agents (LLMs, APIs) in complex workflows
An Intelligent Router: Directs requests to appropriate agents based on patterns, rules, and context
A Conversation Coordinator: Maintains stateful conversations across multiple agent interactions
A Configuration Engine: Agent orchestration defined through JSON configurations, not code
A Middleware Service: Acts as an intermediary that adds intelligence and control to conversation flows
Business System Integrator: Connects AI agents with your existing APIs, databases, and services
Cloud-Native: Built with Quarkus for fast startup, low memory footprint, and containerized deployment
Stateful: Maintains complete conversation history and context throughout interactions

EDDI Is Not:

Not a standalone LLM: It doesn't train or run machine learning models
Not a chatbot platform: It's the infrastructure that powers chatbots
Not just a proxy: It provides orchestration, state management, and complex behavior rules beyond simple API forwarding

Core Architecture

Architectural Principles

EDDI's architecture is built on several key principles:

Modularity: Every component is pluggable and replaceable
Composability: Bots are assembled from reusable packages and extensions
Asynchronous Processing: Non-blocking I/O for handling concurrent conversations
State-Driven: All operations transform or query the conversation state
Cloud-Native: Designed for containerized, distributed deployments

High-Level Architecture Diagram

┌─────────────────────────────────────────────────────────────┐
│                      User Application                        │
│                  (Web, Mobile, Chat Client)                  │
└────────────────────────────┬────────────────────────────────┘
                             │ HTTP/REST API
                             ▼
┌─────────────────────────────────────────────────────────────┐
│                      RestBotEngine                           │
│          (Entry Point - JAX-RS AsyncResponse)                │
└────────────────────────────┬────────────────────────────────┘
                             │
                             ▼
┌─────────────────────────────────────────────────────────────┐
│                  ConversationCoordinator                     │
│           (Ensures Sequential Processing per                 │
│            Conversation, Concurrent Across)                  │
└────────────────────────────┬────────────────────────────────┘
                             │
                             ▼
┌─────────────────────────────────────────────────────────────┐
│                   IConversationMemory                        │
│       (Stateful Object - Complete Conversation Context)      │
└────────────────────────────┬────────────────────────────────┘
                             │
                             ▼
┌─────────────────────────────────────────────────────────────┐
│                     LifecycleManager                         │
│          (Executes Sequential Pipeline of Tasks)             │
└────────────────────────────┬────────────────────────────────┘
                             │
        ┌────────────────────┼────────────────────┐
        ▼                    ▼                    ▼
┌──────────────┐   ┌──────────────┐    ┌──────────────┐
│Input Parsing │   │Behavior Rules│    │LLM/API Calls │
│  (NLP, etc)  │   │(IF-THEN Logic│    │(LangChain4j, │
│              │   │              │    │ HTTP Calls)  │
└──────────────┘   └──────────────┘    └──────────────┘
        │                    │                    │
        └────────────────────┼────────────────────┘
                             ▼
                   ┌──────────────────┐
                   │Output Generation │
                   │  (Templating)    │
                   └──────────────────┘
                             │
                             ▼
┌─────────────────────────────────────────────────────────────┐
│                    MongoDB + Cache                           │
│         (Persistent Storage + Fast Retrieval)                │
└─────────────────────────────────────────────────────────────┘

The Lifecycle Pipeline

The Lifecycle is EDDI's most distinctive architectural feature. Instead of hard-coded bot logic, EDDI processes every user interaction through a configurable, sequential pipeline of tasks called the Lifecycle.

How the Lifecycle Works

Pipeline Composition: Each bot defines a sequence of ILifecycleTask components
Sequential Execution: Tasks execute one after another, each transforming the IConversationMemory
Stateless Tasks: Each task is stateless; all state resides in the memory object passed through
Interruptible: The pipeline can be stopped early based on conditions (e.g., STOP_CONVERSATION)

Standard Lifecycle Tasks

A typical bot lifecycle includes these task types:

Task Type	Purpose	Example
Input Parsing	Normalizes and understands user input	Extracting entities, intents from text
Semantic Parsing	Uses dictionaries to parse expressions	Matching "hello" → `greeting(hello)`
Behavior Rules	Evaluates IF-THEN rules to decide actions	"If `greeting(*)` then `action(welcome)`"
Property Extraction	Extracts and stores data in conversation memory	Saving user name, preferences
HTTP Calls	Calls external REST APIs	Weather API, CRM systems
LangChain Task	Invokes LLM APIs (OpenAI, Claude, etc.)	Conversational AI responses
Output Generation	Formats final response using templates	Thymeleaf templating with conversation data

Lifecycle Task Interface

public interface ILifecycleTask {
    String getId();
    String getType();
    void execute(IConversationMemory memory, Object component) 
        throws LifecycleException;
}

Every task receives:

IConversationMemory: Complete conversation state
component: Task-specific configuration/resources

Conversation Flow

Step-by-Step: User Interaction Flow

Here's what happens when a user sends a message to an EDDI bot:

1. API Request

POST /bots/{environment}/{botId}/{conversationId}
Body: { "input": "Hello, what's the weather?", "context": {...} }

2. RestBotEngine Receives Request

Validates bot ID and environment
Wraps response in AsyncResponse for non-blocking processing
Increments metrics counters

3. ConversationCoordinator Queues Message

Ensures messages for the same conversation are processed sequentially
Allows different conversations to process concurrently
Prevents race conditions in conversation state

4. IConversationMemory Loaded/Created

If existing conversation: Loads from MongoDB
If new conversation: Creates fresh memory object
Includes all previous steps, user data, context

5. LifecycleManager Executes Pipeline

Input → Parser → Behavior Rules → API/LLM → Output → Save

Each task in sequence:

Reads current conversation state
Performs its operation (parsing, rule evaluation, API call, etc.)
Writes results back to conversation memory
Passes control to next task

6. State Persistence

Updated IConversationMemory saved to MongoDB
Cache updated with latest conversation state
Metrics recorded (duration, success/failure)

7. Response Returned

{
  "conversationState": "READY",
  "conversationOutputs": [
    {
      "output": ["The weather today is sunny with a high of 75°F"],
      "actions": ["weather_response"]
    }
  ]
}

Bot Composition Model

EDDI bots are not monolithic. They are composite objects assembled from version-controlled, reusable components.

Hierarchy: Bot → Package → Extensions

Bot (.bot.json)
  ├─ Package 1 (.package.json)
  │   ├─ Behavior Rules Extension (.behavior.json)
  │   ├─ HTTP Calls Extension (.httpcalls.json)
  │   └─ Output Extension (.output.json)
  ├─ Package 2 (.package.json)
  │   ├─ Dictionary Extension (.dictionary.json)
  │   └─ LangChain Extension (.langchain.json)
  └─ Package 3 (.package.json)
      └─ Property Extension (.property.json)

1. Bot Level

File: {botId}.bot.json

A bot is simply a list of package references:

{
  "packages": [
    "eddi://ai.labs.package/packagestore/packages/{packageId}?version={version}",
    "eddi://ai.labs.package/packagestore/packages/{anotherPackageId}?version={version}"
  ]
}

2. Package Level

File: {packageId}.package.json

A package is a container of functionality with a list of extensions:

{
  "packageExtensions": [
    {
      "type": "eddi://ai.labs.behavior",
      "extensions": {
        "uri": "eddi://ai.labs.behavior/behaviorstore/behaviorsets/{behaviorId}?version={version}"
      },
      "config": {
        "appendActions": true
      }
    },
    {
      "type": "eddi://ai.labs.httpcalls",
      "extensions": {
        "uri": "eddi://ai.labs.httpcalls/httpcallsstore/httpcalls/{httpCallsId}?version={version}"
      }
    }
  ]
}

3. Extension Level

Files: {extensionId}.{type}.json

Extensions are the actual bot logic:

Behavior Rules Extension

{
  "behaviorGroups": [
    {
      "name": "Greetings",
      "behaviorRules": [
        {
          "name": "Welcome User",
          "conditions": [
            {
              "type": "inputmatcher",
              "configs": {
                "expressions": "greeting(*)",
                "occurrence": "currentStep"
              }
            }
          ],
          "actions": ["welcome_action"]
        }
      ]
    }
  ]
}

HTTP Calls Extension

{
  "targetServerUrl": "https://api.weather.com",
  "httpCalls": [
    {
      "name": "getWeather",
      "actions": ["fetch_weather"],
      "request": {
        "method": "GET",
        "path": "/current?location=${context.userLocation}"
      },
      "postResponse": {
        "propertyInstructions": [
          {
            "name": "currentWeather",
            "fromObjectPath": "weatherResponse.temperature",
            "scope": "conversation"
          }
        ]
      }
    }
  ]
}

LangChain Extension

{
  "tasks": [
    {
      "actions": ["send_to_ai"],
      "id": "openaiChat",
      "type": "openai",
      "parameters": {
        "apiKey": "...",
        "modelName": "gpt-4o",
        "systemMessage": "You are a helpful assistant",
        "sendConversation": "true",
        "addToOutput": "true"
      }
    }
  ]
}

Key Components

RestBotEngine

Location: ai.labs.eddi.engine.internal.RestBotEngine

Purpose: Main entry point for all bot interactions

Responsibilities:

Receives HTTP requests via JAX-RS
Validates bot and conversation IDs
Handles async responses
Records metrics
Coordinates with IConversationCoordinator

ConversationCoordinator

Location: ai.labs.eddi.engine.runtime.internal.ConversationCoordinator

Purpose: Ensures proper message ordering and concurrency control

Key Feature: Uses a queue system to guarantee that:

Messages within the same conversation are processed sequentially
Different conversations can be processed in parallel
No race conditions occur in conversation state updates

IConversationMemory

Location: ai.labs.eddi.engine.memory.IConversationMemory

Purpose: The stateful object representing a complete conversation

Contains:

Conversation ID, bot ID, user ID
All previous conversation steps (history)
Current step being processed
User properties (name, preferences, etc.)
Context data (passed with each request)
Actions and outputs generated

Key Methods:

String getConversationId();
IWritableConversationStep getCurrentStep();
IConversationStepStack getPreviousSteps();
ConversationState getConversationState();
void undoLastStep();
void redoLastStep();

LifecycleManager

Location: ai.labs.eddi.engine.lifecycle.internal.LifecycleManager

Purpose: Executes the lifecycle pipeline

Key Method:

void executeLifecycle(
    IConversationMemory conversationMemory, 
    List<String> lifecycleTaskTypes
) throws LifecycleException

How It Works:

Iterates through registered ILifecycleTask instances
For each task, calls task.execute(conversationMemory, component)
Checks for interruption or stop conditions
Continues until all tasks complete or stop condition is met

PackageConfiguration

Location: ai.labs.eddi.configs.packages.model.PackageConfiguration

Purpose: Defines the structure of a bot package

Model:

public class PackageConfiguration {
    private List<PackageExtension> packageExtensions;
    
    public static class PackageExtension {
        private URI type;
        private Map<String, Object> extensions;
        private Map<String, Object> config;
    }
}

ToolExecutionService

Location: ai.labs.eddi.modules.langchain.tools.ToolExecutionService

Purpose: Unified execution pipeline for all AI agent tool invocations

Pipeline:

Tool Call ──▶ Rate Limiter ──▶ Cache Check ──▶ Execute ──▶ Cost Tracker ──▶ Result

Features:

Token-bucket rate limiting per tool (configurable per-tool or global default)
Smart caching — deduplicates calls with identical arguments
Cost tracking with per-conversation budgets and automatic eviction
Security: tools that accept URLs are validated against private/internal addresses (SSRF protection via UrlValidationUtils)
Security: math expressions are evaluated in a sandboxed parser (SafeMathParser)

See the Security documentation for details.

Technology Stack

Core Framework

Quarkus: Supersonic, subatomic Java framework
- Fast startup times (~0.05s)
- Low memory footprint
- Native compilation support
- Built-in observability (metrics, health checks)

Language & Runtime

Java 21: Latest LTS with modern language features
GraalVM: Optional native compilation for even faster startup

Dependency Injection

CDI (Contexts and Dependency Injection): Jakarta EE standard
@ApplicationScoped, @Inject: Clean, testable component wiring

REST Framework

JAX-RS: Jakarta REST API standard
AsyncResponse: Non-blocking, scalable request handling
JSON-B: JSON binding for serialization/deserialization

Database

MongoDB 6.0+: Document store for bot configurations and conversation logs
- Stores bot, package, and extension configurations
- Persists conversation history
- Enables version control of bot components

Caching

Infinispan: Distributed in-memory cache
- Caches conversation state for fast retrieval
- Reduces database load
- Enables horizontal scaling

LLM Integration

LangChain4j: Java library for LLM orchestration
- Unified interface to multiple LLM providers
- Supports OpenAI, Claude, Gemini, Ollama, Hugging Face, etc.
- Handles chat message formatting, streaming, tool calling

Observability

Micrometer: Metrics collection
Prometheus: Metrics exposition
Kubernetes Probes: Liveness and readiness endpoints

Security

OAuth 2.0: Authentication and authorization
Keycloak: Identity and access management

Templating

Thymeleaf: Output templating engine
- Dynamic output generation
- Access to conversation memory in templates
- Expression language support

Design Patterns Used

1. Strategy Pattern

Where: Lifecycle tasks
Why: Different behaviors (parsing, rules, API calls) implement the same ILifecycleTask interface

2. Chain of Responsibility

Where: Lifecycle pipeline
Why: Each task processes the memory object and passes it to the next task

3. Composite Pattern

Where: Bot composition (Bot → Packages → Extensions)
Why: Bots are built from hierarchical, reusable components

4. Repository Pattern

Where: Data access (stores: botstore, packagestore, etc.)
Why: Abstracts data persistence from business logic

5. Factory Pattern

Where: IBotFactory
Why: Complex bot instantiation from multiple packages and configurations

6. Coordinator Pattern

Where: ConversationCoordinator
Why: Manages concurrent access to shared conversation state

Performance Characteristics

Startup Time

JVM mode: < 2 seconds
Native mode: < 50ms (with GraalVM)

Memory Footprint

JVM mode: ~200MB baseline
Native mode: ~50MB baseline

Request Latency

Without LLM: 10-50ms (parsing, rules, simple API calls)
With LLM: 500-5000ms (depends on LLM provider)

Scalability

Vertical: Handles thousands of concurrent conversations per instance
Horizontal: Stateless design allows infinite horizontal scaling
Bottleneck: MongoDB becomes bottleneck; use replica sets and sharding

Cloud-Native Features

Containerization

Official Docker images: labsai/eddi
Certified by IBM/Red Hat
Multi-stage builds for minimal image size

Orchestration

Kubernetes-ready
OpenShift certified
Health checks built-in

Configuration

Externalized configuration via environment variables
ConfigMaps and Secrets support
No rebuild needed for configuration changes

Observability

Prometheus metrics endpoint: /q/metrics
Health checks: /q/health/live, /q/health/ready
Structured logging with correlation IDs

Case Study: The "Bot Father"

The Bot Father is a meta-bot that demonstrates EDDI's architecture in action. It's a bot that creates other bots.

For a comprehensive, step-by-step walkthrough of Bot Father, see Bot Father: A Deep Dive

How It Works

Conversation Start: User starts chat with Bot Father
Information Gathering: Bot Father asks questions:
- "What do you want to call your bot?"
- "What should it do?"
- "Which LLM API should it use?"
Memory Storage: Property setters save answers to conversation memory:
- context.botName
- context.botDescription
- context.llmType

Condition Triggers: Behavior rule monitors memory:

{
  "conditions": [
    {
      "type": "contextmatcher",
      "configs": {
        "contextKey": "botName",
        "contextType": "string"
      }
    }
  ],
  "actions": ["httpcall(create-bot)"]
}

API Call Execution: HTTP Calls extension triggers:

{
  "name": "create-bot",
  "request": {
    "method": "POST",
    "path": "/botstore/bots",
    "body": "{\"botName\": \"${context.botName}\"}"
  }
}

Self-Modification: Bot Father calls EDDI's own API to create a new bot configuration

Key Insight

Bot Father isn't special code—it's a regular EDDI bot that uses:

Behavior rules to control conversation flow
Property extraction to gather data
HTTP Calls to invoke EDDI's REST API
Output templates to guide the user

This demonstrates EDDI's power: the same architecture that powers chatbots can orchestrate complex, multi-step workflows, even self-modifying the system itself.

See the Bot Father Deep Dive for complete implementation details, code examples, and real-world applications.

Summary

EDDI's architecture is built on principles of modularity, composability, and orchestration. It's not a chatbot—it's the infrastructure for building sophisticated conversational AI systems that can:

Orchestrate multiple APIs and LLMs
Apply complex business logic through configurable rules
Maintain stateful, context-aware conversations
Scale horizontally in cloud environments
Be assembled from reusable, version-controlled components

The Lifecycle Pipeline is the heart of this architecture, providing a flexible, pluggable system where bot behavior is configuration, not code.

FilesExpand file tree

architecture.md

Latest commit

History

architecture.md

File metadata and controls

EDDI Architecture

Table of Contents

Overview

What EDDI Is (and Isn't)

EDDI Is Not:

Core Architecture

Architectural Principles

High-Level Architecture Diagram

The Lifecycle Pipeline

How the Lifecycle Works

Standard Lifecycle Tasks

Lifecycle Task Interface

Conversation Flow

Step-by-Step: User Interaction Flow

1. API Request

2. RestBotEngine Receives Request

3. ConversationCoordinator Queues Message

4. IConversationMemory Loaded/Created

5. LifecycleManager Executes Pipeline

6. State Persistence

7. Response Returned

Bot Composition Model

Hierarchy: Bot → Package → Extensions

1. Bot Level

2. Package Level

3. Extension Level

Behavior Rules Extension

HTTP Calls Extension

LangChain Extension

Key Components

RestBotEngine

ConversationCoordinator

IConversationMemory

LifecycleManager

PackageConfiguration

ToolExecutionService

Technology Stack

Core Framework

Language & Runtime

Dependency Injection

REST Framework

Database

Caching

LLM Integration

Observability

Security

Templating

Design Patterns Used

1. Strategy Pattern

2. Chain of Responsibility

3. Composite Pattern

4. Repository Pattern

5. Factory Pattern

6. Coordinator Pattern

Performance Characteristics

Startup Time

Memory Footprint

Request Latency

Scalability

Cloud-Native Features

Containerization

Orchestration

Configuration

Observability

Case Study: The "Bot Father"

How It Works

Key Insight

Summary

Related Documentation