Memory

An agent keeps the context of a conversation through two mechanisms: the persistent chat history, which is on by default, and the memory compactor, which is optional.

Context window

On every turn, the agent sends the language model these messages, in this order:

1. The system prompt of the agent, see Prompts. It is passed separately from the chat history.
2. (optional) A summary of the earlier conversation, sent as a second system message. It is included only when the memory compactor is enabled and has produced a summary.
3. The most recent messages of the chat history, trimmed to the conversation history token limit, see Context Window Builder.
4. The new message from the person.

Chat History

PersistentChatHistory, a BaseChatMessageHistory, stores every message of the conversation in the database. It is available to agents and tools as chat_history on the injector.

• It stores all message types: human, AI, tool calls, tool results and intermediate steps.
• It interleaves parallel tool calls with their results, so the conversation is reconstructed correctly.
• messages_after(message_id) returns the messages added after a given one, which the compactor uses to summarize incrementally.

Context Window Builder

ContextWindowBuilder assembles the messages for each turn. It is part of the suvereo-agent-tool-calling plugin, and BaseToolCallingAgent uses it.

• It keeps the most recent messages that fit the token limit, starting on a human message.
• It counts tokens approximately.
• When the compactor has a summary, it adds it at the start as a SystemMessage.

The limit is the agent’s conversation_history_token_limit. By default it is 10,000 tokens, the value of the DEFAULT_CONVERSATION_HISTORY_TOKEN_LIMIT instance setting, which can be set through an environment variable of the same name. An agent can use a different limit by setting the CONVERSATION_HISTORY_TOKEN_LIMIT class variable, see Agent Configuration.

context_messages = ContextWindowBuilder(
    chat_history=self._injector.chat_history,
    memory_compactor=self._injector.memory_compactor,
).build(max_tokens=self._conversation_history_token_limit)

Memory Compactor

The memory compactor keeps a summary of the conversation, written by the language model and stored with the conversation. It is meant to be used together with the trimming above: the trimming keeps recent messages, and the summary carries what was said before them.

How it works

After each turn, the agent calls compact_if_needed(). Once 10 human messages have been added since the last summary, the compactor asks the language model for a new one. The summary is stored on the Conversation record and added to the context of each following turn.

Summaries are incremental. A new one is built from the previous summary and the messages added since, so the cost of a compaction does not grow with the length of the conversation.

Enabling the compactor

The compactor is off by default. An agent turns it on with the MEMORY_COMPACTOR_ENABLED class variable:

class ArticlesAgent(BaseToolCallingAgent):
    NAME = "Articles"
    AGENT_KEY = "articles-agent"
    SYSTEM_PROMPT = ARTICLES_SYSTEM_PROMPT
    MEMORY_COMPACTOR_ENABLED = True

The compactor class comes from memory.compactor_class in the configuration, which the default configuration (the Suvereo server’s DefaultAgentConfig) sets to LLMMemoryCompactor. If the compactor is enabled and no compactor class is set, building the agent fails.

Customization

Custom memory compactor

To change how summaries are made, subclass BaseMemoryCompactor from suvereo-common:

class MyMemoryCompactor(BaseMemoryCompactor):
    def get_summary(self) -> Optional[str]:
        """Return the persisted summary, or None if there is none yet."""
        ...
 
    def compact_if_needed(self) -> None:
        """Generate and persist a new summary when the threshold is reached."""
        ...

Then point the agent’s configuration at it by overriding get_config():

class ArticlesAgent(BaseToolCallingAgent):
    NAME = "Articles"
    AGENT_KEY = "articles-agent"
    SYSTEM_PROMPT = ARTICLES_SYSTEM_PROMPT
    MEMORY_COMPACTOR_ENABLED = True
 
    @classmethod
    def get_config(cls) -> AgentConfig:
        config = super().get_config()
        config.memory = MemoryConfig(
            chat_history_class=PersistentChatHistory,
            compactor_class=MyMemoryCompactor,
        )
        return config

Alternatively, the class can be set as the default compactor, by using it as compactor_class in the memory of the Suvereo server’s DefaultAgentConfig (server/agent/core/agents/default_config.py). It is then used by every agent whose configuration leaves memory empty.

The builder creates the compactor as compactor_class(conversation_repo, conversation_id, llm, chat_history).

Custom injector

The injector exposes chat_history and memory_compactor. The default Injector already carries both, so a domain definition’s injector that extends it can pass them along, see Injector.