We need memory to store the previous conversational history.
For example:
Previous question is:
User: File handling in Python
Assistant: Explain about file handling.
Next time, the user asks:
User: What are the modes in it?
The LLM should understand the context and then respond.
Below are the details that can be stored in the memory.
- Past Conversations
- User Preferences that we specify in the system prompts, e.g., JSON format
- Past Decisions
- Previous Tasks
Based on the details we are storing, we will choose between long-term or short-term memory.
Short-Term Memory
Redis, Valkey, Memcached, and Cosmos, which are cached databases, can be used for short-term memory to store the last few conversations or a summary.
We can set a general data invalidation rule to erase the content or use an LRU cache eviction policy, where the least recently used data will be erased from the database.
Long-Term Memory
Postgres, Pinecone, and MongoDB can be used for long-term memory to store long conversation histories.
How It Is Functioning
A summary of the entire conversation history will be stored in short-term memory to reduce latency whenever needed.
Episodic Memory
Episodic memory is a type of memory that stores specific events or experiences that happened in the past, usually together with information about what happened, when it happened, and the context surrounding it.
We can use either a short-term or long-term memory database depending on the use case. It is a kind of combination of short-term and long-term memory.
User:
I am planning a trip to Paris.
Agent:
How many days will you stay?
User:
5 days.
Later,
User:
Can you suggest an itinerary?
Episode 1
User wants to travel to Paris.
Trip duration: 5 days.
User previously mentioned:
Destination = Paris
Duration = 5 days
This information can be used to provide a more relevant response.
This helps the LLM understand what happened previously.
Semantic Memory
Semantic memory contains facts extracted from previous conversational history. Semantic memory is generally considered long-term memory.
Example
Paris is the capital of France.
The Louvre is a museum in Paris.
France uses the Euro.
That’s general knowledge.
Sliding Window Memory
It is a short-term memory. Here, we store the last 3 to 4 conversations.
Redis or Valkey, like any cache memory, can be used.
Summarized Memory
Each and every time, the conversation, which includes the user query and response, will be summarized.
Even though token consumption during summarization is more, overall token consumption will be less.
It is a long-term memory.
Entity Fact Memory
This memory is used to store facts about a particular entity.
The difference between entity fact memory and semantic memory is that semantic memory is the broader category. Entity fact memory is one way of organizing and storing semantic knowledge about specific entities.
It can be used as either long-term or short-term memory.
Semantic Memory
Entity: Python
Python → is a programming language
Python → is used for AI
Python → supports object-oriented programming
Python → was created by Guido van Rossum
Entity Fact Memory
Entity: Alice
Alice → works at ABC Company
Alice → prefers Python
Alice → is working on Project X
It is not a good practice to store the entire conversation. We can make decisions based on the conversation and then store the relevant information. This is a good practice.
