Building a RAG Travel Bot — Part 2: Adding Conversational Memory

ByCoder Soul

This is Part 2 of a series where I build trip-pal, a RAG-powered travel assistant, from scratch. In Part 1 we built the foundation — loading a travel guide, chunking it, embedding it, and storing it in ChromaDB. By the end of Part 1 we had a working bot that could answer questions about Stockholm based on a real travel guide.

But it had a significant limitation. Every question was answered in isolation. Ask it about neighborhoods, then ask “which of those is best for families?” — and it had no idea what “those” referred to. No conversation. No memory. Just a very smart search bar.

In this post we fix that. We’ll add conversational memory so trip-pal remembers what was said earlier in the session and can handle natural follow-up questions.

What problem are we solving?

Let’s make the problem concrete. Open src/query.py from Part 1 and run it twice with these questions:

question = "What are the best neighborhoods to stay in Stockholm?"

Run it. Then change the question to:

question = "Which of those is best for families?"

Run it again. The bot either gives a confused answer or makes something up — because it genuinely has no idea what “those” refers to. Each call to chain.invoke() is completely stateless. There is no thread connecting one question to the next.

This is fine for a simple Q&A tool. But it’s not a conversation. And for a travel assistant, conversation is the whole point — you want to be able to say “tell me more about that neighborhood” or “what about something cheaper?” and have it understand.

How conversational memory works

The fix is surprisingly straightforward. Instead of sending just the current question to the LLM, we send the entire conversation history along with it. Every time a new question comes in, the model sees all the previous messages — what the human said, what it replied — before generating the next answer.

In LangChain this is handled through a MessagesPlaceholder in the prompt. It’s a slot that gets filled with the conversation history on every call. The prompt goes from this:

Context: {context}
Question: {question}

To this:

Context: {context}
[conversation history]
Question: {question}

The conversation history is just a Python list that grows as the dialogue continues. After each exchange we append the human’s message and the bot’s response. Next time a question comes in, the whole list goes into the prompt.

What we’re building

We’ll create a new file src/chat.py. The reason we create a new file rather than modify query.py is intentional — it keeps the evolution of the project visible. query.py stays as the Part 1 baseline. chat.py is the Part 2 version. Anyone reading the repo can see the progression.

By the end of this post you’ll be able to have conversations like:

You: What are the best neighborhoods to stay in Stockholm?
trip-pal: Based on the travel guide, Södermalm, Vasastan, and Kungsholmen are
          among the best neighborhoods...

You: Which of those is best for families?
trip-pal: Based on the context, Kungsholmen tends to be quieter and more
          residential, which makes it a good fit for families...

You: And what about budget travellers?
trip-pal: For budget travellers, Södermalm offers more affordable options
          with plenty of cafés and local spots...

Each follow-up question is understood in context. That’s what we’re building.

Step 1: Create src/chat.py with the base setup

Start by creating the new file with imports and the same setup from Part 1. Nothing new here yet — just making sure the foundation is solid before we add the memory layer.

from langchain_chroma import Chroma
from langchain_openai import OpenAIEmbeddings, ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder
from langchain_core.output_parsers import StrOutputParser
from langchain_core.runnables import RunnablePassthrough
from langchain_core.messages import HumanMessage, AIMessage
from dotenv import load_dotenv

load_dotenv()

embeddings = OpenAIEmbeddings()
vectorstore = Chroma(persist_directory="data/chroma", embedding_function=embeddings)
retriever = vectorstore.as_retriever()

llm = ChatOpenAI(model_name="gpt-3.5-turbo", temperature=0)

Two new imports worth noting: MessagesPlaceholder is the slot in the prompt that will hold the conversation history, and HumanMessage / AIMessage are LangChain’s message types — the standard way to represent conversation turns.

Step 2: Update the prompt

In Part 1 the prompt was a simple template string. Now we need a structured prompt with multiple message types. Replace the prompt with this:

prompt = ChatPromptTemplate.from_messages([
    ("system", """You are a helpful travel assistant.
Answer the question based only on the following context:
{context}

If you can't find the answer in the context, say so honestly."""),
    MessagesPlaceholder(variable_name="chat_history"),
    ("human", "{question}")
])

The structure here matters. The system message comes first — it sets the rules and provides the retrieved context. Then comes the conversation history, sandwiched in the middle. Then the current human question at the end. This ordering tells the model: here are the rules, here is what we’ve discussed, here is the new question.

The MessagesPlaceholder with variable_name="chat_history" is just a slot — it will be filled automatically when we pass the history into the chain.

Step 3: Add the memory list and the chain

Now the core of Part 2. Add a format_docs helper and the get_response function:

chat_history = []

def format_docs(docs):
    return "\n\n".join(doc.page_content for doc in docs)

def get_response(question):
    chain = (
        {
            "context": retriever | format_docs,
            "question": RunnablePassthrough(),
            "chat_history": lambda _: chat_history
        }
        | prompt
        | llm
        | StrOutputParser()
    )

    response = chain.invoke(question)

    chat_history.append(HumanMessage(content=question))
    chat_history.append(AIMessage(content=response))

    return response

A few things to explain here.

chat_history is a plain Python list. It starts empty and grows with every exchange. There’s no magic — it’s just a list of message objects that we maintain ourselves.

format_docs converts the list of document objects returned by the retriever into a single plain text string. This is important — the prompt expects a string for {context}, not a list of objects. Without this helper the chain fails silently and the bot answers as if it has no context at all. (More on this in the debugging section below.)

lambda _: chat_history passes the current history into the prompt on every call. The underscore is just a convention for an argument we don’t use — the lambda ignores the input and always returns the full history list.

After getting the response, we append both the human message and the AI response to the list. This is what makes memory work — the list grows, and next time the model sees the full conversation so far.

Step 4: Add the conversation loop

The final piece — a simple terminal loop that keeps the conversation going until you type quit:

print("trip-pal 🌍 (type 'quit' to exit)")
print("-" * 40)

while True:
    question = input("You: ").strip()

    if not question:
        continue

    if question.lower() == "quit":
        print("Bye!")
        break

    response = get_response(question)
    print(f"\ntrip-pal: {response}\n")

Nothing complex here. The loop reads input, skips empty lines, exits on quit, and otherwise calls get_response and prints the result.

Running it

python src/chat.py

Try this sequence to verify memory is working:

You: What are the best neighborhoods to stay in Stockholm?
You: Which of those is best for families?
You: And what about budget travellers?

The second and third questions have no meaning without the first. If the bot answers them correctly, memory is working.

What we simplified — and what production looks like

The memory implementation here is intentionally simple. It works well for learning the concept but there are a few things a production system would do differently.

The most obvious limitation is that our history is a Python list in RAM — it disappears the moment the process stops. A real system would persist conversation history to a database, tied to a session or user ID, so conversations can resume across restarts.

The second issue is cost and context limits. Every message we send includes all previous messages. A 20-turn conversation means the 20th API call carries all 19 previous turns. This gets expensive and eventually hits the model’s context window limit — GPT-3.5-turbo tops out at 16k tokens.

Production systems solve this in a few ways: a sliding window that only keeps the last N messages, summarization memory that compresses older turns into a short summary (“the user is travelling with kids and prefers quiet neighborhoods”), or vector memory that stores past exchanges as embeddings and retrieves only the relevant ones — essentially RAG applied to conversation history.

LangChain has built-in classes for most of these patterns. We’re keeping things simple for now because the goal is to understand the fundamentals first. But it’s worth knowing the tradeoffs you’d face if you took this to production.

Summary

The change from Part 1 to Part 2 is small in code but meaningful in behaviour. We went from a stateless Q&A tool to something that actually feels like a conversation. The key pieces:

  • A MessagesPlaceholder in the prompt to hold conversation history
  • A plain Python list that grows with every exchange
  • A format_docs helper to convert retrieved documents into a usable string
  • Appending both sides of each exchange to the history after every response

The memory only lasts for the current session — close the terminal and it’s gone. That’s a deliberate simplification for now. Persistent memory across sessions is a different problem, and a more complex one. We’ll get there eventually.

What’s next?

trip-pal can now hold a conversation. But it’s still working only from a static document. It has no idea what the weather in Stockholm is right now. It can’t tell you whether a restaurant is currently open. It’s grounded in real text, but that text doesn’t change.

In Part 3 we’ll take the first step toward a real agent — connecting trip-pal to a live weather API so it can answer questions like “should I pack a rain jacket for my trip next week?” with actual current data, not just what the travel guide says about Stockholm’s climate in general.

The full code is on GitHub: https://github.com/C0derSoul/trip-pal

Similar Posts