Single-turn Q&A is relatively easy: the user asks a question, the AI answers. Multi-turn conversation is harder: the user asks a follow-up that references the previous answer, and the AI must maintain context across turns. This is what separates a useful BI assistant from a glorified search engine.
The Context Window Problem
Large language models have a context window — the amount of text they can process at once. In a multi-turn conversation, each previous turn consumes part of this window. After 5-10 turns, the model starts 'forgetting' earlier context. The solution: maintain a structured conversation state outside the model and inject only the relevant context for each new turn.
Reference Resolution
When a user says 'now show that by month', the AI must resolve 'that' to the previous query's subject (e.g., 'revenue by region'). This requires tracking the conversation's entity graph: what metrics, dimensions, and filters have been discussed, and which are active in the current turn.
Context Window Management Strategy
Effective context management: (1) Store each turn's query, result, and metadata in a conversation state object. (2) For each new turn, generate a compact context summary — not the full history. (3) Include the summary, the current question, and any specific entities referenced. This keeps the context window focused and relevant.
When to Reset Context
Not all turns belong to the same analytical thread. If a user switches from 'revenue analysis' to 'headcount trends', the AI should detect the topic shift and reset the analytical context — while keeping the conversation history available if the user returns to the previous topic.
核心要點
- The Context Window Problem
- Reference Resolution
- Context Window Management Strategy
- When to Reset Context
總結
Not all turns belong to the same analytical thread. If a user switches from 'revenue analysis' to 'headcount trends', the AI should detect the topic shift and reset the analytical context — while keep...
At Beehive Strategy, we help enterprises build the data foundations, semantic layers, and AI agent ecosystems that turn data into decisions. Our MCP-powered platform connects to 50+ data sources, deploys in 2 weeks, and delivers insights directly inside the IM tools your teams already use. Book a free demo to see how we can help your organisation.