Skip to content

Context Compaction

Long conversations eventually approach the model’s context window. Instead of failing or silently dropping history, ChatShell compacts older messages automatically so the conversation can keep going.

After a turn completes, ChatShell estimates the context size of the conversation. When it gets close to the model’s window limit:

  1. A recent window (about 20,000 tokens) is kept verbatim — the AI always sees your latest exchange in full.
  2. The older prefix is compacted: either into a bitmap archive (vision-capable models) or a text summary (everything else).
  3. Cut points only fall on user-message boundaries, so tool-call chains are never split mid-flight.

Compaction is method-aware:

ModelMethodResult
Vision-capableBitmap archive — the discarded transcript is rendered into compact text images the model reads visuallyNear-verbatim recall of compacted history at a fraction of the token cost
Text-onlyLLM-generated summary of the discarded prefixFacts carried forward in condensed form

The bitmap technique is adapted from the open-source oh-my-pi project (MIT), with CJK text fully supported.

  • Your history is never trimmed in the UI. The transcript stays complete on this device and in sync; only the model’s working context shrinks.
  • No configuration required. Compaction triggers automatically, per conversation, based on each model’s actual context window.
  • Summaries use the tiny model when one is configured (see Auxiliary Models), keeping compaction cheap instead of billing your main model for it.

The threshold adapts to the model: compaction starts when the conversation’s context reaches the model’s window minus a safety reserve (15% of the window, with a 16k-token floor). If the model’s window size is unknown, a 128k window is assumed.