Docs/Core concepts

Context & compaction

How Gray resolves an AI agent's context window, and how auto-compact summarizes older turns before you hit the limit and lose the thread.

Source: crates/gray/src/compact

The context window resolves in order: the --context-window flag or GRAY_CONTEXT_WINDOW, then the value auto-fetched from the provider, then the LiteLLM model table, then a hardcoded per-model fallback.

bash
$/context # visual breakdown of what fills the window
$/context 128k # set — 128000, 128k and 1m all parse
$/context auto # clear the override
$/context reserve 16k # headroom kept free before auto-compact fires
$/context keep 20k # recent tail kept beside the summary (off = summary only)

Auto-compact

When usage crosses the window minus the reserve (16k by default), Gray compacts before the next turn: older history becomes a two-message summary and the recent tail is kept. On a context_length or max_tokens overflow error it compacts and retries once. Auto is the default; no flag needed.

bash
$/compact # compact now (alias /compress)
$/compact keep the API design decisions # steer what the summary keeps
FlagEnvMeaning
--context-windowGRAY_CONTEXT_WINDOWwindow size in tokens, or auto
--context-reserveGRAY_CONTEXT_RESERVEtokens kept free before compaction
--context-keepGRAY_CONTEXT_KEEPtail kept alongside the summary

Join the Discord

Chat with the people building and running Gray. Show what you made with it.

Join Discord →

Need help?

Open an issue on GitHub, run /feedback from the REPL, or ask in Discord.

Open an issue →