Conversation compression #16

New issue

Open

opened 2026-04-07 20:23:34 +00:00 by austin · 0 comments

austin commented

2026-04-07 20:23:34 +00:00

Owner

Implement lossy conversation history management.

Maintain a structured summary of the session: what the user asked, what was changed, what was accepted/rejected
The summary is a harness-maintained data structure, not a model-generated summary (don't spend inference on this)
Format the summary into a short context block that fits within a fixed token budget (~256 tokens)
Update the summary after each turn
Include enough detail that the model understands the trajectory of the session without seeing raw history

Raw conversation history will blow the context budget within a few turns. The harness needs to compress aggressively while preserving the information the model needs for coherent multi-turn sessions.

Implement lossy conversation history management. - Maintain a structured summary of the session: what the user asked, what was changed, what was accepted/rejected - The summary is a harness-maintained data structure, not a model-generated summary (don't spend inference on this) - Format the summary into a short context block that fits within a fixed token budget (~256 tokens) - Update the summary after each turn - Include enough detail that the model understands the trajectory of the session without seeing raw history Raw conversation history will blow the context budget within a few turns. The harness needs to compress aggressively while preserving the information the model needs for coherent multi-turn sessions.