{"schemaVersion":1,"id":"https://lucioduran.com/projects/agent-context-budget","url":"https://lucioduran.com/projects/agent-context-budget","title":"Agent Context Budget","description":"Compare full-window input, truncation, selection and extractive compression while measuring evidence loss separately from answer quality.","language":"en","datePublished":"2026-11-09","dateModified":"2026-11-09","contentFormat":"text/markdown","content":"[Projects](<https://lucioduran.com/projects>) / Agent Context Budget\n\n# Agent Context Budget\n\nCompare full-window input, truncation, selection and extractive compression while measuring evidence loss separately from answer quality.\n\nAuthor: [Lucio Durán](<https://lucioduran.com/>) · Python 3.11.13 · License: [MIT](<https://github.com/lucioduran/agent-context-budget/blob/72d150669a9fa4ab85496b4b62440ce95a9be7fe/LICENSE>)\n\nPage published: 2026-11-09 · Page updated: 2026-11-09\n\n[Repository](<https://github.com/lucioduran/agent-context-budget>) · [Original evidence](<https://raw.githubusercontent.com/lucioduran/agent-context-budget/72d150669a9fa4ab85496b4b62440ce95a9be7fe/results/evaluation.json>) · [Release v0.1.1](<https://github.com/lucioduran/agent-context-budget/releases/tag/v0.1.1>)\n\n## Problem and relevance to agents\n\nA compressed record can keep a citation identifier but lose the sentence that supports its answer. Token savings alone cannot establish useful context.\n\nFrozen tasks with late and contradictory facts expose which policies lose decisive evidence and whether a small local model uses what remains.\n\n## Architecture\n\n1. **Freeze the experiment.** Twelve original fictional tasks and the evaluation protocol were committed before inference.\n2. **Apply exact token budgets.** A pinned tokenizer accounts for prompt overhead across chronological input, prefix truncation, record selection and sentence extraction.\n3. **Preserve raw decisions.** Prompts, fragments, source IDs, responses, token traces and resource observations support deterministic replay.\n\n- [Design and trust boundaries](<https://github.com/lucioduran/agent-context-budget/blob/72d150669a9fa4ab85496b4b62440ce95a9be7fe/docs/design.md>)\n- [Frozen evaluation protocol](<https://github.com/lucioduran/agent-context-budget/blob/72d150669a9fa4ab85496b4b62440ce95a9be7fe/docs/protocol.md>)\n\n## What this demonstrates\n\n- Separate scoring for retained evidence, retained citation identifiers, generated citations and exact answers.\n- Explicit recency and first-sentence ablations.\n- Actual pinned FLAN-T5-small CPU inference with recorded failures.\n\n## Published results\n\nOne local Xeon CPU run made 132 model calls across eleven settings using two PyTorch CPU threads.\n\nEach setting evaluates the same twelve tasks. Original prompts contain 142 to 418 tokens; none exceeds the model's 512-token input limit. Selected evidence and generated answers are scored independently.\n\n### Context policy evaluation\n\n| Policy and token budget | Exact answers | Evidence retained | Citation IDs retained | Mean input tokens |\n| --- | --- | --- | --- | --- |\n| full:512 | 10/12 | 1.000 | 1.000 | 280.4 |\n| head:128 | 0/12 | 0.000 | 0.000 | 128.0 |\n| head:256 | 5/12 | 0.500 | 0.500 | 205.2 |\n| select:128 | 6/12 | 1.000 | 1.000 | 111.4 |\n| select:256 | 3/12 | 1.000 | 1.000 | 195.9 |\n| compress:128 | 1/12 | 1.000 | 1.000 | 106.4 |\n| compress:256 | 2/12 | 1.000 | 1.000 | 141.4 |\n| select\\_no\\_recency:128 | 6/12 | 1.000 | 1.000 | 112.4 |\n| select\\_no\\_recency:256 | 4/12 | 1.000 | 1.000 | 196.4 |\n| compress\\_first:128 | 2/12 | 0.667 | 1.000 | 111.7 |\n| compress\\_first:256 | 3/12 | 0.667 | 1.000 | 157.2 |\n\nEvery setting produced zero correct output citations and zero grounded exact matches. First-sentence compression lost four corrections while retaining their citation IDs. Retaining evidence did not guarantee correct model use.\n\n## Reproduce\n\nPortable replay requires Python 3.11 or newer and no runtime model packages. Inference uses the pinned Intel macOS Python 3.11.13 environment with PyTorch 2.2.2 and Transformers 4.46.3.\n\nRun measurements in a new destination and serialize heavy workloads. Within the original project series, use the shared benchmark\\_lock.py wrapper. Commands below use POSIX shell syntax.\n\n### Verify saved evidence\n\n```\ngit clone https://github.com/lucioduran/agent-context-budget.git\ncd agent-context-budget\ngit checkout 72d150669a9fa4ab85496b4b62440ce95a9be7fe\npython -m context_budget verify results/evaluation.json\n```\n\n### Repeat the local experiment\n\n```\npython3.11 -m venv .venv\n. .venv/bin/activate\npython -m pip install -r requirements-local.lock\npython scripts/download_model.py\npython -m context_budget evaluate --model-path .cache/model --output build/new-run.json\npython -m context_budget verify build/new-run.json\n```\n\n## Scope and limitations\n\n- Small author-known synthetic tasks, a single model and one observation per cell do not establish production quality or stable latency rankings.\n- Compression is extractive, not learned summarization. Selection uses lexical overlap and can introduce chronology ambiguity after ranking.\n- The implementation tests overflow handling, but the recorded run does not measure prompts longer than the declared model window.\n- Replay trusts recorded tokenization and checks consistency, not inference authenticity. Energy and hardware costs are unmeasured; no paid API was used.\n\n- [Related architecture article](<https://lucioduran.com/blog/agentic-ai-architecture-patterns>)\n\n## Sources and revision\n\nThe raw report identifies its implementation revision, timestamp, source hashes, task hashes and model hashes. Release documentation is separate from the recorded execution.\n\n- Reviewed revision: [` 72d150669a9fa4ab85496b4b62440ce95a9be7fe `](<https://github.com/lucioduran/agent-context-budget/commit/72d150669a9fa4ab85496b4b62440ce95a9be7fe>)\n- Revision recorded by the experiment: [` 4b38a6150b152758f99e4a12002d9b7e03a5d057 `](<https://github.com/lucioduran/agent-context-budget/commit/4b38a6150b152758f99e4a12002d9b7e03a5d057>)\n- Report timestamp (UTC): 2026-11-09T18:43:55.650481+00:00\n- Release published: [v0.1.1](<https://github.com/lucioduran/agent-context-budget/releases/tag/v0.1.1>) · 2026-11-09T18:54:01Z\n\n- [README and usage](<https://github.com/lucioduran/agent-context-budget/blob/72d150669a9fa4ab85496b4b62440ce95a9be7fe/README.md>)\n- [Original evidence](<https://raw.githubusercontent.com/lucioduran/agent-context-budget/72d150669a9fa4ab85496b4b62440ce95a9be7fe/results/evaluation.json>)\n- [MIT license](<https://github.com/lucioduran/agent-context-budget/blob/72d150669a9fa4ab85496b4b62440ce95a9be7fe/LICENSE>)\n- [Design and trust boundaries](<https://github.com/lucioduran/agent-context-budget/blob/72d150669a9fa4ab85496b4b62440ce95a9be7fe/docs/design.md>)\n- [Results summary](<https://raw.githubusercontent.com/lucioduran/agent-context-budget/72d150669a9fa4ab85496b4b62440ce95a9be7fe/results/summary.json>)\n- [Frozen evaluation protocol](<https://github.com/lucioduran/agent-context-budget/blob/72d150669a9fa4ab85496b4b62440ce95a9be7fe/docs/protocol.md>)\n- [Failure analysis](<https://github.com/lucioduran/agent-context-budget/blob/72d150669a9fa4ab85496b4b62440ce95a9be7fe/docs/error-analysis.md>)\n- [Model revision and license](<https://raw.githubusercontent.com/lucioduran/agent-context-budget/72d150669a9fa4ab85496b4b62440ce95a9be7fe/data/model.json>)","headings":[{"level":1,"text":"Agent Context Budget","url":"https://lucioduran.com/projects/agent-context-budget#title"},{"level":2,"text":"Problem and relevance to agents","url":"https://lucioduran.com/projects/agent-context-budget#problem"},{"level":2,"text":"Architecture","url":"https://lucioduran.com/projects/agent-context-budget#architecture"},{"level":2,"text":"What this demonstrates","url":"https://lucioduran.com/projects/agent-context-budget#demonstrates"},{"level":2,"text":"Published results","url":"https://lucioduran.com/projects/agent-context-budget#results"},{"level":3,"text":"Context policy evaluation","url":"https://lucioduran.com/projects/agent-context-budget#results-Context%20policy%20evaluation"},{"level":2,"text":"Reproduce","url":"https://lucioduran.com/projects/agent-context-budget#reproduce"},{"level":3,"text":"Verify saved evidence","url":"https://lucioduran.com/projects/agent-context-budget#reproduce-replay"},{"level":3,"text":"Repeat the local experiment","url":"https://lucioduran.com/projects/agent-context-budget#reproduce-experiment"},{"level":2,"text":"Scope and limitations","url":"https://lucioduran.com/projects/agent-context-budget#scope"},{"level":2,"text":"Sources and revision","url":"https://lucioduran.com/projects/agent-context-budget#sources"}],"sections":[{"heading":"Agent Context Budget","level":1,"url":"https://lucioduran.com/projects/agent-context-budget","content":"[Projects](<https://lucioduran.com/projects>) / Agent Context Budget"},{"heading":"Agent Context Budget","level":1,"url":"https://lucioduran.com/projects/agent-context-budget#title","content":"Compare full-window input, truncation, selection and extractive compression while measuring evidence loss separately from answer quality.\n\nAuthor: [Lucio Durán](<https://lucioduran.com/>) · Python 3.11.13 · License: [MIT](<https://github.com/lucioduran/agent-context-budget/blob/72d150669a9fa4ab85496b4b62440ce95a9be7fe/LICENSE>)\n\nPage published: 2026-11-09 · Page updated: 2026-11-09\n\n[Repository](<https://github.com/lucioduran/agent-context-budget>) · [Original evidence](<https://raw.githubusercontent.com/lucioduran/agent-context-budget/72d150669a9fa4ab85496b4b62440ce95a9be7fe/results/evaluation.json>) · [Release v0.1.1](<https://github.com/lucioduran/agent-context-budget/releases/tag/v0.1.1>)"},{"heading":"Problem and relevance to agents","level":2,"url":"https://lucioduran.com/projects/agent-context-budget#problem","content":"A compressed record can keep a citation identifier but lose the sentence that supports its answer. Token savings alone cannot establish useful context.\n\nFrozen tasks with late and contradictory facts expose which policies lose decisive evidence and whether a small local model uses what remains."},{"heading":"Architecture","level":2,"url":"https://lucioduran.com/projects/agent-context-budget#architecture","content":"1. **Freeze the experiment.** Twelve original fictional tasks and the evaluation protocol were committed before inference.\n2. **Apply exact token budgets.** A pinned tokenizer accounts for prompt overhead across chronological input, prefix truncation, record selection and sentence extraction.\n3. **Preserve raw decisions.** Prompts, fragments, source IDs, responses, token traces and resource observations support deterministic replay.\n\n- [Design and trust boundaries](<https://github.com/lucioduran/agent-context-budget/blob/72d150669a9fa4ab85496b4b62440ce95a9be7fe/docs/design.md>)\n- [Frozen evaluation protocol](<https://github.com/lucioduran/agent-context-budget/blob/72d150669a9fa4ab85496b4b62440ce95a9be7fe/docs/protocol.md>)"},{"heading":"What this demonstrates","level":2,"url":"https://lucioduran.com/projects/agent-context-budget#demonstrates","content":"- Separate scoring for retained evidence, retained citation identifiers, generated citations and exact answers.\n- Explicit recency and first-sentence ablations.\n- Actual pinned FLAN-T5-small CPU inference with recorded failures."},{"heading":"Published results","level":2,"url":"https://lucioduran.com/projects/agent-context-budget#results","content":"One local Xeon CPU run made 132 model calls across eleven settings using two PyTorch CPU threads.\n\nEach setting evaluates the same twelve tasks. Original prompts contain 142 to 418 tokens; none exceeds the model's 512-token input limit. Selected evidence and generated answers are scored independently."},{"heading":"Context policy evaluation","level":3,"url":"https://lucioduran.com/projects/agent-context-budget#results-Context%20policy%20evaluation","content":"| Policy and token budget | Exact answers | Evidence retained | Citation IDs retained | Mean input tokens |\n| --- | --- | --- | --- | --- |\n| full:512 | 10/12 | 1.000 | 1.000 | 280.4 |\n| head:128 | 0/12 | 0.000 | 0.000 | 128.0 |\n| head:256 | 5/12 | 0.500 | 0.500 | 205.2 |\n| select:128 | 6/12 | 1.000 | 1.000 | 111.4 |\n| select:256 | 3/12 | 1.000 | 1.000 | 195.9 |\n| compress:128 | 1/12 | 1.000 | 1.000 | 106.4 |\n| compress:256 | 2/12 | 1.000 | 1.000 | 141.4 |\n| select\\_no\\_recency:128 | 6/12 | 1.000 | 1.000 | 112.4 |\n| select\\_no\\_recency:256 | 4/12 | 1.000 | 1.000 | 196.4 |\n| compress\\_first:128 | 2/12 | 0.667 | 1.000 | 111.7 |\n| compress\\_first:256 | 3/12 | 0.667 | 1.000 | 157.2 |\n\nEvery setting produced zero correct output citations and zero grounded exact matches. First-sentence compression lost four corrections while retaining their citation IDs. Retaining evidence did not guarantee correct model use."},{"heading":"Reproduce","level":2,"url":"https://lucioduran.com/projects/agent-context-budget#reproduce","content":"Portable replay requires Python 3.11 or newer and no runtime model packages. Inference uses the pinned Intel macOS Python 3.11.13 environment with PyTorch 2.2.2 and Transformers 4.46.3.\n\nRun measurements in a new destination and serialize heavy workloads. Within the original project series, use the shared benchmark\\_lock.py wrapper. Commands below use POSIX shell syntax."},{"heading":"Verify saved evidence","level":3,"url":"https://lucioduran.com/projects/agent-context-budget#reproduce-replay","content":"```\ngit clone https://github.com/lucioduran/agent-context-budget.git\ncd agent-context-budget\ngit checkout 72d150669a9fa4ab85496b4b62440ce95a9be7fe\npython -m context_budget verify results/evaluation.json\n```"},{"heading":"Repeat the local experiment","level":3,"url":"https://lucioduran.com/projects/agent-context-budget#reproduce-experiment","content":"```\npython3.11 -m venv .venv\n. .venv/bin/activate\npython -m pip install -r requirements-local.lock\npython scripts/download_model.py\npython -m context_budget evaluate --model-path .cache/model --output build/new-run.json\npython -m context_budget verify build/new-run.json\n```"},{"heading":"Scope and limitations","level":2,"url":"https://lucioduran.com/projects/agent-context-budget#scope","content":"- Small author-known synthetic tasks, a single model and one observation per cell do not establish production quality or stable latency rankings.\n- Compression is extractive, not learned summarization. Selection uses lexical overlap and can introduce chronology ambiguity after ranking.\n- The implementation tests overflow handling, but the recorded run does not measure prompts longer than the declared model window.\n- Replay trusts recorded tokenization and checks consistency, not inference authenticity. Energy and hardware costs are unmeasured; no paid API was used.\n\n- [Related architecture article](<https://lucioduran.com/blog/agentic-ai-architecture-patterns>)"},{"heading":"Sources and revision","level":2,"url":"https://lucioduran.com/projects/agent-context-budget#sources","content":"The raw report identifies its implementation revision, timestamp, source hashes, task hashes and model hashes. Release documentation is separate from the recorded execution.\n\n- Reviewed revision: [` 72d150669a9fa4ab85496b4b62440ce95a9be7fe `](<https://github.com/lucioduran/agent-context-budget/commit/72d150669a9fa4ab85496b4b62440ce95a9be7fe>)\n- Revision recorded by the experiment: [` 4b38a6150b152758f99e4a12002d9b7e03a5d057 `](<https://github.com/lucioduran/agent-context-budget/commit/4b38a6150b152758f99e4a12002d9b7e03a5d057>)\n- Report timestamp (UTC): 2026-11-09T18:43:55.650481+00:00\n- Release published: [v0.1.1](<https://github.com/lucioduran/agent-context-budget/releases/tag/v0.1.1>) · 2026-11-09T18:54:01Z\n\n- [README and usage](<https://github.com/lucioduran/agent-context-budget/blob/72d150669a9fa4ab85496b4b62440ce95a9be7fe/README.md>)\n- [Original evidence](<https://raw.githubusercontent.com/lucioduran/agent-context-budget/72d150669a9fa4ab85496b4b62440ce95a9be7fe/results/evaluation.json>)\n- [MIT license](<https://github.com/lucioduran/agent-context-budget/blob/72d150669a9fa4ab85496b4b62440ce95a9be7fe/LICENSE>)\n- [Design and trust boundaries](<https://github.com/lucioduran/agent-context-budget/blob/72d150669a9fa4ab85496b4b62440ce95a9be7fe/docs/design.md>)\n- [Results summary](<https://raw.githubusercontent.com/lucioduran/agent-context-budget/72d150669a9fa4ab85496b4b62440ce95a9be7fe/results/summary.json>)\n- [Frozen evaluation protocol](<https://github.com/lucioduran/agent-context-budget/blob/72d150669a9fa4ab85496b4b62440ce95a9be7fe/docs/protocol.md>)\n- [Failure analysis](<https://github.com/lucioduran/agent-context-budget/blob/72d150669a9fa4ab85496b4b62440ce95a9be7fe/docs/error-analysis.md>)\n- [Model revision and license](<https://raw.githubusercontent.com/lucioduran/agent-context-budget/72d150669a9fa4ab85496b4b62440ce95a9be7fe/data/model.json>)"}],"links":[{"text":"Projects","url":"https://lucioduran.com/projects"},{"text":"Lucio Durán","url":"https://lucioduran.com/"},{"text":"MIT","url":"https://github.com/lucioduran/agent-context-budget/blob/72d150669a9fa4ab85496b4b62440ce95a9be7fe/LICENSE"},{"text":"Repository","url":"https://github.com/lucioduran/agent-context-budget"},{"text":"Original evidence","url":"https://raw.githubusercontent.com/lucioduran/agent-context-budget/72d150669a9fa4ab85496b4b62440ce95a9be7fe/results/evaluation.json"},{"text":"Release v0.1.1","url":"https://github.com/lucioduran/agent-context-budget/releases/tag/v0.1.1"},{"text":"Design and trust boundaries","url":"https://github.com/lucioduran/agent-context-budget/blob/72d150669a9fa4ab85496b4b62440ce95a9be7fe/docs/design.md"},{"text":"Frozen evaluation protocol","url":"https://github.com/lucioduran/agent-context-budget/blob/72d150669a9fa4ab85496b4b62440ce95a9be7fe/docs/protocol.md"},{"text":"Results summary","url":"https://raw.githubusercontent.com/lucioduran/agent-context-budget/72d150669a9fa4ab85496b4b62440ce95a9be7fe/results/summary.json"},{"text":"Related architecture article","url":"https://lucioduran.com/blog/agentic-ai-architecture-patterns"},{"text":"72d150669a9fa4ab85496b4b62440ce95a9be7fe","url":"https://github.com/lucioduran/agent-context-budget/commit/72d150669a9fa4ab85496b4b62440ce95a9be7fe"},{"text":"4b38a6150b152758f99e4a12002d9b7e03a5d057","url":"https://github.com/lucioduran/agent-context-budget/commit/4b38a6150b152758f99e4a12002d9b7e03a5d057"},{"text":"v0.1.1","url":"https://github.com/lucioduran/agent-context-budget/releases/tag/v0.1.1"},{"text":"README and usage","url":"https://github.com/lucioduran/agent-context-budget/blob/72d150669a9fa4ab85496b4b62440ce95a9be7fe/README.md"},{"text":"MIT license","url":"https://github.com/lucioduran/agent-context-budget/blob/72d150669a9fa4ab85496b4b62440ce95a9be7fe/LICENSE"},{"text":"Failure analysis","url":"https://github.com/lucioduran/agent-context-budget/blob/72d150669a9fa4ab85496b4b62440ce95a9be7fe/docs/error-analysis.md"},{"text":"Model revision and license","url":"https://raw.githubusercontent.com/lucioduran/agent-context-budget/72d150669a9fa4ab85496b4b62440ce95a9be7fe/data/model.json"}],"structuredData":[{"@context":"https://schema.org","@graph":[{"@type":"Person","@id":"https://lucioduran.com/#person","name":"Lucio Durán","url":"https://lucioduran.com/"},{"@type":"WebSite","@id":"https://lucioduran.com/#website","url":"https://lucioduran.com/","name":"Lucio Durán","inLanguage":["en","es"],"publisher":{"@id":"https://lucioduran.com/#person"}},{"@type":"WebPage","@id":"https://lucioduran.com/projects/agent-context-budget#page","url":"https://lucioduran.com/projects/agent-context-budget","name":"Agent Context Budget","description":"Compare full-window input, truncation, selection and extractive compression while measuring evidence loss separately from answer quality.","inLanguage":"en","author":{"@id":"https://lucioduran.com/#person"},"publisher":{"@id":"https://lucioduran.com/#person"},"isPartOf":{"@id":"https://lucioduran.com/#website"},"isAccessibleForFree":true,"mainEntity":{"@id":"https://lucioduran.com/projects/agent-context-budget#software"},"datePublished":"2026-11-09","dateModified":"2026-11-09","breadcrumb":{"@id":"https://lucioduran.com/projects/agent-context-budget#breadcrumb"}},{"@type":"SoftwareSourceCode","@id":"https://lucioduran.com/projects/agent-context-budget#software","name":"Agent Context Budget","description":"Compare full-window input, truncation, selection and extractive compression while measuring evidence loss separately from answer quality.","inLanguage":"en","url":"https://lucioduran.com/projects/agent-context-budget","codeRepository":"https://github.com/lucioduran/agent-context-budget","programmingLanguage":"Python","runtimePlatform":"Python 3.11.13","license":"https://github.com/lucioduran/agent-context-budget/blob/72d150669a9fa4ab85496b4b62440ce95a9be7fe/LICENSE","version":"v0.1.1","identifier":{"@type":"PropertyValue","propertyID":"git-commit","value":"72d150669a9fa4ab85496b4b62440ce95a9be7fe"},"author":{"@type":"Person","@id":"https://lucioduran.com/#person","name":"Lucio Durán","url":"https://lucioduran.com"},"isAccessibleForFree":true,"citation":["https://github.com/lucioduran/agent-context-budget/blob/72d150669a9fa4ab85496b4b62440ce95a9be7fe/README.md","https://raw.githubusercontent.com/lucioduran/agent-context-budget/72d150669a9fa4ab85496b4b62440ce95a9be7fe/results/evaluation.json","https://github.com/lucioduran/agent-context-budget/blob/72d150669a9fa4ab85496b4b62440ce95a9be7fe/docs/design.md","https://raw.githubusercontent.com/lucioduran/agent-context-budget/72d150669a9fa4ab85496b4b62440ce95a9be7fe/results/summary.json","https://github.com/lucioduran/agent-context-budget/blob/72d150669a9fa4ab85496b4b62440ce95a9be7fe/docs/protocol.md","https://github.com/lucioduran/agent-context-budget/blob/72d150669a9fa4ab85496b4b62440ce95a9be7fe/docs/error-analysis.md","https://raw.githubusercontent.com/lucioduran/agent-context-budget/72d150669a9fa4ab85496b4b62440ce95a9be7fe/data/model.json","https://github.com/lucioduran/agent-context-budget/releases/tag/v0.1.1"],"publisher":{"@id":"https://lucioduran.com/#person"},"mainEntityOfPage":{"@id":"https://lucioduran.com/projects/agent-context-budget#page"}},{"@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Projects","item":"https://lucioduran.com/projects"},{"@type":"ListItem","position":2,"name":"Agent Context Budget","item":"https://lucioduran.com/projects/agent-context-budget"}],"@id":"https://lucioduran.com/projects/agent-context-budget#breadcrumb"}]}],"contentHash":"195b6a34f94d13a73922321603c2195a47c372a58944b13871704c26686ad1b6","provenance":{"method":"Deterministic serialization of the server-rendered main element","source":"https://lucioduran.com/projects/agent-context-budget","hashAlgorithm":"sha256","hashScope":"JSON document fields before contentHash and provenance","sourceHTMLBytes":44790,"contentBytes":6952}}