Secondary sources, overlapping papers, and useful references that are worth keeping but not prominent enough for the main research shelf.
Quantified the operational cost of long context. Llama-3.1-70B showed a 719% latency increase at 15k-word context, while accuracy only dropped from 98.5% to 98%. Models don’t lose their way; they become operationally expensive. Memory bandwidth is the bottleneck, not compute.
Systematic study of how LLMs fail in multi-turn settings. Models make assumptions in early turns and prematurely commit to solutions, then over-rely on those early conclusions for the remainder of the conversation. The failure compounds: a wrong assumption in turn 3 typically persists through turn 20. Reinforces the case for explicit state management and periodic context resets; assumption drift compounds when left unchecked.
OpenAI’s latest model supports up to 1M tokens of context, enabling agents to plan, execute, and verify tasks across long horizons. The window size is a milestone, but the context engineering question remains: a million tokens of poorly structured context will still degrade. The ETH Zurich AGENTS.md study and the PAPerBench attention dilution findings apply at any window size. Bigger windows make context engineering more important, not less.
Memory-specific reranking model for the awkward cases where semantically similar memories miss the information needed to answer. The paper focuses on temporal constraints, causal reasoning, and dialogue disambiguation, which are exactly where generic rerankers tend to look good while selecting the wrong evidence. Worth keeping as a technical reference for retrieval pipelines, but too model-specific for the main shelf unless the site grows a dedicated memory retrieval implementation section.
Constructs instance-specific few-shot prompts by synthesizing on-the-fly examples matched to the current input. Addresses the core limitation of fixed example sets: examples selected for average cases perform poorly on edge cases. Dataset-free approach makes this practical for low-resource scenarios. Directly relevant to the Few-Shot Selection pattern: when no labeled pool exists, synthesize.
Domain-specific agent architecture for automated data science, with two pieces relevant beyond Kaggle-style tasks: autonomous skill acquisition and adaptive context compression. The reported 28.9% average gain over open-source data science agents is useful, and eliminating out-of-token failures is the part that belongs in this archive. It belongs in the archive for now; the lesson is real, but the domain is narrower than the coding-agent and general memory papers already there.
Formalizes agents as a tuple of (Instruction, Context, Tools, Model) and automates their creation. Achieved a 16.28% improvement over the strongest baseline. Directly relevant to the Isolate and Recursive Delegation patterns: the orchestrator curates task-relevant context and delegates via on-the-fly agent creation.
Practical hybrid approach combining observation masking and LLM summarization on SWE-bench. Achieved 7-11% cost reduction. Useful as a real-world engineering case study rather than a benchmark paper.
Demonstrated a +10.6% improvement on agent benchmarks and +8.6% on finance tasks through better context engineering alone. No model changes. The key insight: contexts should function as “comprehensive, evolving playbooks,” not concise summaries. Also introduced the concept of “context collapse,” where iterative rewriting erodes detail over time.
Technical breakdown triggered by IBM’s $11B Confluent acquisition. Argues that RAG is document retrieval, but context is operational state. Proposes an event-driven architecture (Kafka + Snowplow) for real-time context that changes minute-by-minute. Useful for understanding where static context patterns break down.
Production-oriented guidance: treat context as infrastructure, prune aggressively, store and reuse context development. The emphasis on memory layers (short-term session + long-term cross-session) as essential infrastructure rather than optional features.
Security architecture for the agent execution layer, where tool use, context management, and state persistence meet. The four layers are directly relevant to context infrastructure: adversarial context filtering, causal verification before decisions, privilege-separated tool control, and safe rollback during state updates. It belongs in the archive because the site has a narrower security scope, but the premise is right: once the execution layer controls context and tools, it becomes the attack surface.
GAAP applies information-flow control to agent execution, tracking how private data is accessed and where it may be disclosed across both single tasks and later tasks. The important context engineering lesson is that context access and data release cannot be governed by prompt instructions alone. If an agent can see private data, prompt injection can try to route it somewhere else. GAAP makes the permission model part of the execution environment, which is where this control belongs.
Product management perspective on why fine-tuning lost to context engineering: high effort, foundation models improved too fast, context engineering extracts enough value without retraining. Also frames the shift from “context window size” to “context relevance” as the real bottleneck.