A coding agent reads a file, runs a test, searches the repository and inspects a log. Each action is reasonable. A hundred actions later, the conversation contains yesterday’s hypothesis, three versions of the same function and a successful installation nobody needs to read again.
The next model call inherits the accumulated record. Buying a larger context window can postpone the limit. It does not decide which information deserves to remain.
This is an architectural problem: what enters memory, what stays, what can be recovered, and who gets to see it? A context optimiser can help. It can also add a new processor, another sensitive cache and a new way to lose evidence.
We evaluate AI as engineers, not as technology promoters.
A token can leave a long shadow.
For a single request, input tokens × input price plus output tokens × output price is a useful starting point. An agent is a sequence of requests. Context admitted early can be included repeatedly in later invocations, until it is removed, summarised or excluded by the orchestration system.
A 50k-token log retained across 20 later calls contributes roughly one million token-exposures to those calls. That excludes its first appearance. “Exposure” here counts repeated presentation to the model; it is neither a provider billing unit nor a measurement of internal attention weights.
One long conversation outweighed seventeen agents.
We counted one working session of our own coding agent from its local logs. The main conversation accounted for 91% of the session’s tokens, and 98% of those were cached context being read again. The seventeen parallel agents it launched accounted for 9%.
Weighted by price, with a cache read at about a tenth of fresh input, the split is closer to 86% and 14%. The long conversation, not the many agents, carried the cost. Method: API calls de-duplicated by request ID; weights from Anthropic’s published cache multipliers, with output at five times input, so this is an estimate, not an invoice.[2] It is also the Evidence step of the adoption gate in section 07, in miniature.
Caching changes the price of repeated input. Compaction changes what survives. Different agents construct different requests. The architectural observation still matters: old material competes for a finite input budget. Claude Code’s own cost guidance identifies context management as part of controlling usage.[1]
Not a billing calculator. Each call adds the selected tool volume before invocation; retained output stays for all later calls. A fixed 4k instruction budget is never pruned. This deliberately excludes context limits, caching, output tokens, judge overhead, recall and compaction. “Pruning” is hypothetical removal at admission, not measured accuracy.
Show the calculation and data
For call t: raw = 4k + t × incoming; filtered = 4k + t × incoming × (1 − pruning). Exposure = sum of each call’s context.
| Call | Raw tokens | Filtered tokens |
|---|---|---|
| 1 | 18,000 | 11,000 |
| 2 | 32,000 | 18,000 |
| 3 | 46,000 | 25,000 |
| 4 | 60,000 | 32,000 |
| 5 | 74,000 | 39,000 |
| 6 | 88,000 | 46,000 |
| 7 | 102,000 | 53,000 |
| 8 | 116,000 | 60,000 |
| 9 | 130,000 | 67,000 |
| 10 | 144,000 | 74,000 |
| 11 | 158,000 | 81,000 |
| 12 | 172,000 | 88,000 |
| 13 | 186,000 | 95,000 |
| 14 | 200,000 | 102,000 |
| 15 | 214,000 | 109,000 |
| 16 | 228,000 | 116,000 |
| 17 | 242,000 | 123,000 |
| 18 | 256,000 | 130,000 |
| 19 | 270,000 | 137,000 |
| 20 | 284,000 | 144,000 |
The prompt is only the beginning.
User instructions and project rules establish the task. Files, search matches, test output, compiler diagnostics and Docker or server logs can then dominate a long session. Tool-call arguments, prior assistant answers, exposed reasoning where retained, and failed approaches also contribute. Project instructions such as CLAUDE.md, loaded skills and rules have their own footprint.
User requirements, security rules and architecture decisions.
Current code, errors, search results and test outcomes.
Prior decisions, superseded output and paths already explored.
Those categories need different retention rules. A one-line security constraint may be more important than a thousand-line log. Age and size are useful signals; neither is a sufficient deletion policy.
Remember the decision.
Recover the evidence.
The objective is useful information per token of context, subject to correctness and policy constraints. Preserve the user’s goal, hard rules, unresolved failures, decisions and relevant changes. Preserve why a failed approach failed. Successful progress logs and obsolete directory listings are usually better candidates for removal.
Memory management is a useful analogy. It is not literal garbage collection: software can prove that an object is unreachable; an agent often cannot prove that evidence will never matter again. A safer design separates immediate visibility from recoverability.
In working context
The frontier model sees this now: the goal, hard constraints, current evidence and unresolved errors. It has immediate access, but every retained token occupies the context budget. Limit admission and preserve provenance.
- Full data
- Full selected content
- Retrieval
- No retrieval call
- Security
- Sent to the current model provider
A policy model, not a required sequence: cold evidence can return to hot; some output should expire immediately. Storage location alone does not establish a trust boundary.
Read all four states
HOT: The frontier model sees this now: the goal, hard constraints, current evidence and unresolved errors. It has immediate access, but every retained token occupies the context budget. Limit admission and preserve provenance. Full selected content. No retrieval call. Sent to the current model provider.
WARM: A structured summary or index preserves decisions, file versions and open questions. Include the pointer in hot context when needed. A summary can omit an exception; keep an approved source for critical details. Summary here; original elsewhere if retained. Small lookup or expansion. Summaries can still contain sensitive data.
COLD: Full approved evidence sits outside the active prompt. A stable identifier, source revision and access policy make retrieval possible. Returning a file from yesterday without its revision can quietly restore the wrong state. Yes, subject to retention policy. Tool call, latency and reintroduced tokens. Access control, encryption and expiry required.
EXPIRED: Evidence with no remaining operational or retention requirement is deleted under policy. This is not just hiding it from the model. Account for caches and backups, and never expire unresolved constraints merely because they are old. No recoverable copy promised. May require fresh evidence. Deletion must follow retention obligations.
Retrieval is part of the design, not an escape hatch.
A stub should say what was hidden, where it came from and how to recover it. Record a file revision or content hash, not just a path. A tool must enforce the same access policy when recalling data as when first reading it. Expiry needs an owner; “recoverable forever” is an unbounded retention policy.
Keep uncertain information. If the relevance service times out, retaining approved output is usually a better reliability fallback than deleting it. If the security gate fails, however, do not forward unclassified data to an external judge. Reliability fallback and data-protection fallback need different rules.
Test still failing: expected one delivery, received two. The trace identifies the retry handler.
Choose an action to see the engineering reasoning.
Dependencies installed successfully. Hundreds of progress lines; no warnings or errors.
Choose an action to see the engineering reasoning.
Search found 180 matches. The target function has been located and its current file opened.
Choose an action to see the engineering reasoning.
Customer records must stay inside the approved processing boundary.
Choose an action to see the engineering reasoning.
A directory listing from before the module move. The current tree has been checked.
Choose an action to see the engineering reasoning.
We tried disabling retries. It hid the failure but lost deliveries, so we reverted the change.
Choose an action to see the engineering reasoning.
Token saving is not
the same as cost saving.
Three interventions are often bundled into one savings claim. They act at different points in the system, have different overheads and require different evaluation.
| Question | Context pruning | Compaction | Model routing |
|---|---|---|---|
| What changes? | Which evidence enters or remains in context | How accumulated history is represented | Which model handles a request |
| Less frontier input? | Yes, if material stays removed | Yes, if the replacement is shorter | Yes for requests diverted elsewhere |
| Fewer total tokens? | Possibly; include judge and recall tokens | Possibly; include summarisation and rework | Not necessarily; similar input may move to a cheaper model |
| Lower total cost? | Only if avoided work exceeds overhead | Only if later savings exceed compaction cost | Only if cheaper execution survives retries and quality checks |
| New data boundary? | Possible with an external judge | Possible with an external compactor | Possible with another provider |
| Main trade-off | Missing needed evidence | Losing detail or relationships | Routing a task beyond the model’s capability |
Prompt caching is a fourth, separate mechanism. It can make repeated input cheaper; cache writes, reads and eligibility have their own rules. Changing an early prefix can also change cache reuse. Compare actual cached and uncached usage, not an assumed flat input price.[2]
A smaller prompt is not automatically cheaper end to end. A pruner could remove cheap cached material, pay for a judge, trigger fresh cache writes and then force an expensive reread. Nor does a cache hit establish that stale material is useful. Measure both economics and task quality.
Jev: a bounded decision model.
TypeSafe describes Jev as a System One model: unstructured state goes in; typed decisions with probabilities come out. It gives up free-form string generation. That makes it a plausible candidate for keep/prune, duplicate/novel, ranking candidates or selecting a route.[3]
“How should we solve this?”
Architecture, substantial code, ambiguous debugging and deep security analysis need open-ended reasoning and verification.
“Which choice fits?”
A small, predefined decision space can suit a cheaper judge. A well-typed answer can still be the wrong answer.
As checked on 21 September 2026, TypeSafe lists Jev 1.13 at US$0.042 per million input tokens, with no output-token charge. That is a model price, not an integration cost or an ROI result. Model aliases can move; pin and record the evaluated version.[4]
Low cost is a reason to test a bounded workload, not permission to delegate an ambiguous engineering decision. A probability threshold needs calibration against your own tasks and the asymmetric cost of deleting useful evidence. TypeSafe also publishes integration skills; a skill is integration guidance, not proof that a workflow is safe or economical.[7]
Two things matter more than the interface details: a legal answer can still be the wrong answer, and a probability is only worth acting on once it has been checked against your own labelled outcomes.
Context pruning also exposes a broader lesson that section 07 returns to: not every intelligent decision belongs in the same model. Open-ended reasoning, bounded judgement and deterministic execution can be different components.
Technical notes · Jev’s three answer types, parallel questions and calibration
Three interfaces for three kinds of judgment.
The documented primitives make the decision interface concrete.[14]
Returns the probability of yes, from 0 to 1. There is no separate confidence field. A value near 0.5 expresses uncertainty, not medium importance.
Returns a selected option, probabilities across the options and a confidence statistic. Include an explicit fallback when the options may be incomplete.
Returns a position along defined levels, their legend, probabilities and confidence. Define what the levels mean; urgency is different from certainty.
These outputs remove the need to extract an answer from generated prose. The surrounding application still needs transport-error handling, permission checks, version compatibility and validation of the combined decision. A fixed “above 90% means execute” rule is not a universal automation policy: choose thresholds using observed errors and the consequences of each action.
One state, many questions, one call.
Another useful capability is parallel judgment. TypeSafe documents evaluating multiple questions in one request; Jev ingests the shared state once and evaluates questions against it in parallel.[4] This can avoid a chain of network round trips for judgments that can all use the same evidence.
For a home-control request, an application could ask about intent, room, device category, target device, desired on/off state, group scope, compound instructions, missing information, ambiguity and whether clarification is needed. These are illustrative question definitions, not a live call or a guarantee of consistent answers.
TypeSafe calls its pattern speculative fan-out: ask even potentially irrelevant branch questions upfront, then discard answers that do not apply. Its documentation says additional questions usually have little effect on response time, not that question count has no cost or limit.[13]
Distinguish a branch dependency from an evidence dependency. A question about a lamp may be answered speculatively from the same state before code decides whether the request concerns lamps. A question about whether the lamp actually switched on needs new device feedback and must wait. Parallel judgments do not make a sequential physical process parallel.
Validate the combined result: individually plausible answers may conflict. Keep authorisation in deterministic policy, and treat model-reported ambiguity as a signal rather than proof that all ambiguity was detected. Measure request size, billed usage and end-to-end latency; ten questions do not establish a tenfold speedup. More questions also consume the documented request budget.[4]
For context management, this suggests judging multiple blocks against one approved task state, then applying retention rules in code. The benefit comes from shared evidence and fewer sequential calls; the data boundary and false-prune risk still need evaluation.
Calibration: knowing when to abstain.
Speed and price are only part of the argument. TypeSafe calls its training method Reinforcement Learning for Calibrated Decisions (RLCD), with the stated goal of producing decisions accompanied by honest probabilities. That is a vendor-described training objective, not independent evidence that every enterprise workload is calibrated.[3]
Calibration asks whether predicted probabilities match observed frequencies. Across many comparable predictions assigned about 80% probability, a well-calibrated system should be correct about 80% of the time. It is a property assessed over a population, not a guarantee about one decision.[12]
A device choice stays within the declared options. This constrains output structure.
Choosing a real device can still target the wrong room. Legal output is not correct intent.
Compare predicted likelihood with labelled outcomes on representative tasks.
TypeSafe’s launch explanation grounds its zero-hallucination/type-error claim in guaranteed schema matching, and says the figure is not empirical. That does not establish zero semantic errors: selecting “unlock” instead of “leave locked” can be well-formed and still wrong.[3]
The API also distinguishes probabilities from confidence. For Choice and Score, the documentation defines confidence as a statistic computed from the distribution; it makes no claim that this is the probability the selected answer is correct. Do not interpret confidence = 0.9 as an independently validated 90% success rate.[11]
Uncertainty becomes an input to the control policy.
For context pruning, uncertain evidence can stay. For a device request, ambiguity can trigger clarification. A validated, low-risk action may proceed after permission and state checks. These are application policies, not permissions granted by a confidence score.
Evaluate calibration and accuracy separately. Use held-out, human-labelled examples; compare predicted probabilities with observed outcomes by action and data class. Report sample sizes, accepted-action error rates and how much work is deferred. Test incomplete option sets, unfamiliar inputs and changes in model versions. A model can be calibrated overall yet unreliable on the particular action that matters most. Keep consequential actions behind independent controls.
Admission control: filter before accumulation.
Winnow is one concrete example. Its repository documents block-level filtering of large Read, Bash and Grep results, keeping uncertain blocks and replacing hidden material with recallable stubs. The documented external payload includes output blocks, a short task description and tool arguments. Optional summaries go to Anthropic whichever judge is configured, and the adapter judge sends judging there too. Shadow mode judges and logs without rewriting or summarising; it is not a no-transmission mode.[5]
Compaction: even useful history grows.
Admission control cannot solve an indefinitely growing session. Traditional compaction replaces old history with a shorter summary. This can preserve meaning economically, but exact commands, exceptions and the reasoning behind a decision can be lost. Selective compaction instead tries to retain chosen evidence verbatim and remove obsolete tool history.
The original fast-jev-compaction repository documents preserving user and assistant text, pinning recent history, and selectively deleting or truncating tool calls/results. Its judge state contains conversation text and tool inputs, with tool results represented by metadata; that state is fitted to a budget. “The judge receives every full result” would therefore be an inaccurate blanket description. The integration is a case study, not a deployment recommendation.[6]
Preserving some text verbatim does not make the whole process lossless. Removing an old result can destroy the evidence needed to interpret a retained answer. Test semantic continuity, tool-call pairing and the recovery path after compaction.
To save context, the optimiser first has to see it.
An open-source codebase may be a reasonable trial environment. An enterprise development session can also contain customer records, production logs, proprietary code or credentials accidentally printed by a command. Public source code does not make every tool result public.
This example already uses an external frontier model. Its approval does not automatically extend to a second processor.
Security classification comes before an external relevance decision.
Local-only handling, approved isolation or human review
Optional external relevance judge
Detection is imperfect. A clean regex scan is not a proof of safety. Use source classification and destination allowlists alongside detection, and keep unknown material inside the approved boundary. Running the classifier externally would move the same problem one stage earlier.
Zero retention does not mean zero transmission.
A zero-data-retention commitment concerns how a provider retains information. It does not mean the provider never received or processed it. Review the applicable endpoint, contract, operational exceptions, subprocessors and processing locations rather than treating “ZDR” as a universal property of a product.
External processing may create additional privacy, contractual or data-governance obligations. In Australia, OAIC’s APP 8 guidance distinguishes use from disclosure and describes cross-border responsibilities and exceptions. Whether a particular arrangement is a disclosure depends on its circumstances and effective control; geography alone does not settle that question. Privacy, residency, contracts and jurisdiction need assessment in the specific enterprise context. This is architecture guidance, not a legal conclusion.[8]
A judge that stays inside the boundary: Laya.
A judge that runs inside your own environment removes the extra recipient. Laya, from ConvAI Innovations, is an open model built for the same job as Jev: typed choice, score and noul answers with probabilities, and no generated text. It is released under Apache-2.0, so it can run offline. Its repository appeared on 18 September 2026.[15]
Three cautions keep it in proportion. It is not a drop-in replacement for Jev’s API. Its headline results are self-reported and come from a checkpoint fine-tuned for the benchmark. And for the review in section 05, it answers the data-boundary objection, not the evidence one: it would still need labelled examples from our own sessions, calibration, and the staged evaluation in section 06.
Technical notes · How to read Laya’s published results
Its checkpoints are ModernBERT and mmBERT encoders of 322–421 million parameters, about 1.7 GB as fp32 ONNX weights. What its own README reports, and how to read it:[16]
- The headline typed-decisions score, 0.766 against Jev 1.13’s 0.727, comes from a checkpoint fine-tuned for that benchmark. The base checkpoints score about 0.36, close to chance. The Jev figures are third-party measurements, not a side-by-side run.
- 33–40 ms per question is measured on a T4 GPU and compared with Jev’s cloud round trip. On CPU, the project reports 193–464 ms.
- Jev leads where there are many options (Banking77: 0.870 against 0.425). It also has better raw calibration before temperature fitting (ECE 0.144 against 0.213).
- The English checkpoint fails confidently on non-Latin scripts: 0.000 accuracy at 0.952 confidence on Khmer. A confidence threshold on its own would not catch that.
A repository a few days old is also a maintenance risk.
Local is a location, not a security guarantee.
Winnow documents full hidden output in ~/.winnow/cache/, manual cleanup and a clean command. Do not infer an automatic retention guarantee from the existence of that command.[5]
A recoverable cache is another copy of the data. For any implementation, inspect whether its storage is plaintext, who can read it and whether encryption covers it. Source code, logs, customer information and accidentally printed secrets can outlive the session in backups or synced home directories.
- Who owns expiry, and does it actually run?
- Can another user, process, agent or tenant recall this entry?
- Are file permissions and encryption adequate for the data class?
- Do backups, device sync, telemetry and crash reports make further copies?
- Can recall identifiers or summaries themselves reveal sensitive information?
Observability should normally record identifiers, sizes and decisions rather than duplicate payloads. Logging the entire judge request to debug a privacy layer defeats the purpose.
When the right answer
was “not yet”.
During an architecture review of a production AI system handling sensitive business and customer information, we investigated a Jev-based context-pruning layer for Claude Code. The initial question was straightforward: could it reduce accumulated tool context?
The more consequential question was what the optimiser itself would receive. The development environment could encounter sensitive records, credentials and proprietary source. An external judge would introduce a processing boundary that needed its own approval. Shadow evaluation would not remove that concern, and recoverable local storage introduced another lifecycle to manage.
We did not start by installing the optimiser. We reviewed deterministic noise, reduced avoidable command verbosity and reviewed secret and data hygiene. We considered local analysis of historical sessions and planned to measure the remaining waste before adding another model.
The technology may be useful.
The architecture was not yet ready for it.
This is a qualitative account, not a benchmark. We are not claiming a measured token reduction, cost saving or production outcome from an optimiser we did not deploy.
Sometimes the best AI optimisation requires no AI.
Repeated Git warnings, stale worktrees, verbose successful commands, duplicate diagnostics and unbounded searches are deterministic problems. Fix their causes; constrain output to the evidence the next step needs. Preserve errors and meaningful warnings rather than blindly discarding stderr or piping away an exit status.
For example, a test runner can return the failing test names, relevant frames and a pointer to the full approved log. A search can limit its scope to the suspected module. A successful installation can record its outcome and changed dependency versions. None requires a model to classify the progress bar.
Measure again after those changes. If the residual waste is small, the correct architecture may be the one with fewer moving parts.
Compression ratio
is the easy metric.
A filter removing 80% of context can be worse than one removing 30% if it creates a long debugging loop. Those percentages are illustrative, not results. We need to observe what happens after information disappears.
How often did we remove information the agent later needed?
Operational definition for a trial: confirmed needed removals ÷ all evaluated removal decisions. Report the sample size and a fixed observation window, alongside severity and downstream rework.
subject to an acceptable regret rate.
Recall frequency is a useful signal, but it is not ground truth: an agent can recover missing evidence by searching again, or fail without noticing what it lost. Conversely, a precautionary recall does not prove a harmful deletion. Winnow itself uses regret terminology and reports recall-based signals; the measurement proposed here adds human confirmation and outcome review.[5]
Label whether removed evidence was needed, inspect stalled and failed runs, and compare completion quality on matched tasks. Set a stricter tolerance for security constraints and irreversible actions than for recoverable diagnostic output. Zero observed failures in a small sample does not establish zero risk.
A staged evaluation, with an exit at every stage.
- 00
Baseline
Record session duration, model calls, fresh input, cached input, output, tool-result volume, compaction count, developer intervention and task success. Include unsuccessful tasks; averages over successes hide rework.
- 01
Offline / local replay
Use authorised historical sessions locally to identify source volume, deterministic noise and theoretically removable context. Treat transcripts as sensitive. A replay command is only offline if its implementation actually avoids external calls.
- 02
Approved shadow evaluation
Use a non-sensitive dataset or an explicitly approved processor boundary. Measure eligible outputs, proposed reduction, false-prune labels, latency and judge cost. Shadow mode cannot by itself establish downstream behaviour because the agent still sees the original context.
- 03
Conservative active filtering
Keep uncertain evidence, preserve recall and compare matched task outcomes. Record rereads, retries, human interventions and tail latency. Test unavailable judges, expired cache entries and policy failures.
- 04
Selective history compaction
Only after admission is understood, evaluate long-session continuity. Check that goals, constraints, current file state and failed approaches survive, and that the agent can resume correctly.
- 05
Optional routing
Route bounded repetitive decisions to a cheaper model where evidence supports it. Evaluate routing separately from pruning so that the source of any savings remains identifiable.
Compare actual provider charges for matched workloads and separately value developer time. Record model versions, prompt versions and cache behaviour. Use a holdout set for thresholds and a rollback rule for outcome degradation. A controlled trial should be able to conclude “do not deploy”.
Agent memory needs observability.
We observe CPU, RAM, queries and network traffic. Agent memory deserves a similarly explicit record: what entered, why, when, how large it was, how long it survived, why it left and whether it returned. Attribute cost from usage telemetry; do not pretend a character count is an invoice.
- Evidence
- test-result / revision 7
- Admitted
- call 12 · unresolved retry failure
- Transition
- hot → cold at call 18 · failure resolved
- Decision
- policy v3 · approved cache · expires under policy
- Later outcome
- recalled at call 22 · review whether removal was premature
These records make failures diagnosable without filling another database with raw customer data. Keep the audit trail useful, bounded and access-controlled.
Does the architecture
actually need it?
The Ingendia AI Adoption Gate applies beyond context pruning: retrieval systems, MCP integrations, local models, fine-tuning and routing all deserve the same sequence of questions. Progress depends on evidence, not novelty.
What failure are we trying to change?
Name a workload, owner and measurable outcome: slow debugging, excessive input or avoidable developer intervention.
No concrete problem → do not deploy.Read the complete gate
- Problem: What failure are we trying to change?
Name a workload, owner and measurable outcome: slow debugging, excessive input or avoidable developer intervention. No concrete problem → do not deploy.
- Evidence: Is the problem large enough to matter?
Measure representative completed tasks. Separate cached input, fresh input, retries, output and unsuccessful sessions. No meaningful baseline → instrument first.
- Data boundary: What will see, store or transmit our data?
Map judges, adapters, summaries, caches and telemetry. Approve destinations, data classes, access and retention. Unapproved sensitive data → local handling or stop.
- Simplest intervention: Can we remove the cause at source?
Quiet successful commands, bound search output, fix repeated warnings and clean stale environments after checking ownership. Deterministic waste → fix it before adding a model.
- Controlled trial: Can we measure failure safely?
Start with authorised local replay, then an approved shadow dataset. Label needed evidence and test recall plus fallback. Unsafe data boundary or no recovery → stop.
- Economics: Does the whole system improve?
Subtract judging, summaries, cache misses, retrieval, rework and operating effort from measured savings. Compare task quality and tail latency. No material net benefit → keep the simpler system.
- Deployment: Can we operate and reverse the change?
Pin versions, set regret and latency limits, assign an owner, keep an audit trail and define rollback triggers. Deploy only while measured benefit exceeds the added risk and complexity.
Going further · From language to action: interfaces, robots and real-time control
Beyond context: from language to action.
Context pruning is only one use of a bounded decision model. The same pattern can select a link, identify a target device or choose the next permitted action. A useful description is a semantic decision interface: natural language and current state become typed choices that ordinary software can consume. Jev is a model behind that interface, not a new programming language or a complete controller.
In TypeSafe’s smart-home demonstration, one API call evaluates several questions about intent, device scope and action, and a mock home changes accordingly; compound requests are first split into atomic commands by an LLM, then evaluated one at a time. It illustrates semantic command selection, not a measured physical-control loop.[9] TypeSafe’s Wikiracing demonstration similarly selects among available links. The published latency figures describe vendor measurements, not guaranteed response deadlines.[3]
For phone or browser control, an analogous design could select among actions extracted from the current interface. That is an architectural possibility, not a verified phone-control benchmark here. Jev’s current documented input is text: screen perception, speech recognition or sensor interpretation must supply an appropriate representation first.[4]
The application defines the action vocabulary. The model can propose a device and an on/off action; code checks that the device exists, the caller is authorised and the operation is currently permitted. A valid output type does not establish correct intent, permission or physical safety.
A promising decision layer
Smart-home requests, interface action selection and choosing an approved robot task are candidates for evaluation. Keep uncertain requests on a clarification path and verify the resulting state.
A separate control contract
Servo loops, motor control and emergency stops require their own timing and safety design. A fast cloud inference demo does not establish those guarantees.
Real-time engineering concerns meeting deadlines, not simply a low average latency.[10] Our proposed role for a Jev-like service is supervisory: select a bounded task while local controllers retain responsibility for time-critical execution and independent safety enforcement. It should not sit in the emergency-stop path.
Evaluate the full loop: perception, network, inference, validation, execution and feedback. Measure tail latency, jitter, deadline misses and action success. Reject stale decisions, define timeout behaviour for the specific device and prevent retries from executing an action twice. A robot’s safe response to a timeout must be engineered; it cannot be inferred from a model’s confidence.
This connects back to memory architecture. An action selector needs a small, fresh state with explicit permissions, not a long conversation containing contradictory device states. Fast decisions and well-managed context belong to the same control-plane design.
Use intelligence according to the shape of the problem.
Generation and bounded decision-making do not always require the same model. A frontier model can reason about architecture and code. A smaller control plane can classify, rank or route. Deterministic software should handle deterministic noise. Policy gates constrain every path.
Models, prices and native agent memory systems will change. The durable investment is the ability to inspect these boundaries, compare alternatives and remove a component when it no longer earns its complexity.
Better memory.
Better forgetting.
A good engineer does not keep every terminal window, obsolete stack trace and failed hypothesis open forever. An agent should not either. But forgetting also has a cost.
The next generation of enterprise AI infrastructure will need to decide what a model no longer needs to know — without losing what matters.
The job is not to deploy every new AI technology.The job is to know when the architecture needs it.
Sources & scope.
Primary documentation reviewed 21 September 2026; Laya on 22 September 2026, days after its repository appeared. Repository behaviour is a dated documentation snapshot, not an independent runtime audit. Recheck the exact revision, configuration and processor terms before a trial.
- Claude Code · Manage costs effectively ↗
- Anthropic · Prompt caching ↗
- TypeSafe · System One and Jev ↗
- TypeSafe · Current models and data handling ↗
- Winnow · Original repository and data-flow documentation ↗
- fast-jev-compaction · Original repository ↗
- TypeSafe · Official integration skills ↗
- OAIC · APP 8 cross-border disclosure guidance ↗
- TypeSafe · Smart home demo (page with embedded video) ↗
- ROS 2 design · Introduction to real-time systems ↗
- TypeSafe · Confidence and probability ↗
- Guo et al. · On Calibration of Modern Neural Networks (ICML 2017) ↗
- TypeSafe · Speculative fan-out ↗
- TypeSafe · Primitives and answer fields ↗
- ConvAI Innovations · Laya repository and benchmarks ↗
- Laya · Model card (Hugging Face) ↗
The simulations, architecture diagrams and decision exercises are illustrative. No benchmark establishes enterprise ROI here. No universal savings percentage, hallucination-free guarantee or claim of legal compliance is made. The anonymised case is an engineering decision record, with no identifying operational details.