The release worth reading closely this week is architectural, not product. DeepSeek-V4.1-Flash is an open-weight multimodal Mixture-of-Experts model with a 552B-parameter backbone under an MIT license, using a Causal Encoder-Decoder design — 40 Transformer layers split into 20 encoder and 20 decoder layers, where the decoder's KV cache is projected from final encoder states rather than recomputed per layer. It activates 8B parameters during prefill and 16B during decode.
The number that matters for anyone budgeting a retrieval system: cross-layer KV reuse in Compressed Sparse Attention 2 combined with FP4 KV caching brings the global KV footprint, always resident in HBM, to 890 bytes per token . That is roughly a quarter of its predecessor's cache and, per DeepSeek's figures, a 437-fold reduction versus DeepSeek V1. A full 1M-token prompt holds under 1 GB of global KV, plus a fixed 128-token sliding window per layer — meaning weights and batch size, not cache, now set your capacity limit.
Why this isn't "long context kills RAG" again
Two supporting mechanisms are the interesting part for memory architecture. First, a hierarchical sparse indexer: one Full Mode layer selects a blockwise candidate pool, and subsequent Reindex Mode layers only score candidates inside that restricted pool, keeping per-query cost bounded whether the context is 10,000 or 1,000,000 tokens . That is retrieval, implemented in attention, with the same coarse-to-fine structure as a two-stage recall-then-rerank pipeline. Second, SWA Bounded Replay reconstructs recent local KV states by replaying recent tokens rather than persisting a separate sliding-window cache — DeepSeek reports this alone cuts the persistent KV footprint to about one-eighth of V4-Flash's, before sparse attention is applied . Overall the model claims a 4x reduction in runtime KV storage and 8x in persistent storage against predecessors.
The persistent-storage number is the one to sit with. If you run long-lived agents, your real cost is often not the first prefill — it is storing and rehydrating session state across turns, days, and users. An 8x drop in what has to be kept on disk makes cached prefixes viable as a genuine memory tier rather than a five-minute TTL optimization. It does not make your knowledge base obsolete: nothing here gives you access control, freshness, provenance, or the ability to update a fact without re-encoding a corpus.
Caveats worth stating plainly. Benchmark results are mixed — ahead on DeepSWE and CyberGym, behind on Terminal-Bench 3.0/4.0 and ExploitGym , so treat it as an efficiency play, not a frontier swap. And deployment is early: it needs vLLM 0.30.0+, with no pip wheel serving the architecture, so installation is Docker-only .
If you build retrieval, the practical read is that the boundary between "in context" and "in the index" is moving, and your chunking and cache-eviction policies were probably tuned against last generation's economics.