KV cache compression via block-diagonal rotation. Beats TurboQuant: better PPL (6.91 vs 7.07), 28% faster decode, 5.3x faster prefill, 44x fewer params. Drop-in llama.cpp integration.
Pulse
87
Velocity
1.8x
Stars
502
KV cache compression for LLMs using block-diagonal rotation, outperforming TurboQuant.
KV cache compression for LLMs achieves better PPL, faster decode, and fewer parameters.
Boost LLM performance and reduce memory with block-diagonal KV cache compression.
Why Trending
Novel KV cache compression techniques significantly boost LLM inference efficiency.
Target Audience
LLM developers optimizing inference speed and memory usage.
Similar Projects