25
Where the Savings Go: The Limits of Hiding Cross-Tenant Cache Hits in LLM Serving
arXiv:2611.20291v1 Announce Type: new Abstract: LLM serving engines cache prompt prefixes; a cache hit skips computation and returns faster, so a shared cache reveals whether others have sent a given text. Per-account salting closes this channel but forgoes cross-tenant reuse an…
🅲 hạng C