23
LoopFill: Pipelined Multi-Token Speculative Decoding for Looped Transformers
Looped Transformers improve parameter efficiency by repeatedly applying the same Transformer layers. During autoregressive decoding, however, generating each token requires several sequential passes through these layers, leading to high decoding latency. Pipelined self-speculativ…
🅲 hạng C