Topic: inference speedup

Mixture-of-Recursions Boosts Inference Speed 2x-Implementation Guide

MoR (Mixture-of-Recursions) is an innovative architecture that improves LLM efficiency, achieving up to 2x faster inference speeds without accuracy loss by combining parameter sharing and adaptive computation. MoR introduces a lightweight router for dynamic recursion depth allocation and ...