Moonshot AI Unveils K3 Report Detailing Advanced Technologies for DeepSeek
Moonshot AI has released a comprehensive 47-page technical report on the K3 project, highlighting several innovative technologies. Key features include the KDA+Gated MLA hybrid attention mechanism, AttnRes cross-layer retrieval, and Stable LatentMoE, which utilizes 896 experts. Additionally, the rep...