Moonshot AI releases Kimi-K2.6 model with 1T parameters, attention optimizations
Moonshot AI has released Kimi-K2.6, a large language model with 1 trillion parameters, outperforming GPT-4.5 and Claude Opus 4.6 in several benchmarks. The model features advanced technologies such as Swish-Gated Linear Unit and multi-head latent attention.
Moonshot AI has released Kimi-K2.6, the latest addition to its Kimi series of open-source large language models. The model has 1 trillion parameters and outperforms GPT-4.5 and Claude Opus 4.6 in several AI benchmarks. Kimi-K2.6 features an activation function called Swish-Gated Linear Unit, which is more hardware-efficient than earlier algorithms. The model's neural networks use multi-head latent attention to identify the most important part of a prompt. Kimi-K2.6 can process text and multimedia input, and can generate complete websites from simple user instructions. The model scored 54 on the HLE-Full benchmark, surpassing Opus 4.6 and GPT-4.5, which scored 53 and 52.1, respectively.
This content was automatically generated and/or translated by AI. It may contain inaccuracies. Please refer to the original sources for verification.