SmolLM3: smol, multilingual, long-context reasoner

SmolLM3 is a competitive open 3B model (11T tokens) with 128k long context via NoPE and YaRN, and multilingual support across six languages. It uses a Llama-based decoder with Grouped Query Attention, intra-document masking, and training-stability tweaks, plus a dual Instruct/Reasoning model.

Source

HuggingFace Blog

Ingéré

08 JUIL. 2025 · 19:10

Score édito

4.0 / 5