Xet is on the Hub

Xet migration moved roughly 6% of Hugging Face Hub download traffic to Xet storage, validating the architecture under real-world usage. Unlike LFS that deduplicates at the file level, Xet uses content-defined chunking (~64 KB) to deduplicate at the byte level, so edits affect only changed chunks and large files transfer faster. The system combines a Xet-aware client, a content-addressed store with an LFS bridge, and S3 for final storage, all geared toward production readiness.

Source

HuggingFace Blog

Ingéré

18 MARS 2025 · 19:10

Score édito

3.0 / 5