【Savanna：为卷积多混合模型（StripedHyena 2）预训练提供强大基础设施。亮点：1. 支持大规模分布式训练，优化千卡集群性能；2. 提供多种优化技术，如a2a和p2p上下文并行化；3. 已成功训练多个模型，如StripedHyena 7B和Evo 2 40B，覆盖超9T tokens】
'Savanna: Pretraining infrastructure for research and application of convolutional multi-hybrid models (StripedHyena 2).'
GitHub: github.com/Zymrael/savanna
#深度学习# #预训练模型# #大规模训练# #AI创造营#
'Savanna: Pretraining infrastructure for research and application of convolutional multi-hybrid models (StripedHyena 2).'
GitHub: github.com/Zymrael/savanna
#深度学习# #预训练模型# #大规模训练# #AI创造营#
