Model Merging in Pre-training of Large Language Models
Yunshui Li; Yiyuan Ma; Shen Yan; Chaoyi Zhang; Jing Liu; Jianqiao Lu; Ziwen Xu; Mengzhao Chen; Minrui Wang; Shiyi Zhan; Jin Ma; Xunhao Lai; Yao Luo; Xingyan Bin; Hongbin Ren; Mingji Han; Wenhao Hao; Bairen Yi; LingJun Liu; Bole Ma
- Published
- Publisher
- Neural Information Processing Systems Foundation, Inc. (NeurIPS)
- Journal / collection
- Advances in Neural Information Processing Systems 38
- Research area
- AI & compute
- Source type
- Conference paper
- DOI
- 10.52202/085713-4456
The original publisher provides the full text and its access options.
Cite this source
Yunshui Li; Yiyuan Ma; Shen Yan; Chaoyi Zhang; Jing Liu; Jianqiao Lu; Ziwen Xu; Mengzhao Chen; Minrui Wang; Shiyi Zhan; Jin Ma; Xunhao Lai; Yao Luo; Xingyan Bin; Hongbin Ren; Mingji Han; Wenhao Hao; Bairen Yi; LingJun Liu; Bole Ma. Model Merging in Pre-training of Large Language Models. Advances in Neural Information Processing Systems 38. 2025. https://doi.org/10.52202/085713-4456