How does Architecture Influence the Base Capabilities of Pre-trained Language Models? A Case Study Based on FFN-Wider and MoE Transformers - Lu, Xin; Zhao, Yanyan; Qin, Bing; Huo, Liangyu; Yang, Qing; Xu, Dongliang (Advances in Neural Information Processing Systems, 2024)