Chaojun Xiao


Post-Doctoral Researcher
Natural Language Processing Lab,
Department of Computer Science and Technology,
Tsinghua University, Beijing, China.

Contact

  xcjthu [at] gmail [dot] com
xcj [at] tsinghua [dot] edu [dot] cn
  Room 4-506, FIT Building, Tsinghua University
Beijing, 100084, China
  Github
  Google Scholar

About me

Hi! I am a post-doctoral researcher in the Department of Computer Science and Technology at Tsinghua University. I am advised by Professor Maosong Sun and Professor Zhiyuan Liu from Natural Language Processing Lab (THUNLP). My research interests lie within the intersection of natural language processing and large-scale language models. Before I becoming a post-doctoral researcher, I also received my doctoral degree and bachelor degree from Tsinghua University.

PUBLICATIONS

2025

1.   Densing Law of LLMs. Nature Machine Intelligence 2025. [pdf] Chaojun Xiao, Jie Cai, Weilin Zhao, Biyuan Lin, Guoyang Zeng, Jie Zhou, Zhi Zheng, Xu Han, Zhiyuan Liu, Maosong Sun.
2.   InfLLM-V2: Dense-sparse switchable attention for seamless short-to-long adaptation. Preprint 2025. [pdf] Weilin Zhao, Zihan Zhou, Zhou Su, Chaojun Xiao†, Yuxuan Li, Yanghao Li, Yudi Zhang, Weilun Zhao, Zhen Li, Yuxiang Huang, Ao Sun, Xu Han, Zhiyuan Liu.
3.   BlockFFN: Towards end-side acceleration-friendly mixture-of-experts with chunk-level activation sparsity. COLM 2025. [pdf] Chenyang Song, Weilin Zhao, Xu Han, Chaojun Xiao, Yingfa Chen, Yuxuan Li, Zhiyuan Liu, Maosong Sun.
4.   Document Segmentation Matters for Retrieval-Augmented Generation. ACL 2025 Findings. [pdf] Zhitong Wang, Cheng Gao, Chaojun Xiao†, Yufei Huang, Shuzheng Si, Kangyang Luo, Yuzhuo Bai, Wenhao Li, Tangjian Duan, Chuancheng Lv, Guoshan Lu, Gang Chen, Fanchao Qi, Maosong Sun.
5.   MiniCPM4: Ultra-efficient LLMs on end devices. Preprint 2025. [pdf] MiniCPM Team (Chaojun Xiao as technical lead).
6.   APB: Accelerating Distributed Long-Context Inference by Passing Compressed Context Blocks across GPUs. ACL 2025. [pdf] Yuxiang Huang, Mingye Li, Xu Han, Chaojun Xiao†, Weilin Zhao, Sun Ao, Hao Zhou, Jie Zhou, Zhiyuan Liu, Maosong Sun.
7.   Ultra-FineWeb: Efficient data filtering and verification for high-quality LLM training data. Preprint 2025. [pdf] Yudong Wang, Zixuan Fu, Jie Cai, Peijun Tang, Hongya Lyu, Yewei Fang, Zhi Zheng, Jie Zhou, Guoyang Zeng, Chaojun Xiao, Xu Han, Zhiyuan Liu.

2024

1.   InfLLM: Training-Free Long-Context Extrapolation for LLMs with an Efficient Context Memory. NeurIPS 2024. [pdf] Chaojun Xiao, Pengle Zhang, Xu Han, Guangxuan Xiao, Yankai Lin, Zhengyan Zhang, Zhiyuan Liu, Maosong Sun.
2.   Fine-Grained Legal Argument-Pair Extraction via Coarse-Grained Pre-training. COLING 2024. [pdf] Chaojun Xiao, Yutao Sun, Yuan Yao, Xu Han, Wenbin Zhang, Zhiyuan Liu and Maosong Sun.
3.   Exploring the Benefit of Activation Sparsity in Pre-training. ICML 2024. [pdf] Zhengyan Zhang, Chaojun Xiao, Qiujieli Qin, Yankai Lin, Zhiyuan Zeng, Xu Han, Zhiyuan Liu, Ruobing Xie, Maosong Sun, Jie Zhou.
4.   Configurable Foundation Models: Building LLMs from a Modular Perspective. Preprint 2024. [pdf] Chaojun Xiao, Zhengyan Zhang, Chenyang Song, Dazhi Jiang, Feng Yao, Xu Han, Xiaozhi Wang, Shuo Wang, Yufei Huang, Guanyu Lin, Yingfa Chen, Weilin Zhao, Yuge Tu, Zexuan Zhong, Ao Zhang, Chenglei Si, Khai Hao Moo, Chenyang Zhao, Huimin Chen, Yankai Lin, Zhiyuan Liu, Jingbo Shang, Maosong Sun.
5.   Enhancing Legal Case Retrieval via Scaling High-quality Synthetic Query-Candidate Pairs. EMNLP 2024. [pdf] Cheng Gao*, Chaojun Xiao*, Zhenghao Liu, Huimin Chen, Zhiyuan Liu, Maosong Sun.

2023

1.   Plug-and-play document modules for pre-trained models. ACL 2023. [pdf] Chaojun Xiao, Zhengyan Zhang, Xu Han, Chi-Min Chan, Yankai Lin, Zhiyuan Liu, Xiangyang Li, Zhonghua Li, Zhao Cao, Maosong Sun.
2.   Plug-and-play knowledge injection for pre-trained language models. ACL 2023. [pdf] Zhengyan Zhang, Zhiyuan Zeng, Yankai Lin, Huadong Wang, Deming Ye, Chaojun Xiao, Xu Han, Zhiyuan Liu, Peng Li, Maosong Sun, Jie Zhou.
3.   UPRec: User-Aware Pre-training for Recommender Systems. AI Open. [pdf] Chaojun Xiao, Ruobing Xie, Yuan Yao, Zhiyuan Liu, Maosong Sun, Xu Zhang, Leyu Lin.
4.   Variator: Accelerating pre-trained models with plug-and-play compression modules. EMNLP 2023 Findings. [pdf] Chaojun Xiao, Yuqi Luo, Wenbin Zhang, Pengle Zhang, Xu Han, Yankai Lin, Zhengyan Zhang, Ruobing Xie, Zhiyuan Liu, Maosong Sun, Jie Zhou.
5.   MUSER: A Multi-View Similar Case Retrieval Dataset. CIKM 2023 Resources. Best Resource Paper Honorable Mention [pdf] Qingquan Li, Yiran Hu, Feng Yao, Chaojun Xiao, Zhiyuan Liu, Maosong Sun, Weixing Shen.

2022

1.   LEVEN: A Large-Scale Chinese Legal Event Detection Dataset. ACL Findings 2022. Feng Yao*, Chaojun Xiao*, Xiaozhi Wang, Zhiyuan Liu, Lei Hou, Cunchao Tu, Juanzi Li, Yun Liu, Weixing Shen, Maosong Sun.

2021

1.   Adversarial Language Games for Advanced Natural Language Intelligence. AAAI 2021. Long paper. Yuan Yao, Haoxi Zhong, Zhengyan Zhang, Xu Han, Xiaozhi Wang, Chaojun Xiao, Guoyang Zeng, Zhiyuan Liu, Maosong Sun.
2.   Equality before the Law: Legal Judgment Consistency Analysis for Fairness. Preprint. Yuzhong Wang, Chaojun Xiao, Shirong Ma, Haoxi Zhong, Cunchao Tu, Tianyang Zhang, Zhiyuan Liu, Maosong Sun.
3.   Lawformer: A Pre-trained Language Model for Chinese Legal Long Documents. AI Open. SMP Best Paper Award. Chaojun Xiao, Xueyu Hu, Zhiyuan Liu, Cunchao Tu, Maosong Sun.
4.   CPM-2: Large-scale Cost-effective Pre-trained Language Models. AI Open. Zhengyan Zhang*, Yuxian Gu*, Xu Han*, Shengqi Chen*, Chaojun Xiao*, Zhenbo Sun, Yuan Yao, Fanchao Qi, Jian Guan, Pei Ke, Yanzheng Cai, Guoyang Zeng, Zhixing Tan, Zhiyuan Liu, Minlie Huang, Wentao Han, Yang Liu, Xiaoyan Zhu, Maosong Sun.

2020

1.   Denoising Relation Extraction from Document-level Distant Supervision. EMNLP 2020. Short paper. Chaojun Xiao, Yuan Yao, Ruobing Xie, Xu Han, Zhiyuan Liu, Maosong Sun, Fen Lin and Leyu Lin.
2.   How Does NLP Benefit Legal System: A Summary of Legal Artificial Intelligence. ACL 2020. Theme paper. Haoxi Zhong, Chaojun Xiao, Cunchao Tu, Tianyang Zhang, Zhiyuan Liu, Maosong Sun.
3.   More Data, More Relations, More context and More Openness: A Review and Outlook for Relation Extraction. AACL 2020. Long paper. Xu Han, Tianyu Gao, Yankai Lin, Hao Peng, Yaoliang Yang, Chaojun Xiao, Zhiyuan Liu, Peng Li, Maosong Sun, Jie Zhou.
4.   JEC-QA: A Legal-Domain Question Answering Dataset. AAAI 2020. Long paper. (* indicates equal contribution). Haoxi Zhong*, Chaojun Xiao*, Cunchao Tu, Tianyang Zhang, Zhiyuan Liu, Maosong Sun.
5.   Knowledge Transfer via Pre-training for Recommendation: A Review and Prospect. Frontiers in Big Data. (* indicates equal contribution). Zheni Zeng*, Chaojun Xiao*, Yuan Yao, Ruobing Xie, Zhiyuan Liu, Fen Lin, Leyu Lin, Maosong Sun.

Before 2020

1.   Legal Judgment Prediction via Topological Learning. EMNLP 2018. Long Paper. Haoxi Zhong, Zhipeng Guo, Cunchao Tu, Chaojun Xiao, Zhiyuan Liu, Maosong Sun.
2.   CAIL2019-SCM: A Dataset of Similar Case Matching in Legal Domain. Preprint. (* indicates equal contribution). Chaojun Xiao*, Haoxi Zhong*, Zhipeng Guo, Cunchao Tu, Zhiyuan Liu, Maosong Sun, Tianyang Zhang, Xianpei Han, Zhen Hu, Heng Wang, Jianfeng Xu.
3.   CAIL2018: A Large-Scale Legal Dataset for Judgment Prediction. Preprint. (* indicates equal contribution). Chaojun Xiao*, Haoxi Zhong*, Zhipeng Guo, Cunchao Tu, Zhiyuan Liu, Maosong Sun, Yansong Feng, Xianpei Han, Zhen Hu, Heng Wang, Jianfeng Xu.

PROJECTS

Open-Source Models

Datasets

EXPERIENCE

Postdoctoral Researcher

Department of Computer Science and Technology,
Tsinghua University, Beijing, China.
July 2025 - June 2027

Ph.D. student

Department of Computer Science and Technology,
Tsinghua University, Beijing, China.
August 2020 - June 2025

Bachelor of Engineering

Department of Computer Science and Technology,
Tsinghua University, Beijing, China.
August 2016 - July 2020

High School

Pingchuan High School, Xingguo, Jiangxi, China.
August 2013 - July 2016

Reviewer

AAAI, WWW, COLING, ACL, EMNLP, NAACL, ACL ARR, NeurIPS.

TA

Towards Artificial General Intelligence, Tsinghua University.
2024
Object-Oriented Programming, Tsinghua University.
2020 - 2022
Media Programming, Tsinghua University.
2019 - 2021.

Awards

Fellowship of China National Postdoctoral Program for Innovative Talents
2025
Felloship of Shui Mu Tsinghua Scholar Program
2025
Outstanding Doctoral Dissertation of Tsinghua University
2025

First Prize of Qian Weichang Award for Chinese Information Processing Science and Technology.
2024
First-class Tencent Rhino-Bird Elite Training Program Excellent Student.
2023
Second-class Overall Excellence Scholarship, Tsinghua University.
SMP Best Paper Award.
2022

Excellent Graduate, Beijing.
Excellent Graduate, Tsinghua University.
Excellent Graduate, Dept. of CS&T, Tsinghua University.
2020

First-class Price in Challenge Cup Contest, Beijing.
First-class Science and Technology Innovation Excellence Scholarship, Tsinghua University.
2019

First-class Price in Challenge Cup Contest, Tsinghua University.
First-class Overall Excellence Scholarship, Tsinghua University.
2018

First-class Overall Excellence Scholarship, Tsinghua University.
Gaotong Scholarship, Tsinghua University.
2017