I am an Associate Professor and Master’s supervisor at the Institute of Information Engineering, Chinese Academy of Sciences (CAS), working on large-scale statistical learning theory and large language models (LLMs). In July 2020, I obtained my Ph.D. degree from Institute of Information Engineering, CAS, advised by Associate Prof. Yong Liu and Prof. Weiping Wang.

In response to the challenges faced by large language models, such as high computational resource demands and weak foundational theories, my research is dedicated to exploring the underlying principles of large language models. The goal is to design efficient and interpretable lightweight language models, thereby narrowing the gap between foundational theory and practical algorithms. Specific research interests include, but are not limited to:

Lightweight Language Models: Investigating the underlying principles of scaling guidelines for large language models to guide the design of the next generation of efficient and deployable lightweight language models. This involves technologies such as model architecture design, model compression, and high-quality instruction fine-tuning.

LLMs and Deep Learning Theories: Delving into the unique phenomena of large language models, such as scaling guidelines, contextual learning abilities, complex reasoning capabilities, and the underlying principles of benign overfitting and the double descent phenomenon in deep learning.

Large-Scale Machine Learning Methods and Theory: Researching the generalization theory of large-scale machine learning methods and using the results from generalization theory to enhance large-scale algorithms. This includes studying various methods like federated learning, distributed learning, random features, Nyström methods, sketching methods, and more.

Curriculum Vitae (CV)

Selected Papers [Full List] [Google Scholar]

* corresponding author

Preprint

• A Survey on Model Compression for Large Language Models. [pdf]
  Xunyu Zhu, Jian Li*, Yong Liu, Can Ma, Weiping Wang.
  arXiv:2308.07633.

Journal Papers

• Optimal Convergence Rates for Distributed Nyström Approximation. [pdf] [code]
  Jian Li, Yong Liu, Weiping Wang.
  Journal of Machine Learning Research (JMLR), 2023. CCF-A.

• Optimal Convergence for Agnostic Kernel Learning With Random Features. [pdf] [code]
  Jian Li, Yong Liu, Weiping Wang.
  IEEE Transactions on Neural Networks and Learning Systems (TNNLS), 2023. CCF-B.

• Semi-supervised vector-valued learning: Improved bounds and algorithms. [pdf]
  Jian Li, Yong Liu, Weiping Wang.
  Pattern Recognition (PR), 2023. CCF-B.

• Improving Differentiable Architecture Search via Self-distillation. [pdf]
  Xunyu Zhu, Jian Li*, Yong Liu, Weiping Wang.
  Neural Networks, 2023. CCF-B.

• Convolutional Spectral Kernel Learning with Generalization Guarantees. [Paper] [Code]
  Jian Li, Yong Liu, Weiping Wang.
  Artificial Intelligence (AIJ), 2022. CCF-A.

• Non-IID Federated Learning with Sharper Risk Bound. [pdf]
  Bojian Wei, Jian Li*, Yong Liu, Weiping Wang.
  IEEE Transactions on Neural Networks and Learning Systems (TNNLS), 2022. CCF-B.

Conference Papers

• Optimal Convergence Rates for Agnostic Nyström Kernel Learning. [pdf]
  Jian Li, Yong Liu, Weiping Wang.
  International Conference on Machine Learning (ICML), 2023. CCF-A.

• Towards Sharp Analysis for Distributed Learning with Random Features. [pdf]
  Jian Li, Yong Liu.
  International Joint Conference on Artificial Intelligence (IJCAI), 2023. CCF-A.

• Ridgeless Regression with Random Features. [pdf] [code]
  Jian Li, Yong Liu, Yingying Zhang.
  International Joint Conference on Artificial Intelligence (IJCAI), 2022. CCF-A.

• Federated learning for non-iid data: From theory to algorithm. [pdf] [presentation] [🏆Best student paper award]
  Bojian Wei, Jian Li*, Yong Liu, Weiping Wang.
  Pacific Rim International Conference on Artificial Intelligence (PRICAI), 2021. CCF-C.

• Automated Spectral Kernel Learning. [pdf] [poster] [code]
  Jian Li, Yong Liu, Weiping Wang.
  AAAI Conference on Artificial Intelligence (AAAI), 2020. CCF-A.

• Multi-Class Learning: From Theory to Algorithm. [pdf] [poster] [sildes] [3-minute video] [code]
  Jian Li, Yong Liu, Rong Yin, Hua Zhang, Lizhong Ding, Weiping Wang.
  Advances in Neural Information Processing Systems 31 (NeurIPS), 2018. CCF-A.

Projects

• National Key R&D Program of China (2022YFB3105302.2), 2022.12 - 2025.11, ¥1,200,000.
  Aggregation and Collaborative Techniques for Cross-platforms Heterogenous Data.

• National Natural Science Foundation of China (No. 62106257), 2022.01 - 2024.12, ¥300,000.
  Large Scale Structured Prediction with Automated Spectral Kernel Learning.

• China Postdoctoral Science Foundation (Special Support, No. 2023T160680), 2023.07 - 2024.03, ¥180,000.
  Research on Deep Differentiable Gaussian Processes for Structured Prediction.

• Special Research Assistant Project of CAS, 2020.09 - 2022.09, ¥800,000.
  Large-scale Few-shot Automated Machine Learning.

• Talent Program Class A of Institute of Information Engineering, CAS, Tenure-track Professor, 2023.11 - 2026.09.

• Talent Program Class B of Institute of Information Engineering, CAS, Tenure-track Young Professor, 2020.09 - 2023.11.

Patents

Pending

• Jian Li, Yong Liu, Liubin Wang, Yiguo Yang, Juhong Wang.Neural Network Architecture Search Method, Device, Computer Equipment, and Storage Medium. CN：202011567991.3. App. Date: December 25, 2020.
• Jian Li, Jiaoyang Li, Bojian Wei, Yong Liu, Weiping Wang. A Federated Learning Method and System Based on Attention Mechanism. CN: 202311073645.3. App. Date: August 24, 2023
• Jian Li, Jiaoyang Li, Zheng Lin, Yong Liu, Weiping Wang. A Vertical Domain Large Model Method and System Based on Knowledge Distillation and Prompt Engineering. CN: 202311073641.5. App. Date: August 24, 2023.

Granted

• Hailun Lin, Yong Liu, Jian Li, Weiping Wang. A Large-Scale Ontology Merging Method that Integrates Representation Learning and Divide-and-Conquer Strategy: China. Granted No.CN110059194A. Granted Date: April 8, 2022.

Students

• Ph.D. students
  • 🎓Yilin Kang (2020.09 - 2023.06 ), Differential Privacy. Papers: Computers & Security, CIKM 2022, ICCS 2023. Post-graduation: Researcher in Purple Mountain Laboratories.
  • Xunyu Zhu (2020.09 - present), Neural Architecture Search. Papers: ICDM 2021.
  • Boxuan Che (2022.09 - present), Efficient Graph Neural Networks.
• Master’s students
  • 🎓Bojian Wei (2020.09 - 2022.06), Federated Learning on Heterogenous Data. Papers: PRICAI 2021 (best student paper award), ECML-PKDD 2022, TNNLS, IJCNN 2023. Post-graduation: Management Trainee in Bank of China Head Office.
  • Xuning Zhang (2022.09 - present), Federated Learning. Excellent Bachelor’s Thesis in Wuhan University in 2023.

Honors and Awards

• PRICAI 2021 best student paper award, 2021.
• Outstanding Graduate of Institute of Information Engineering, CAS, 2021.
• Outstanding Graduates of Beijing, 2020.
• Outstanding Graduates of University of Chinese Academy of Sciences (UCAS), 2020.
• Outstanding Graduates of Institute of Information Engineering, CAS, 2020.
• National Scholarship for Doctoral students, 2019.
• ZhuLiYueHua Scholarship for Excellent Doctoral Student, 2019.
• CAS Presidential Scholarship, 2019.
• National Scholarship for Doctoral students, 2018.
• IIE Presidential Scholarship, 2018.

Academic Service

• Mathematics Guest Editor
• Program committee of Conference: ICML, NeurIPS, ICLR, AAAI, IJCAI, ECAI, etc.
• Reviewers of Journals: TPAMI, JMLR, Pattern Recognition, etc.