Lu, Xudong

About Me

I am currently a third-year Ph.D. student at the Multimedia Laboratory (MMLab), The Chinese University of Hong Kong. My research advisor is Prof. Hongsheng Li. I am also an intern at Huawei Leibniz Research Center (Hong Kong Research Center) (2025.8-). Previously, I had the privilege to intern at vivo AI Lab (2024.6-2025.7) and Shanghai AI Laboratory (2023.2-2023.8).

I received the B.E. degree from the Department of Computer Science and Engineering at Shanghai Jiao Tong University in 2023, ranking 2/119 in my major. I was awarded the National Scholarship (top 0.2%) in 2020, 2021, and 2022, and the Hong Kong PhD Fellowship Scheme (HKPFS) in 2023.

During my undergraduate studies, from 2020 to 2022, I worked closely with Prof. Yonglu Li and Prof. Cewu Lu on human activity understanding. From 2022 to 2023, I collaborated with Prof. Xue Yang and Prof. Junchi Yan on AI for Science (AI4Science). My current research interests include model compression, large language models, and multimodal large language models.

My research focuses on advancing AI hardware and applications across diverse environments, spanning from high-computing power devices like NVIDIA A100 to low-computing power devices like Jetson Orin, including consumer GPUs such as RTX 4090 and NPUs like Qualcomm Snapdragon 8 Elite for smartphones.

News

August 2025: One paper on the evaluation of end-side LLM (first author) is accepted by EMNLP 2025.
June 2025: One paper on MLLM (first author) is accepted by ICCV 2025.
May 2025: One paper on oriented object detection is accepted by IJCV (IF 11.6, 2024).
February 2025: Two papers on MLLMs (one first author) are accepted by CVPR 2025.
February 2025: Two papers (one on KV Cache pruning, another on autoformalization) are accepted by ICLR 2025 (both Spotlight).
November 2024: We are happy to introduce BlueLM-V-3B, an algorithm and system co-design method for multimodal large language models on mobile devices.🔥🔥🔥
July 2024: One paper on video generation is accepted by ECCV 2024 (Oral).
May 2024: One paper (first author) on MoE LLM pruning is accepted by ACL 2024.
May 2024: One paper (first author) on LLaMA-alike LLM pruning is accepted by ICML 2024.
February 2024: One paper on Human Action Understanding is accepted by CVPR 2024 (Highlight).
August 2023: I am joining the MMLab of CUHK as a Ph.D. student.
June 2023: I am honored to have the opportunity to speak as the student representative at the undergraduate commencement ceremony of the college.
April 2023: Two papers (one first author) on Quantum Machine Learning are accepted by ICML 2023.
March 2023: I am awarded with HKPFS (Hong Kong PhD Fellowship Scheme).

Preprint

Point2RBox-v3: Self-Bootstrapping from Point Annotations via Integrated Pseudo-Label Refinement and Utilization
Teng Zhang*, Ziqian Fan*, Mingxin Liu, Xin Zhang, Xudong Lu, Wentong Li, Yue Zhou, Yi Yu, Xiang Li, Junchi Yan, Xue Yang
Arxiv Preprint 2025
[Paper]

GLEAM: Learning to Match and Explain in Cross-View Geo-Localization
Xudong Lu*, Zhi Zheng*, Yi Wan, Yongxiang Yao, Annan Wang, Renrui Zhang, Panwang Xia, Qiong Wu, Qingyun Li, Weifeng Lin, Xiangyu Zhao, Peifeng Ma, Xue Yang, Hongsheng Li
Arxiv Preprint 2025
[Paper]

Rethinking Video Tokenization: A Conditioned Diffusion-based Approach
Nianzu Yang*, Pandeng Li*, Liming Zhao, Yang Li, Chen-Wei Xie, Yehui Tang, Xudong Lu, Zhihang Liu, Yun Zheng, Yu Liu, Junchi Yan
Arxiv Preprint 2025
[Paper]

TerDiT: Ternary Diffusion Models with Transformers
Xudong Lu, Aojun Zhou, Ziyi Lin, Qi Liu, Yuhui Xu, Renrui Zhang, Xue Yang, Junchi Yan, Peng Gao, Hongsheng Li
Arxiv Preprint 2024
[Paper]

Selected Publications

SmartBench: Is Your LLM Truly a Good Chinese Smartphone Assistant?
Xudong Lu*, Haohao Gao*, Renshou Wu*, Shuai Ren, Xiaoxin Chen, Hongsheng Li, Fangyuan Li
The 2025 Conference on Empirical Methods in Natural Language Processing (EMNLP Main Conference), 2025
[Paper][新智元 (Aiera)]

GenieBlue: Integrating both Linguistic and Multimodal Capabilities for Large Language Models on Mobile Devices

Xudong Lu*, Yinghao Chen*, Renshou Wu*, Haohao Gao, Xi Chen, Xue Yang, Xiangyu Zhao, Aojun Zhou, Fangyuan Li, Yafei Wen, Xiaoxin Chen, Shuai Ren, Hongsheng Li
International Conference on Computer Vision (ICCV), 2025
[Paper][量子位 (QbitAI)]

PointOBB-v3: Expanding Performance Boundaries of Single Point-Supervised Oriented Object Detection
Peiyuan Zhang*, Junwei Luo*, Xue Yang*, Yi Yu, Qingyun Li, Yue Zhou, Xiaosong Jia, Xudong Lu, Jingdong Chen, Xiang Li, Junchi Yan, Yansheng Li
International Journal of Computer Vision (IJCV), 2025
[Paper]

Adaptive Markup Language Generation for Contextually-Grounded Visual Document Understanding

Han Xiao, Yina Xie, Guanxin Tan, Yinghao Chen, Rui Hu, Ke Wang, Aojun Zhou, Hao Li, Hao Shao, Xudong Lu, Peng Gao, Yafei Wen, Xiaoxin Chen, Shuai Ren, Hongsheng Li
The IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025
[Paper]

BlueLM-V-3B: Algorithm and System Co-Design for Multimodal Large Language Models on Mobile Devices

Xudong Lu*, Yinghao Chen*, Cheng Chen*, Hui Tan*, Boheng Chen, Yina Xie, Rui Hu, Guanxin Tan, Renshou Wu, Yan Hu, Yi Zeng, Lei Wu, Liuyang Bian, Zhaoxiong Wang, Long Liu, Yanzhou Yang, Han Xiao, Aojun Zhou, Yafei Wen, Xiaoxin Chen, Shuai Ren, Hongsheng Li

The IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025
[Paper][机器之心 (Synced China)]

Rethinking and Improving Autoformalization: Towards a Faithful Metric and a Dependency Retrieval-based Approach
Qi Liu, Xinhao Zheng, Xudong Lu, Qinxiang Cao, Junchi Yan
International Conference on Learning Representations (ICLR), 2025, (Spotlight)
[Paper]

ThinK: Thinner Key Cache by Query-Driven Pruning
Yuhui Xu, Zhanming Jie, Hanze Dong, Lei Wang, Xudong Lu, Aojun Zhou, Amrita Saha, Caiming Xiong, Doyen Sahoo
International Conference on Learning Representations (ICLR), 2025, (Spotlight)
[Paper]

ZoLA: Zero-Shot Creative Long Animation Generation with Short Video Model
Fu-Yun Wang, Zhaoyang Huang, Qiang Ma, Guanglu Song, Xudong Lu, Weikang Bian, Yijin Li, Yu Liu, Hongsheng Li
European Conference on Computer Vision (ECCV), 2024, (Oral)
[Paper]

Not All Experts are Equal: Efficient Expert Pruning and Skipping for Mixture-of-Experts Large Language Models
Xudong Lu*, Qi Liu*, Yuhui Xu, Aojun Zhou, Siyuan Huang, Bo Zhang, Junchi Yan, Hongsheng Li
Annual Meeting of the Association for Computational Linguistics (ACL Main Conference), 2024
[Paper]

SPP: Sparsity-Preserved Parameter-Efficient Fine-Tuning for Large Language Models
Xudong Lu*, Aojun Zhou*, Yuhui Xu*, Renrui Zhang, Peng Gao, Hongsheng Li
International Conference on Machine Learning (ICML), 2024
[Paper]

From isolated islands to pangea: Unifying semantic space for human action understanding
Yong-Lu Li*, Xiaoqian Wu*, Xinpeng Liu, Zehao Wang, Yiming Dou, Yikun Ji, Junyi Zhang, Yixing Li, Xudong Lu, Jingru Tan, Cewu Lu
The IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024, (Highlight)
[Paper]

QuantumDARTS: Differentiable Quantum Architecture Search for Variational Quantum Algorithms
Wenjie Wu, Ge Yan, Xudong Lu, Kaisen Pan, Junchi Yan
International Conference on Machine Learning (ICML), 2023
[Paper]

QAS-Bench: Rethinking Quantum Architecture Search and A Benchmark
Xudong Lu, Kaisen Pan, Ge Yan, Jiaming Shan, Wenjie Wu, Junchi Yan
International Conference on Machine Learning (ICML), 2023
[Paper]

Educational Background

Aug. 2023 - Present, Department of Electronic Engineering, the Chinese University of Hong Kong

Ph.D. Student

Aug. 2019 - Jun. 2023, Department of Computer Science and Engineering, Shanghai Jiao Tong University

Bachelor

Internship and Cooperation

Academic Service

Journal Reviewer

IEEE TCSVT, IEEE TPAMI

Conference Reviewer

CVPR, NeurIPS, ICLR, ICCV, ICML, AAAI