Lu Yin

Professor, Shenzhen University of Advanced Technology (深圳理工大学), School of Computer Science and Artificial Intelligence · Lead of the LUMI Lab

CPAL 2026 Rising Star

Also reachable at l.yin@surrey.ac.uk · l.yin@tue.nl · lu.yin@tue.ellis.eu

prof_pic_uk.jpg

Greetings! I’m Lu. I lead the Lightweight & Universal Machine Intelligence (LUMI) lab. Before joining SUAT I was an Assistant Professor at the University of Surrey, a Postdoctoral Fellow at TU/e, and a research scientist intern at Google’s New York City office.

Feel free to reach out if you’d like to discuss anything with me :)

01

Efficient and Scalable Foundation Models

Making large models cheaper to train, adapt and serve — sparse training, post-training pruning and layer-wise sparsity allocation, and memory-efficient optimisation that keeps quality intact at high compression ratios.

02

Understanding and Enhancing LLMs

Studying why large language models behave the way they do — depth and layer utilisation, normalisation and scaling choices, weight-decay and heavy-tailed structure — and turning those findings into concrete training recipes.

03

LLM Agents and Self-Evolution

Agents built on large language models, and the mechanisms that let them improve themselves.

04

Foundation Models for Vertical Domains

Bringing foundation models into practical domains — healthcare and medical imaging, urban computing, and other applied areas.

Join us

The LUMI lab is recruiting at every level:

  • Research Assistants (RA)
  • PhD students — jointly supervised with Beihang University (北京航空航天大学) and Nanjing University (南京大学)
  • Research Assistant Professors (RAP)
  • Postdocs

If our interests overlap, email me your CV and a short note on what you would like to work on.

News

Papers Activities Grants
Aug 2026
Papers 3 EMNLP’26 accepted - DiaLLM + W2T + MAPLE.
Jul 2026
Papers 3 MICCAI’26 accepted - MedFM-Robust Oral + ToothSegAgent + CPR-Classification.
May 2026
Activities I serve as the Area Chair of NeurIPS 2026.
Activities Our LayerNorm Scaling has already become a winning technique in OpenAI’s Parameter Golf challenge.
Apr 2026
Papers 2 ACL Findings’26 accepted - SemiPA + Modality Alignment.
Mar 2026
Papers 1 CVPR Findings’26 accepted - Few-Shot 3D Segmentation.
Activities Honoured to deliver a tutorial on LLM Layerwise Efficiency at CPAL 2026. [Slides]
Feb 2026
Activities Honoured to join the ELLIS Society e.V. Many thanks to the ELLIS Board and my endorser for their support.
Activities I was selected as Rising Star of CPAL 2026.
Grants We gratefully acknowledge support from AIRR advanced supercomputers (UK; NVIDIA H200)
Papers 3 ICLR’26 accepted - LLM Merging + DLM Early Commit Decoding + Medical Image Analysis.
Jan 2026
Papers 2 ICASSP’26 accepted - CoT Compression + LLM understanding.
Nov 2025
Activities I have given invited talks at Tsinghua University.
Activities I serve as Area Char of CPAL 2026.
Activities I serve as the visiting scholar at ELLIS - Max-Planck-Campus Tübingen.
Oct 2025
Activities I serve as the Publicity Chair for International Conference on Artificial Intelligence and Agents (ICAIAgent 2026).
Sep 2025
Papers 1 Sustainable Cities and Society (IF = 12) accepted - urban green space + AI.
Grants Our project has secured 144,000 GPU hours (NVIDIA A100) through EuroHPC. Our sincere thanks go to EuroHPC
Aug 2025
Papers 1 BMVC’25 accepted - CLIMB-3D, about continual learning for imbalanced 3D instance segmentation.
Papers 1 ACM SIGSPATIAL’25 accepted - Into the Unknown, an urban foundation model for individuals’ next-location prediction.
Jun 2025
Papers 1 ACL’25 accepted - OWS, about LLM PEFT.
Papers 1 Computers, Environment and Urban Systems accepted - urban space foundation model CaLLiPer.
May 2025
Papers 2 ICML’25 accepted - Weight low-rank training WeLore + weight low-rank fine-tuning LIFT.
Feb 2025
Papers 1 CPAL’25 accepted - Q-Galore.
Papers 3 ICLR’25 accepted - Normalization for LLMs Mix-LN + Enhancing LLM Alignment with Ternary Preferences TODO + Debiasing via Spuriousness Ranking SEBRA.
Dec 2024
Activities Thrilled to co-organize the CAI 2025 Workshops LLM Stable Pretraining and Federated Optimization and Learning.
Sep 2024
Papers 1 NeurIPS’24 accepted - E2ENet.
Papers 2 EMNLP’24 accepted - FFN-SkipLLM + C4 Pruning Enough?.
Aug 2024
Jun 2024
Papers 2 Interspeech’24 accepted - Data Prunning for ASR + Training ASR from scatch in collaboration with Meta London.
Activities NeurIPS’24 challenge co-organized - Edge-Device Large Language Model Competition.
May 2024
Papers 3 ICML’24 accepted - LLM pruning OWL (with Google Research) + understanding small magnitudes in LLMs JunkDNA hypothesis (with Intel Research) + BiDST.
Activities Honored to be invited to give a talk about The Power of Model Sparsity at Multimedia Analytics (MA) Laboratory in City University of Hong Kong.
Dec 2023
Papers 3 CPAL’24 accepted - spotlight-track papers at CPAL.
Grants We have been awarded 10,000,000 credits for the use of NVIDIA A100 GPUs, totaling 78,120 hours. Our sincere thanks go to NWO.
Jul 2023
Papers 1 NeurIPS’23 accepted - Dynamic Sparsity Is Channel-Level Sparsity Learner.
Activities I am joining Google's NYC office as a researcher intern.
Jun 2023
Papers 2 ECML-PKDD’23 accepted - Robust overfitting + RDebiased Sparse Training.
Apr 2023
Papers 1 ICML’23 accepted - Large Kernel Distillation.
Nov 2022
Papers 1 AAAI’23 accepted - Lottery Pools.
Papers 1 LoG’22 accepted and Best Paper Award - Untrained GNNs Tickets.
Sep 2022
Activities CMU invited talk - Model/supervision Efficency at Xu Lab, Carnegie Mellon University.
May 2022
Papers 1 UAI’22 accepted - Sup-tickets sparse training.
Mar 2022
Papers 1 IDA’22 accepted - my first conference/symposium paper during the first year of my PhD.

Selected publications

Full list →

Selected Publications

  1. NeurIPS 2025
    AlphaDecay: Module-wise Weight Decay for Heavy-Tailed Balancing in LLMs
    He, Di; Jaiswal, Ajay; Tu, Songjun; Shen, Li; Yuan, Ganzhao; Liu, Shiwei; and Yin, Lu† († corresponding author)
  2. NeurIPS 2025
    The Curse of Depth in Large Language Models
    Sun, Wenfang; Song, Xinyuan; Li, Pengxiang; Yin, Lu; Zheng, Yefeng; and Liu, Shiwei
  3. NeurIPS 2025
    GPAS: Accelerating Convergence of LLM Pretraining via Gradient-Preserving Activation Scaling
    Chen, Tianhao; Xu, Xin; Liu, Zijing; Li, Pengxiang; Song, Xinyuan; Jaiswal, Ajay Kumar; Zhang, Fan; Hu, Jishan; Wang, Yang; Chen, Hao; Yin, Lu† and Yang, Can († corresponding author)
  4. ICLR 2025
    Mix-LN: Unleashing the Power of Deeper Layers by Combining Pre-LN and Post-LN
    Li, Pengxiang*; Yin, Lu*; and Liu, Shiwei (* equal contribution)
  5. ICLR 2025
    Sebra: DeBiasing through Self-Guided Bias Ranking
    Kappiyath, Adarsh; Chaudhuri, Abhra; Jaiswal, Ajay; Liu, Ziquan; Li, Yunpeng; Zhu, Xiatian; and Yin, Lu† († corresponding author)
  6. ICML 2024
    Outlier Weighed Layerwise Sparsity OWL: A Missing Secret Sauce for Pruning LLMs to High Sparsity
    Yin, Lu; You, Wu; Zhenyu, Zhang; Cheng-Yu, Hsieh; Yaqing, Wang; Yiling, Jia; Mykola, Pechenizkiy; Yi, Liang; Zhangyang, Wang; and Shiwei, Liu
  7. ICML 2024
    Junk DNA Hypothesis: Pruning Small Pre-Trained Weights Irreversibly and Monotonically Impairs "Difficult" Downstream Tasks in LLMs
    Yin, Lu; Jaiswal, Ajay; Liu, Shiwei; Kundu, Souvik; and Wang, Zhangyang
  8. NeurIPS 2023
    Dynamic Sparsity is Channel-level Sparsity Learner
    Yin, Lu; Li, Gen; Fang, Meng; Shen, Li; Huang, Tianjin; Wang, Zhangyang; Menkovski, Vlado; Ma, Xiaolong; Pechenizkiy, Mykola; Liu, Shiwei; and others,
  9. AAAI 2022
    Lottery Pools: Winning More by Interpolating Tickets without Increasing Training or Inference Cost
    Yin, Lu; Liu, Shiwei; Fang, Meng; Huang, Tianjin; Menkovski, Vlado; and Pechenizkiy,