Lu Yin

prof_pic.jpg

Greetings! I’m Lu, an Assistant Professor in the School of Computer Science and Electronic Engineering at the University of Surrey. I am honored to be a long-term visitor and collaborator with the Visual Informatics Group (VITA) at UT Austin. Additionally, I am a visiting scholar at Eindhoven University of Technology (TU/e) and MPI-ELLIS.

Previously, I served as a Postdoctoral Fellow at TU/e and worked as a research scientist intern at Google’s New York City office.

I was selected as a CPAL 2026 Rising Star. I lead the Lightweight & Universal Machine Intelligence (LUMI) lab. My research interests include:

  • Efficient and Scalable Foundation Models
  • Understanding and Enhancing LLMs
  • Interdisciplinary AI Applications

Feel free to reach out if you’d like to discuss anything with me :)

l.yin@surrey.ac.uk (preferred)
l.yin@tue.nl; lu.yin@tue.ellis.eu

News

Papers Activities Grants
Apr 2026
Papers 2 ACL Findings’26 accepted - SemiPA + Modality Alignment.
Mar 2026
Papers 1 CVPR Findings’26 accepted - Few-Shot 3D Segmentation.
Activities Honoured to deliver a tutorial on LLM Layerwise Efficiency at CPAL 2026. [Slides]
Feb 2026
Activities Honoured to join the ELLIS Society e.V. Many thanks to the ELLIS Board and my endorser for their support.
Activities I was selected as Rising Star of CPAL 2026.
Grants We gratefully acknowledge support from AIRR advanced supercomputers (UK; NVIDIA H200)
Papers 3 ICLR’26 accepted - LLM Merging + DLM Early Commit Decoding + Medical Image Analysis.
Jan 2026
Papers 2 ICASSP’26 accepted - CoT Compression + LLM understanding.
Nov 2025
Activities I have given invited talks at Tsinghua University.
Activities I serve as Area Char of CPAL 2026.
Activities I serve as the visiting scholar at ELLIS - Max-Planck-Campus Tübingen.
Oct 2025
Activities I serve as the Publicity Chair for International Conference on Artificial Intelligence and Agents (ICAIAgent 2026).
Sep 2025
Papers 1 Sustainable Cities and Society (IF = 12) accepted - urban green space + AI.
Papers 3 NeurIPS’25 accepted - Layer-wise Weight Decay in LLM + Gradient-Preserving Activation Scaling in LLM + The Curse of Depth in LLM.
Grants Our project has secured 144,000 GPU hours (NVIDIA A100) through EuroHPC. Our sincere thanks go to EuroHPC
Aug 2025
Papers 1 BMVC’25 accepted - CLIMB-3D, about continual learning for imbalanced 3D instance segmentation.
Papers 1 ACM SIGSPATIAL’25 accepted - Into the Unknown, an urban foundation model for individuals’ next-location prediction.
Jun 2025
Papers 1 ACL’25 accepted - OWS, about LLM PEFT.
Papers 1 Computers, Environment and Urban Systems accepted - urban space foundation model CaLLiPer.
May 2025
Papers 2 ICML’25 accepted - Weight low-rank training WeLore + weight low-rank fine-tuning LIFT.
Feb 2025
Papers 1 CPAL’25 accepted - Q-Galore.
Papers 3 ICLR’25 accepted - Normalization for LLMs Mix-LN + Enhancing LLM Alignment with Ternary Preferences TODO + Debiasing via Spuriousness Ranking SEBRA.
Dec 2024
Papers 1 ICASSP’25 accepted - Low-Rank Weight Training for Modern Speech Recognition Models.
Activities Thrilled to co-organize the CAI 2025 Workshops LLM Stable Pretraining and Federated Optimization and Learning.
Sep 2024
Papers 1 NeurIPS’24 accepted - E2ENet.
Papers 2 EMNLP’24 accepted - FFN-SkipLLM + C4 Pruning Enough?.
Aug 2024
Papers 1 BMVC’24 accepted - Are Sparse Neural Networks Better Hard Sample Learners?.
Jun 2024
Papers 2 Interspeech’24 accepted - Data Prunning for ASR + Training ASR from scatch in collaboration with Meta London.
Activities NeurIPS’24 challenge co-organized - Edge-Device Large Language Model Competition.
May 2024
Papers 3 ICML’24 accepted - LLM pruning OWL (with Google Research) + understanding small magnitudes in LLMs JunkDNA hypothesis (with Intel Research) + BiDST.
Jan 2024
Papers 1 ICLR’24 accepted - NeurRev: Train Better Sparse Neural Network Practically via Neuron Revitalization.
Activities Honored to be invited to give a talk about The Power of Model Sparsity at Multimedia Analytics (MA) Laboratory in City University of Hong Kong.
Dec 2023
Papers 3 CPAL’24 accepted - spotlight-track papers at CPAL.
Grants We have been awarded 10,000,000 credits for the use of NVIDIA A100 GPUs, totaling 78,120 hours. Our sincere thanks go to NWO.
Jul 2023
Papers 1 NeurIPS’23 accepted - Dynamic Sparsity Is Channel-Level Sparsity Learner.
Activities I am joining Google's NYC office as a researcher intern.
Jun 2023
Papers 2 ECML-PKDD’23 accepted - Robust overfitting + RDebiased Sparse Training.
Apr 2023
Papers 1 ICML’23 accepted - Large Kernel Distillation.
Nov 2022
Papers 1 AAAI’23 accepted - Lottery Pools.
Papers 1 LoG’22 accepted and Best Paper Award - Untrained GNNs Tickets.
Sep 2022
Activities CMU invited talk - Model/supervision Efficency at Xu Lab, Carnegie Mellon University.
May 2022
Papers 1 UAI’22 accepted - Sup-tickets sparse training.
Mar 2022
Papers 1 IDA’22 accepted - my first conference/symposium paper during the first year of my PhD.

Selected Publications

  1. NeurIPS 2025
    AlphaDecay: Module-wise Weight Decay for Heavy-Tailed Balancing in LLMs
    He, Di; Jaiswal, Ajay; Tu, Songjun; Shen, Li; Yuan, Ganzhao; Liu, Shiwei; and Yin, Lu† († corresponding author)
    HTMLCode
  2. NeurIPS 2025
    The Curse of Depth in Large Language Models
    Sun, Wenfang; Song, Xinyuan; Li, Pengxiang; Yin, Lu; Zheng, Yefeng; and Liu, Shiwei
    HTMLCode
  3. NeurIPS 2025
    GPAS: Accelerating Convergence of LLM Pretraining via Gradient-Preserving Activation Scaling
    Chen, Tianhao; Xu, Xin; Liu, Zijing; Li, Pengxiang; Song, Xinyuan; Jaiswal, Ajay Kumar; Zhang, Fan; Hu, Jishan; Wang, Yang; Chen, Hao; Yin, Lu† and Yang, Can († corresponding author)
    HTMLCode
  4. ICLR 2025
    Mix-LN: Unleashing the Power of Deeper Layers by Combining Pre-LN and Post-LN
    Li, Pengxiang*; Yin, Lu*; and Liu, Shiwei (* equal contribution)
    HTMLCode
  5. ICLR 2025
    Sebra: DeBiasing through Self-Guided Bias Ranking
    Kappiyath, Adarsh; Chaudhuri, Abhra; Jaiswal, Ajay; Liu, Ziquan; Li, Yunpeng; Zhu, Xiatian; and Yin, Lu† († corresponding author)
    HTMLCode
  6. ICML 2024
    Outlier Weighed Layerwise Sparsity OWL: A Missing Secret Sauce for Pruning LLMs to High Sparsity
    Yin, Lu; You, Wu; Zhenyu, Zhang; Cheng-Yu, Hsieh; Yaqing, Wang; Yiling, Jia; Mykola, Pechenizkiy; Yi, Liang; Zhangyang, Wang; and Shiwei, Liu
    HTMLCode
  7. ICML 2024
    Junk DNA Hypothesis: Pruning Small Pre-Trained Weights Irreversibly and Monotonically Impairs "Difficult" Downstream Tasks in LLMs
    Yin, Lu; Jaiswal, Ajay; Liu, Shiwei; Kundu, Souvik; and Wang, Zhangyang
    HTMLCode
  8. NeurIPS 2023
    Dynamic Sparsity is Channel-level Sparsity Learner
    Yin, Lu; Li, Gen; Fang, Meng; Shen, Li; Huang, Tianjin; Wang, Zhangyang; Menkovski, Vlado; Ma, Xiaolong; Pechenizkiy, Mykola; Liu, Shiwei; and others,
    HTMLCode
  9. AAAI 2022
    Lottery Pools: Winning More by Interpolating Tickets without Increasing Training or Inference Cost
    Yin, Lu; Liu, Shiwei; Fang, Meng; Huang, Tianjin; Menkovski, Vlado; and Pechenizkiy,
    HTMLCode