Chapel Hill, NC
ylsung@cs.unc.edu

Yi-Lin Sung

CS PhD student at UNC

About me

PROFESSIONAL PATH
I am on the job market. With a strong background in cutting-edge areas like Multimodal LLMs, LLM Self-Refinement, Efficiency (PEFT, Quantization, Model Merging), MoE, Personalization, and Text-to-Image Generation. If you're looking for someone who can drive innovation in these fields, feel free to connect with me via email!

I am a fifth-year PhD student at UNC, Chapel Hill. I currently work in the MURGe-Lab, and am advised by Mohit Bansal. My research interests are in the areas of Deep Learning, Machine Learning, and Computer Vision. Recently, I am particularly interested in multi-modal learning and efficient fine-tuning, where my goal is to train large models with limited resources and deploy them to benefit human's daily life. Before joining MURGe-Lab, I also worked with Colin Raffel and Marc Niethammer.

I also spent time working as a research scientist intern in tech company in summers. In 2024 Summer, I interned at Google with Otilia Stretcu on VLM reasoning. In 2023 summer, I interned at Meta with Abhimanyu Dubey, Filip Radenovic and Abhishek Kadian on text-to-image generation. In 2022 summer, I worked at Microsoft with Linjie Li, Kevin Lin and Zhe Gan on VL model merging.

News

Nov 2024: "DAM" is accepted to WACV 2025.

Oct 2024: "SELMA" is accepted to NeurIPS 2024.

Oct 2024: Preprint of "Glider" is online.

Jan 2024: Two papers, "ECoFLaP" and "MC-SMoE", are accepted to ICLR 2024.

Oct 2023: "An Empirical Study of Multimodal Model Merging" is accepted to EMNLP Findings 2023.

Oct 2023: Preprints of "ECoFLaP" and "MC-SMoE" are online.

July 2023: "Unified Coarse-to-Fine Alignment for Video-Text Retrieval" is accepted to ICCV 2023.

May 2023: Start the research internship at Meta.

April 2023: A preprint of "An Empirical Study of Multimodal Model Merging" is online.

Feb 2023: "Vision Transformers are Parameter-Efficient Audio-Visual Learners" is accepted to CVPR 2023.

Publications

MY WORK

2024

Glider: Global and Local Instruction-Driven Expert Router

Pingzhi Li, Prateek Yadav, Jaehong Yoon, Jie Peng, Yi-Lin Sung, Mohit Bansal, Tianlong Chen
arXiv:2410.07172

DAM: Dynamic Adapter Merging for Continual Video QA Learning

Feng Cheng, Ziyang Wang, Yi-Lin Sung, Yan-Bo Lin, Mohit Bansal, Gedas Bertasius
WACV 2025

SELMA: Learning and Merging Skill-Specific Text-to-Image Experts with Auto-Generated Data

Jialu Li, Jaemin Cho, Yi-Lin Sung, Jaehong Yoon, Mohit Bansal
NeurIPS 2024

2023

ECoFLaP: Efficient Coarse-to-Fine Layer-Wise Pruning for Vision-Language Models

Yi-Lin Sung, Jaehong Yoon, Mohit Bansal
ICLR 2024

Merge, Then Compress: Demystify Efficient SMoE with Hints from Its Routing Policy

Pingzhi Li, Zhenyu Zhang, Prateek Yadav, Yi-Lin Sung, Yu Cheng, Mohit Bansal, Tianlong Chen
ICLR 2024

Unified Coarse-to-Fine Alignment for Video-Text Retrieval

Ziyang Wang, Yi-Lin Sung, Feng Cheng, Gedas Bertasius, Mohit Bansal
ICCV 2023

An Empirical Study of Multimodal Model Merging

Yi-Lin Sung, Linjie Li, Kevin Lin, Zhe Gan, Mohit Bansal, Lijuan Wang
EMNLP Findings 2023

2022

Vision Transformers are Parameter-Efficient Audio-Visual Learners

Yan-Bo Lin, Yi-Lin Sung, Jie Lei, Mohit Bansal, Gedas Bertasius
CVPR 2023

LST: Ladder Side-Tuning for Parameter and Memory Efficient Transfer Learning

Yi-Lin Sung, Jaemin Cho, Mohit Bansal
NeurIPS 2022

VL-Adapter: Parameter-Efficient Transfer Learning for Vision-and-Language Tasks

Yi-Lin Sung, Jaemin Cho, Mohit Bansal
CVPR 2022

2021

Training Neural Networks with Fixed Sparse Masks

Yi-Lin Sung*, Varun Nair*, Colin Raffel
NeurIPS, 2021.

The Maximum A Posteriori Estimation of DARTS

Yi-Lin Sung, Jun-Liang Lin, Cheng-Yao Hong, Tyng-Luh Liu
ICIP, 2021.

2020

Video Summarization with Anchors and Multi-head Attention

Yi-Lin Sung, Cheng-Yao Hong, Yen-Chi Hsu
ICIP, 2020.

2019

Difference-Seeking Generative Adversarial Network -- Unseen Data Generation

Yi-Lin Sung, Sung-Hsien Hsieh, Soo-Chang Pei, Chun-Shien Lu
ICLR, 2020.

Tetris Battle -- A New Environment for Single mode and Double Mode Game

Yi-Lin Sung
NeurIPS Workshop on Deep Reinforcement Learning, 2019.

Work Experience

PREVIOUS JOBS

Graduate Research Assistant@MURGe-Lab

Computer Science Department, UNC Chapel Hill
Aug. 2021 - Present

  • Advisor: Mohit Bansal

Research Intern

Google
May. 2024 - Aug. 2024

  • Mentors: Otilia Stretcu, Chun-Ta Lu, Alan Luo, Ranjay Krishna

Research Intern

Meta
May. 2023 - Aug. 2023

  • Mentors: Abhimanyu Dubey, Filip Radenovic, Abhishek Kadian

Research Intern

Microsoft
May. 2022 - Aug. 2022

  • Mentors: Linjie Li, Kevin Lin and Zhe Gan

Summer Intern@UNC-biag

Computer Science Department, UNC Chapel Hill
Jan. 2021 - May. 2021

  • Advisor: Marc Niethammer

Teaching Assistant

Computer Science Department, UNC Chapel Hill
Jan. 2021 - May. 2021

  • Course: Deep learning.

AI Researcher

Cinnamon AI Taiwan
Mar. 2020 - Dec. 2020

Research Assistant

Institute of Information Science, Academia Sinica
Sep. 2018 - Mar. 2020

  • Advisor: Dr. Tyng-Luh Liu

Research Intern

Institute of Information Science, Academia Sinica
July 2018 - Aug. 2018

  • Advisor: Dr. Tyng-Luh Liu

Teaching Assistant

Graduate Institute of Communication Engineering, National Taiwan University
Jan. 2018 - June 2018

  • Course: Machine Learning and Having It Deep and Structured.

Education

ACADEMIC CAREER

PhD student

The University of North Carolina at Chapel Hill - Computer Science
Jan. 2021 - Present

Master of Science

National Taiwan University - Communication Engineering
Sep. 2017 - June 2019

Bachelor of Science

National Taiwan University - Chemical Engineering
Sep. 2012 - Jan. 2017