Chapel Hill, NC

Yi-Lin Sung

CS PhD student at UNC

About me


I am a fourth-year PhD student at UNC, Chapel Hill. I currently work in the MURGe-Lab, and am advised by Mohit Bansal. My research interests are in the areas of Deep Learning, Machine Learning, and Computer Vision. Recently, I am particularly interested in multi-modal learning and efficient fine-tuning, where my goal is to train large models with limited resources and deploy them to benefit human's daily life. Before joining MURGe-Lab, I also worked with Colin Raffel and Marc Niethammer.

I also spent time working as a research scientist intern in tech company in summers. In 2023 summer, I interned at Meta with Abhimanyu Dubey, Filip Radenovic and Abhishek Kadian on text-to-image generation. In 2022 summer, I worked at Microsoft with Linjie Li, Kevin Lin and Zhe Gan on VL model merging.


Jan 2024: Two papers, "ECoFLaP" and "MC-SMoE", are accepted to ICLR 2024.

Oct 2023: "An Empirical Study of Multimodal Model Merging" is accepted to EMNLP Findings 2023.

Oct 2023: Preprints of "ECoFLaP" and "MC-SMoE" are online.

July 2023: "Unified Coarse-to-Fine Alignment for Video-Text Retrieval" is accepted to ICCV 2023.

May 2023: Start the research internship at Meta.

April 2023: A preprint of "An Empirical Study of Multimodal Model Merging" is online.

Feb 2023: "Vision Transformers are Parameter-Efficient Audio-Visual Learners" is accepted to CVPR 2023.




ECoFLaP: Efficient Coarse-to-Fine Layer-Wise Pruning for Vision-Language Models

Yi-Lin Sung, Jaehong Yoon, Mohit Bansal
ICLR 2024

Merge, Then Compress: Demystify Efficient SMoE with Hints from Its Routing Policy

Pingzhi Li, Zhenyu Zhang, Prateek Yadav, Yi-Lin Sung, Yu Cheng, Mohit Bansal, Tianlong Chen
ICLR 2024

Unified Coarse-to-Fine Alignment for Video-Text Retrieval

Ziyang Wang, Yi-Lin Sung, Feng Cheng, Gedas Bertasius, Mohit Bansal
ICCV 2023

An Empirical Study of Multimodal Model Merging

Yi-Lin Sung, Linjie Li, Kevin Lin, Zhe Gan, Mohit Bansal, Lijuan Wang
EMNLP Findings 2023


Vision Transformers are Parameter-Efficient Audio-Visual Learners

Yan-Bo Lin, Yi-Lin Sung, Jie Lei, Mohit Bansal, Gedas Bertasius
CVPR 2023

LST: Ladder Side-Tuning for Parameter and Memory Efficient Transfer Learning

Yi-Lin Sung, Jaemin Cho, Mohit Bansal
NeurIPS 2022

VL-Adapter: Parameter-Efficient Transfer Learning for Vision-and-Language Tasks

Yi-Lin Sung, Jaemin Cho, Mohit Bansal
CVPR 2022


Training Neural Networks with Fixed Sparse Masks

Yi-Lin Sung*, Varun Nair*, Colin Raffel
NeurIPS, 2021.

The Maximum A Posteriori Estimation of DARTS

Yi-Lin Sung, Jun-Liang Lin, Cheng-Yao Hong, Tyng-Luh Liu
ICIP, 2021.


Video Summarization with Anchors and Multi-head Attention

Yi-Lin Sung, Cheng-Yao Hong, Yen-Chi Hsu
ICIP, 2020.


Difference-Seeking Generative Adversarial Network -- Unseen Data Generation

Yi-Lin Sung, Sung-Hsien Hsieh, Soo-Chang Pei, Chun-Shien Lu
ICLR, 2020.

Tetris Battle -- A New Environment for Single mode and Double Mode Game

Yi-Lin Sung
NeurIPS Workshop on Deep Reinforcement Learning, 2019.

Work Experience


Graduate Research Assistant@MURGe-Lab

Computer Science Department, UNC Chapel Hill
Aug. 2021 - Present

  • Advisor: Mohit Bansal

Research Intern

May. 2023 - Aug. 2023

  • Mentors: Abhimanyu Dubey, Filip Radenovic, Abhishek Kadian

Research Intern

May. 2022 - Aug. 2022

  • Mentors: Linjie Li, Kevin Lin and Zhe Gan

Summer Intern@UNC-biag

Computer Science Department, UNC Chapel Hill
Jan. 2021 - May. 2021

  • Advisor: Marc Niethammer

Teaching Assistant

Computer Science Department, UNC Chapel Hill
Jan. 2021 - May. 2021

  • Course: Deep learning.

AI Researcher

Cinnamon AI Taiwan
Mar. 2020 - Dec. 2020

Research Assistant

Institute of Information Science, Academia Sinica
Sep. 2018 - Mar. 2020

  • Advisor: Dr. Tyng-Luh Liu

Research Intern

Institute of Information Science, Academia Sinica
July 2018 - Aug. 2018

  • Advisor: Dr. Tyng-Luh Liu

Teaching Assistant

Graduate Institute of Communication Engineering, National Taiwan University
Jan. 2018 - June 2018

  • Course: Machine Learning and Having It Deep and Structured.



PhD student

The University of North Carolina at Chapel Hill - Computer Science
Jan. 2021 - Present

Master of Science

National Taiwan University - Communication Engineering
Sep. 2017 - June 2019

Bachelor of Science

National Taiwan University - Chemical Engineering
Sep. 2012 - Jan. 2017