Jaewoo Lee | UNC Chapel Hill

About Me

Thanks for visiting my website!😄
I am an incoming Ph.D. student at UNC Chapel Hill, MURGe-Lab (Advisor: Prof. Mohit Bansal).
I received a M.S. degree in Artificial Intelligence at KAIST in Aug 2024 (Prof. Sung Ju Hwang) and a B.S. degree in Electrical Engineering at KAIST in Feb 2023.

Research Interests

My goal is to promote AI democratization by developing resource-efficient multimodal models that make AI more accessible and interactive for anyone.

Multimodal: Vision-Language, Audio-Visual, Interleaved modalities
Efficiency in data & algorithms: Data selection, Continual learning, Curriculum learning
Efficiency in systems: Retrieval-Augmented Generation, Information retrieval
Efficiency in architectures: Mixture of Experts, Token reduction, Model Compression
Interpretability: Interpretable LLM and MLLM, Attribution methods

📝 Publications

(* denotes the equal contribution and ^ denotes the equal advising)

Preprint

TAMP: Token-Adaptive Layerwise Pruning in Multimodal Large Language Models

Jaewoo Lee, Keyang Xuan, Chanakya Ekbote, Sandeep Polisetty, Yi R. (May) Fung, Paul Pu Liang

Preprint.

PDF Code

Preprint

Unified Multi-Modal Interleaved Document Representation for Information Retrieval

Jaewoo Lee*, Joonho Ko*, Jinheon Baek*, Soyeong Jeong, Sung Ju Hwang

Preprint.

PDF

EMNLP

Concept-skill Transferability-based Data Selection for Large Vision-Language Models

Jaewoo Lee, Boyang Li^, Sung Ju Hwang^

Conference on Empirical Methods in Natural Language Processing, Main conference Long paper (EMNLP), 2024.

PDF Code

ICML

STELLA: Continual Audio-Video Pre-training with Spatio-Temporal Localized Alignment

Jaewoo Lee*, Jaehong Yoon*, Wongjae Kim, Yunji Kim, Sung Ju Hwang

International Conference on Machine Learning (ICML), 2024.

PDF Code Project Page

ICSV

Sound-based drone fault classification using multitask learning

Wonjun Yi, Jung-Woo Choi, Jaewoo Lee

International Congress on Sound and Vibration (ICSV), 2023.

PDF Dataset

📖 Educations

2023.03 - 2024.08, M.S. in Artificial Intelligence. KAIST.
- Thesis: Efficient Training Techniques for Multimodal Learning (PDF)
- A half-year early graduation.
2020.03 - 2023.02, B.S. in Electrical Engineering. KAIST.
- Summa Cum Laude
- One-year early graduation.

💻 Research Experiences

2024.08 - 2024.03, AI Researcher, DeepAuto, South Korea.
2023.03 - 2024.08, Master degree Student, MLAI Lab, KAIST, South Korea.
2021.09 - 2022.06, Research Intern, Smart Sound Systems Lab, KAIST, South Korea.
2021.06 - 2021.08, Research Intern, Urban Robotics Lab, KAIST, South Korea.

🎖 Honors and Awards

KAIST Summa Cum Laude Award, Feb. 2023
National Scholarship for Science & Engineering, Sep. 2022 - Feb. 2023
Encouragement Award for the Undergraduate Research Program, Aug. 2022
KAIST Dean’s List (College of Engineering), Aug. 2022
KAIST Dean’s List (School of Freshman), Aug. 2020