Jaewoo Lee | KAIST

About Me

Thanks for visiting my website!😄
I received a M.S. degree in Artificial Intelligence at KAIST, MLAI Lab (Advisor: Prof.Sung Ju Hwang) in Aug 2024, and a B.S. degree in Electrical Engineering at KAIST in Feb 2023.
I’m currently working as an AI researcher at DeepAuto.

Research Interests

My goal is to promote AI democratization by developing resource-efficient multimodal models that make AI more accessible and interactive for anyone.

Multimodal (Vision-Language, Audio-Visual, Interleaved modalities):
I am intrigued by uncovering interactions between modalities and, based on these insights, developing models that understand various multimodal knowledge, leading to more accessible and interactive AI.
Efficiency in data & algorithms (Data selection, Continual learning, Curriculum learning):
To further enhance AI accessibility, I am keen on designing algorithms that minimize the training costs of models. I am currently interested in combining data curation with continual or curriculum learning (Empirical Study).
Efficiency in systems (Retrieval-Augmented Generation, Information retrieval):
Storing enormous and growing world knowledge in models is challenging. I aim to decouple knowledge from task-solving skills.
Efficiency in architectures (Mixture of Experts, Token reduction):
I am passionate about designing architectures for efficient training and deployment of models for complex tasks and multimodalities.
Interpretability (Interpretable LLM and MLLM, Attribution methods):
Effective frameworks stem from understanding model behaviors and data. I am interested in exploring how modalities interact during processing.

📝 Publications

(* denotes the equal contribution and ^ denotes the equal advising)

Preprint

Unified Multi-Modal Interleaved Document Representation for Information Retrieval

Jaewoo Lee*, Joonho Ko*, Jinheon Baek*, Soyeong Jeong, Sung Ju Hwang

Preprint.

PDF

EMNLP

Concept-skill Transferability-based Data Selection for Large Vision-Language Models

Jaewoo Lee, Boyang Li^, Sung Ju Hwang^

Conference on Empirical Methods in Natural Language Processing, Main conference Long paper (EMNLP), 2024.

PDF Code

ICML

STELLA: Continual Audio-Video Pre-training with Spatio-Temporal Localized Alignment

Jaewoo Lee*, Jaehong Yoon*, Wongjae Kim, Yunji Kim, Sung Ju Hwang

International Conference on Machine Learning (ICML), 2024.

PDF Code Project Page

ICSV

Sound-based drone fault classification using multitask learning

Wonjun Yi, Jung-Woo Choi, Jaewoo Lee

International Congress on Sound and Vibration (ICSV), 2023.

PDF Dataset

📖 Educations

2023.03 - 2024.08, M.S. in Artificial Intelligence. KAIST.
- Thesis: Efficient Training Techniques for Multimodal Learning (PDF)
- A half-year early graduation.
2020.03 - 2023.02, B.S. in Electrical Engineering. KAIST.
- Summa Cum Laude
- One-year early graduation.

💻 Research Experiences

2023.08 - current, AI Researcher, DeepAuto, South Korea.
2023.03 - 2024.08, Master degree Student, MLAI Lab, KAIST, South Korea.
2021.09 - 2022.06, Research Intern, Smart Sound Systems Lab, KAIST, South Korea.
2021.06 - 2021.08, Research Intern, Urban Robotics Lab, KAIST, South Korea.

🎖 Honors and Awards

KAIST Summa Cum Laude Award, Feb. 2023
National Scholarship for Science & Engineering, Sep. 2022 - Feb. 2023
Encouragement Award for the Undergraduate Research Program, Aug. 2022
KAIST Dean’s List (College of Engineering), Aug. 2022
KAIST Dean’s List (School of Freshman), Aug. 2020