Yehjin Shin

Ph.D. Student at KAIST

I am a Ph.D. student at Korea Advanced Institute of Science and Technology (KAIST), advised by Professor Noseong Park. Having completed my Master’s at Yonsei University, I am driven by a deep passion for applying deep learning techniques to temporal and tabular data. My current research focuses on sequential recommendation systems, exploring temporal patterns in user behavior to enhance predictive models. Recently, I’ve also developed a strong interest in sequential architectures like Transformers and Mamba, aiming to better understand and improve how these models handle complex sequences. Here is a full Curriculum Vitae.

🔉 News

May, 2024	One paper has been accepted at ICML 2024 😆
Dec, 2023	One paper has been accepted at AAAI 2024 😆 Hope to see you in Vancouver, Canada 🇨🇦
Oct, 2023	One paper has been accepted at WSDM 2024 ☺️ Hope to see you in Mérida, Mexico 🇲🇽

📑 Selected Publications

2024

ICML

Polynomial-based Self-Attention for Table Representation learning

Jayoung Kim, Yehjin Shin, Jeongwhan Choi, and 2 more authors

In International Conference on Machine Learning, 2024

Abs PDF

Structured data, which constitutes a significant portion of existing data types, has been a long-standing research topic in the field of machine learning. Various representation learning methods for tabular data have been proposed, ranging from encoder-decoder structures to Transformers. Among these, Transformer-based methods have achieved state-of-the-art performance not only in tabular data but also in various other fields, including computer vision and natural language processing. However, recent studies have revealed that self-attention, a key component of Transformers, can lead to an oversmoothing issue. We show that Transformers for tabular data also face this problem, and to address the problem, we propose a novel matrix polynomial-based self-attention layer as a substitute for the original self-attention layer, which enhances model scalability. In our experiments with three representative table learning models equipped with our proposed layer, we illustrate that the layer effectively mitigates the oversmoothing problem and enhances the representation performance of the existing methods, outperforming the state-of-the-art table representation methods.
AAAI

An Attentive Inductive Bias for Sequential Recommendation Beyond the Self-Attention

Yehjin Shin, Jeongwhan Choi, Hyowon Wi, and 1 more author

In Proceedings of the AAAI Conference on Artificial Intelligence, 2024

Abs arXiv PDF Code

Sequential recommendation (SR) models based on Transformers have achieved remarkable successes. The self-attention mechanism of Transformers for computer vision and natural language processing suffers from the oversmoothing problem, i.e., hidden representations becoming similar to tokens. In the SR domain, we, for the first time, show that the same problem occurs. We present pioneering investigations that reveal the low-pass filtering nature of self-attention in the SR, which causes oversmoothing. To this end, we propose a novel method called Beyond Self-Attention for Sequential Recommendation (BSARec), which leverages the Fourier transform to i) inject an inductive bias by considering fine-grained sequential patterns and ii) integrate low and high-frequency information to mitigate oversmoothing. Our discovery shows significant advancements in the SR domain and is expected to bridge the gap for existing Transformer-based SR models. We test our proposed approach through extensive experiments on 6 benchmark datasets. The experimental results demonstrate that our model outperforms 7 baseline methods in terms of recommendation performance.