Enhancing Recommendation Systems with Multi-head Attention and Deep Feature Interaction: The MHA_xDeepFM Model

Abstract

The accuracy of the ranking stage in recommendation systems largely hinges on feature interaction. Despite significant progress, existing algorithms and models can realize feature interaction among shallow, deep explicit, and deep implicit features, but they fail to capture the diversity of user interests effectively. To address this limitation, a Multi-head Attention (MHA) module is incorporated into the xDeepFM benchmark model. The enhanced model is referred to as MHA_xDeepFM in this paper. By interactively modeling features between users and items, the model uncovers deep interactions among features and forms combined features to enhance prediction accuracy.The proposed model first integrates the Multi-head Attention mechanism to adaptively model the correlations between input features, emphasizing the key internal relationships within the data. Subsequently, the model incorporates a compressed interaction network to learn the optimal order of combined features, leveraging deep neural networks to understand deep interactive features in complex data. Finally, the model filters and sorts the recall list based on click-through rates. Experimental results indicate that, compared to similar algorithms, MHA_xDeepFM improves AUC indicators by 2.7% and 3.2% on two different datasets.