New conference paper on Transfer Learning

Why transfer learning?

Though the performance of machine learning models (especially deep learning) has been improved significantly over the past few years, there still exist several challenges that need to be addressed in order for ML to be applied in practice at scale. One of the most serious problems is the poor performance when applying pre-trained machine learning models to real-world problems. We, human being, are born with the ability to apply acquired knowledge and skills to new problems/tasks. For example, once you learned to ride a bicycle, you can ride a motorbike with minimum efforts. But, it is not so simple in case of machine learning. Retraining ML models for new tasks requires additional training data that is not available in many cases. Transfer learning is a potential approach to deal with this problem. 

What is transfer learning?

Transfer learning refers to a set of machine learning techniques that allows one to transfer knowledge learned from an auxiliary (or source) domain to help the training task on a target domain. There are three typical types of transfer learning, namely, instance transfer, feature transfer, and network transfer. In instance-based transfer, instances (or data) from the source domain is used in the training at the target domain. Go further, feature-based transfer aims to transfer features which are learned from the data. Different from the raw data, features are more compact and contain more meaningful information that can help improve the performance in many cases. Both instance-based and feature-based approaches assume that the source data is available at the target data. However, due to increasing concerns over user privacy and security, it become more and more difficult to share data between domains. Network transfer can solve this problem by transferring part of the model such as weights. 

Our contribution 

Recommender system is an essential component in many practical applications and services. Recently, significant progress has been made to improve performance of recommender system utilizing deep learning. However, single-domain recommender system suffers from the long-standing data sparsity problem. Transfer learning is a potential approach to deal with the data sparsity problem in recommender system. In this paper, we investigate the transferability of deep neural networks for recommender system. It is found that the neural network responsible for learning the user-item interaction function can be transferred to the target domain, resulting in significant improvement in recommendation performance.

Ref: Duc Nguyen et al., "On the Transferability of Deep Networks for Recommender Systems", DEIM Forum, March. 2020.