Correlation Based Semantic Transfer with Application to Domain Adaptation

In this paper, we introduce a multifaceted contribution. First, we propose a definition, specific to convolutional neural networks (CNN’s), for the notion of semantically similar features. Second, using this definition, we introduce a new loss, which semantically transfers features from one domain to another domain, where the features of both domains are learnt by two CNN’s. Our transfer loss, named CBT, constrains the responses of the corresponding convolutional kernels of the two CNN’s to correlate in similar contexts. When the features of the source domain are discriminative, with respect to a classifier, CBT helps to maintain in the target domain, the semantics of the feature space imposed by that classifier. Third, we show that CBT can be used for unsupervised domain adaptation (UDA) by proposing a novel approach for this problem.