Projections of multidimensional relations for self-supervised representation learning with multi-augmentation

Author: Levan Tsinadze
Keywords: self-supervised learning, representation learning, relational space, multi-dimensional relations
Annotation:

Self-supervised representation learning has shown significant improvements on downstream tasks for various datasets in different fields (NLP, Computer Vision, Graph Neural Networks, etc). The latest achievements with contrastive learning in computer vision involves different data augmentation techniques along with models with ability to preserve representations on augmented data. We consider multiply augmentations as a multi-dimensional relation among augmented (and original) data and train a siamese model to preserve this relation on the representation level. Our method has shown significant results on various datasets and the capacity of improvements with additional augmentations encoded as additional dimensions in our relational structure with little changes in code