VISTEC - Vidyasirimedhi Institute of Science and Technology₁
Rayong, Thailand
CVPR 2021 (Oral)
Segmentation results trained with only one label
While GANs have shown success in realistic image generation, the idea of using GANs for other tasks unrelated to synthesis is underexplored. Do GANs learn meaningful structural parts of objects during their attempt to reproduce those objects? In this work, we test this hypothesis and propose a simple and effective approach based on GANs for semantic part segmentation that requires as few as one label example along with an unlabeled dataset. Our key idea is to leverage a trained GAN to extract pixel-wise representation from the input image and use it as feature vectors for a segmentation network. Our experiments demonstrate that GANs representation is "readily discriminative" and produces surprisingly good results that are comparable to those from supervised baselines trained with significantly more labels. We believe this novel repurposing of GANs underlies a new class of unsupervised representation learning that is applicable to many other tasks.
1-shot
5-shot
10-shot
trained on a dataset auto-generated by our method
1 manual label
5 manual labels
10 manual labels
Input
1 manual label
5 manual labels
10 manual labels
Input
1 manual label
5 manual labels
10 manual labels
BibTex (Extended Version)
@article{article, author = {Rewatbowornwong, Pitchaporn and Tritrong, Nontawat and Suwajanakorn, Supasorn}, year = {2022}, month = {08}, pages = {1-12}, title = {Repurposing GANs for One-shot Semantic Part Segmentation}, volume = {PP}, journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence}, doi = {10.1109/TPAMI.2022.3201285} }
BibTex (Conference Version)
@inproceedings{Tritrong2021RepurposeGANs, author = {Tritrong, Nontawat and Rewatbowornwong, Pitchaporn and Suwajanakorn, Supasorn}, title = {Repurposing GANs for One-shot Semantic Part Segmentation}, booktitle = {IEEE Conference on Computer Vision and Pattern Recognition (CVPR)}, year = {2021}, }
NeX: Real-time View Synthesis with Neural Basis Expansion