nips nips2013 nips2013-335 nips2013-335-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Marcus Rohrbach, Sandra Ebert, Bernt Schiele
Abstract: Category models for objects or activities typically rely on supervised learning requiring sufficiently large training sets. Transferring knowledge from known categories to novel classes with no or only a few labels is far less researched even though it is a common scenario. In this work, we extend transfer learning with semi-supervised learning to exploit unlabeled instances of (novel) categories with no or only a few labeled instances. Our proposed approach Propagated Semantic Transfer combines three techniques. First, we transfer information from known to novel categories by incorporating external knowledge, such as linguistic or expertspecified information, e.g., by a mid-level layer of semantic attributes. Second, we exploit the manifold structure of novel classes. More specifically we adapt a graph-based learning algorithm – so far only used for semi-supervised learning – to zero-shot and few-shot learning. Third, we improve the local neighborhood in such graph structures by replacing the raw feature-based representation with a mid-level object- or attribute-based representation. We evaluate our approach on three challenging datasets in two different applications, namely on Animals with Attributes and ImageNet for image classification and on MPII Composites for activity recognition. Our approach consistently outperforms state-of-the-art transfer and semi-supervised approaches on all datasets. 1
[1] E. Bart & S. Ullman. Single-example learning of novel classes using representation by similarity. In BMVC, 2005.
[2] A. Berg, J. Deng, & L. Fei-Fei. ILSVRC 2010. www.image-net.org/challenges/LSVRC/2010/, 2010.
[3] U. Blanke & B. Schiele. Remember and transfer what you have learned - recognizing composite activities based on activity spotting. In ISWC, 2010.
[4] J. Choi, M. Rastegari, A. Farhadi, & L. S. Davis. Adding Unlabeled Samples to Categories by Learned Attributes. In CVPR, 2013.
[5] S. Ebert, D. Larlus, & B. Schiele. Extracting Structures in Image Collections for Object Recognition. In ECCV, 2010.
[6] R. Farrell, O. Oza, V. Morariu, T. Darrell, & L. S. Davis. Birdlets: Subordinate categorization using volumetric primitives and pose-normalized appearance. In ICCV, 2011.
[7] R. Fergus, Y. Weiss, & A. Torralba. Semi-supervised learning in gigantic image collections. NIPS 2009.
[8] M. Fink. Object classification from a single example utilizing class relevance pseudo-metrics. In NIPS, 2004.
[9] Y. Fu, T. M. Hospedales, T. Xiang, & S. Gong. Learning multi-modal latent attributes. TPAMI, PP(99), 2013.
[10] P. Kankuekul, A. Kawewong, S. Tangruamsub, & O. Hasegawa. Online Incremental Attribute-based Zero-shot Learning. In CVPR, 2012.
[11] C. Lampert, H. Nickisch, & S. Harmeling. Attribute-based classification for zero-shot learning of object categories. TPAMI, PP(99), 2013.
[12] H.-T. Lin, C.-J. Lin, & R. C. Weng. A note on platt’s probabilistic outputs for support vector machines. Machine Learning, 2007.
[13] J. Liu, B. Kuipers, & S. Savarese. Recognizing human actions by attributes. In CVPR, 2011.
[14] U. Luxburg. A tutorial on spectral clustering. Stat Comput, 17(4):395–416, 2007.
[15] M. Maier, U. V. Luxburg, & M. Hein. Influence of graph construction on graph-based clustering measures. In NIPS, 2008.
[16] T. Mensink, J. Verbeek, F. Perronnin, & G. Csurka. Metric Learning for Large Scale Image Classification: Generalizing to New Classes at Near-Zero Cost. In ECCV, 2012.
[17] Y. Moses, S. Ullman, & S. Edelman. Generalization to novel images in upright and inverted faces. Perception, 25:443–461, 1996.
[18] A. Y. Ng, M. I. Jordan, & Y. Weiss. On spectral clustering: Analysis and an algorithm. In NIPS, 2002.
[19] M. Palatucci, D. Pomerleau, G. Hinton, & T. Mitchell. Zero-shot learning with semantic output codes. In NIPS, 2009.
[20] S. J. Pan & Q. Yang. A survey on transfer learning. TKDE, 22:1345–59, 2010.
[21] R. Raina, A. Battle, H. Lee, B. Packer, & A. Ng. Self-taught learning: Transfer learning from unlabeled data. In ICML, 2007.
[22] M. Rohrbach, M. Regneri, M. Andriluka, S. Amin, M. Pinkal, & B. Schiele. Script data for attribute-based recognition of composite activities. In ECCV, 2012.
[23] M. Rohrbach, M. Stark, & B. Schiele. Evaluating Knowledge Transfer and Zero-Shot Learning in a Large-Scale Setting. In CVPR, 2011.
[24] M. Rohrbach, M. Stark, G. Szarvas, I. Gurevych, & B. Schiele. What Helps Where – And Why? Semantic Relatedness for Knowledge Transfer. In CVPR, 2010.
[25] K. Saenko, B. Kulis, M. Fritz, & T. Darrell. Adapting visual category models to new domains. In ECCV, 2010.
[26] V. Sharmanska, N. Quadrianto, & C. H. Lampert. Augmented Attribute Representations. In ECCV, 2012.
[27] A. Shrivastava, S. Singh, & A. Gupta. Constrained Semi-Supervised Learning Using Attributes and Comparative Attributes. In ECCV, 2012.
[28] J. Sivic, B. C. Russell, A. A. Efros, A. Zisserman, & W. T. Freeman. Discovering Object Categories in Image Collections. In ICCV, 2005.
[29] S. Thrun. Is learning the n-th thing any easier than learning the first. In NIPS, 1996.
[30] A. Torralba, K. Murphy, & W. Freeman. Sharing visual features for multiclass and multiview object detection. In CVPR, 2004.
[31] D. Tran & A. Sorokin. Human activity recognition with metric learning. In ECCV, 2008.
[32] M. Weber, M. Welling, & P. Perona. Towards automatic discovery of object categories. In CVPR, 2000.
[33] D. Zhou, O. Bousquet, T. N. Lal, Jason Weston, & B. Sch¨ lkopf. Learning with Local and Global o Consistency. In NIPS, 2004.
[34] X. Zhu, Z. Ghahramani, & J. Lafferty. Semi-supervised learning using gaussian fields and harmonic functions. In ICML, 2003.
[35] A. Zweig & D. Weinshall. Exploiting object hierarchy: Combining models from different category levels. In ICCV, 2007. 9