iccv iccv2013 iccv2013-147 iccv2013-147-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Lukas Bossard, Matthieu Guillaumin, Luc Van_Gool
Abstract: The task of recognizing events in photo collections is central for automatically organizing images. It is also very challenging, because of the ambiguity of photos across different event classes and because many photos do not convey enough relevant information. Unfortunately, the field still lacks standard evaluation data sets to allow comparison of different approaches. In this paper, we introduce and release a novel data set of personal photo collections containing more than 61,000 images in 807 collections, annotated with 14 diverse social event classes. Casting collections as sequential data, we build upon recent and state-of-the-art work in event recognition in videos to propose a latent sub-event approach for event recognition in photo collections. However, photos in collections are sparsely sampled over time and come in bursts from which transpires the importance of specific moments for the photographers. Thus, we adapt a discriminative hidden Markov model to allow the transitions between states to be a function of the time gap between consecutive images, which we coin as Stopwatch Hidden Markov model (SHMM). In our experiments, we show that our proposed model outperforms approaches based only on feature pooling or a classical hidden Markov model. With an average accuracy of 56%, we also highlight the difficulty of the data set and the need for future advances in event recognition in photo collections.
[1] K. Barnard, P. Duygulu, N. de Freitas, D. Forsyth, D. Blei, and M. Jordan. Matching words and pictures. JMLR, 2003.
[2] H. Bay, T. Tuytelaars, and L. Van Gool. SURF: Speeded Up Robust Features. In ICCV, 2006.
[3] T. Berg, A. Berg, J. Edwards, M. Mair, R. White, Y. Teh, E. Learned-Miller, and D. Forsyth. Names and faces in the news. In CVPR, 2004.
[4] M. Bladt and M. Srensen. Statistical inference for discretely observed markov jump processes. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 2005.
[5] M. Boutell. Bayesian fusion of camera metadata cues in semantic scene classification. In CVPR, 2004.
[6] L. Cao, J. Luo, and T. S. Huang. Annotating photo collections by label propagation according to multiple similarity cues. In MM, 2008.
[7] L. Cao, J. Luo, H. Kautz, and T. S. Huang. Image annotation within the context of personal photo collections using hierarchical event and scene models. IEEE Transactions on Multimedia, 11, 2009.
[8] M. Cooper, J. Foote, A. Girgensohn, and L. Wilcox. Temporal event clustering for digital photo collections. ACM Transactions on Multimedia Computing, Communications, and Applications, 1, 2005.
[9] M. Dantone, J. Gall, G. Fanelli, and L. Van Gool. Real-time facial feature detection using conditional regression forests. In CVPR, 2012.
[10] A. Gaidon, Z. Harchaoui, and C. Schmid. Recognizing ac-
[11]
[12]
[13]
[14]
[15]
[16]
[17]
[18]
[19]
[20]
[21]
[22] tivities with cluster-trees of tracklets. In BMVC, 2012. M. Guillaumin, T. Mensink, J. Verbeek, and C. Schmid. Face recognition from caption-based supervision. IJCV, 2012. H. Izadinia and M. Shah. Recognizing complex events using large margin joint low-level event model. In ECCV, 2012. D. E. King. Dlib-ml: A machine learning toolkit. JMLR, 10, 2009. J. Lafferty, A. McCallum, and F. Pereira. Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In ICML, 2001 . S. Lazebnik, C. Schmid, and J. Ponce. Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories. In CVPR, 2006. L. Li-Jia and L. Fei-Fei. What, where and who? Classifying events by scene and object recognition. In ICCV, 2007. M. Marszalek, I. Laptev, and C. Schmid. Actions in context. In CVPR, 2009. R. Mattivi, J. Uijlings, F. G. De Natale, and N. Sebe. Exploitation of time constraints for (sub-)event recognition. In J-MRE, 2011. J. McAuley and J. Leskovec. Image labeling on a network: using social-network metadata for image classification. In ECCV, 2012. S. Nowozin and C. H. Lampert. Structured Learning and Prediction in Computer Vision. Foundations and Trends in Computer Graphics and Vision, 6(3-4): 185–365, 2011. P. Over, G. M. Awad, J. Fiscus, B. Antonishek, M. Michel, A. F. Smeaton, W. Kraaij, and G. Qu´ enot. Trecvid 2011–an overview of the goals, tasks, data, evaluation mechanisms, and metrics. 2011. J. Philbin, O. Chum, M. Isard, J. Sivic, and A. Zisser-
[23]
[24]
[25]
[26]
[27]
[28]
[29]
[30] man. Object retrieval with large vocabularies and fast spatial matching. In CVPR, 2007. A. Quattoni and A. Torralba. Recognizing indoor scenes. In CVPR, 2009. A. Quattoni, S. Wang, L.-P. Morency, M. Collins, and T. Darrell. Hidden conditional random fields. PAMI, 29, 2007. K. Tang, L. Fei-Fei, and D. Koller. Learning latent temporal structure for complex event detection. In CVPR, 2012. S.-F. Tsai, L. Cao, F. Tang, and T. S. Huang. Compositional object pattern: a new model for album event recognition. In MM, 2011. T. Tuytelaars, C. Lampert, M. Blaschko, and W. Buntine. Unsupervised object discovery: A comparison. IJCV, 2009. C.-N. J. Yu and T. Joachims. Learning structural SVMs with latent variables. In ICML, 2009. J. Yuan, J. Luo, H. Kautz, and Y. Wu. Mining GPS traces and visual words for event classification. In MIR, 2008. A. L. Yuille and A. Rangarajan. The Concave-Convex Procedure. Neural Computation, 15, 2003. 1200