nips nips2013 nips2013-98 nips2013-98-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Alessandro Perina, Nebojsa Jojic, Manuele Bicego, Andrzej Truski
Abstract: In text analysis documents are often represented as disorganized bags of words; models of such count features are typically based on mixing a small number of topics [1, 2]. Recently, it has been observed that for many text corpora documents evolve into one another in a smooth way, with some features dropping and new ones being introduced. The counting grid [3] models this spatial metaphor literally: it is a grid of word distributions learned in such a way that a document’s own distribution of features can be modeled as the sum of the histograms found in a window into the grid. The major drawback of this method is that it is essentially a mixture and all the content must be generated by a single contiguous area on the grid. This may be problematic especially for lower dimensional grids. In this paper, we overcome this issue by introducing the Componential Counting Grid which brings the componential nature of topic models to the basic counting grid. We evaluated our approach on document classification and multimodal retrieval obtaining state of the art results on standard benchmarks. 1
[1] Blei, D., Ng, A., Jordan, M.: Latent dirichlet allocation. Journal of machine Learning Research 3 (2003) 993–1022
[2] Reisinger, J., Waters, A., Silverthorn, B., Mooney, R.J.: Spherical topic models. In: ICML ’10: Proceedings of the 27th international conference on Machine learning. (2010)
[3] Jojic, N., Perina, A.: Multidimensional counting grids: Inferring word order from disordered bags of words. In: Proceedings of conference on Uncertainty in artificial intelligence (UAI). (2011) 547–556
[4] Hofmann, T.: Unsupervised learning by probabilistic latent semantic analysis. Machine Learning Journal 42 (2001) 177–196
[5] Blei, D.M., Lafferty, J.D.: Correlated topic models. In: NIPS. (2005)
[6] Banerjee, A., Basu, S.: Topic models over text streams: a study of batch and online unsupervised learning. In: In Proc. 7th SIAM Intl. Conf. on Data Mining. (2007)
[7] Jia, Y., Salzmann, M., Darrell, T.: Learning cross-modality similarity for multinomial data. In: Proceedings of the 2011 International Conference on Computer Vision. ICCV ’11, Washington, DC, USA, IEEE Computer Society (2011) 2407–2414
[8] Neal, R.M., Hinton, G.E.: A view of the em algorithm that justifies incremental, sparse, and other variants. Learning in graphical models (1999) 355–368
[9] Asuncion, A., Welling, M., Smyth, P., Teh, Y.W.: On smoothing and inference for topic models. In: In Proceedings of Uncertainty in Artificial Intelligence. (2009)
[10] Minka, T.P.: Estimating a Dirichlet distribution. Technical report, Microsoft Research (2012)
[11] Frey, B.J., Jojic, N.: Transformation-invariant clustering using the em algorithm. IEEE Trans. Pattern Anal. Mach. Intell. 25 (2003) 1–17
[12] Dunson, D.B., Park, J.H.: Kernel stick-breaking processes. Biometrika 95 (2008) 307–323
[13] Perina, A., Cristani, M., Castellani, U., Murino, V., Jojic, N.: Free energy score spaces: Using generative information in discriminative classifiers. IEEE Trans. Pattern Anal. Mach. Intell. 34 (2012) 1249–1262
[14] Raina, R., Shen, Y., Ng, A.Y., Mccallum, A.: Classification with hybrid generative/discriminative models. In: In Advances in Neural Information Processing Systems 16, MIT Press (2003)
[15] Jebara, T., Kondor, R., Howard, A.: Probability product kernels. J. Mach. Learn. Res. 5 (2004) 819–844
[16] Bosch, A., Zisserman, A., Mu˜ oz, X.: Scene classification using a hybrid generative/discriminative n approach. IEEE Trans. Pattern Anal. Mach. Intell. 30 (2008) 712–727
[17] Bicego, M., Lovato, P., Perina, A., Fasoli, M., Delledonne, M., Pezzotti, M., Polverari, A., Murino, V.: Investigating topic models’ capabilities in expression microarray data classification. IEEE/ACM Trans. Comput. Biology Bioinform. 9 (2012) 1831–1836
[18] Perina, A., Jojic, N.: Image analysis by counting on a grid. In: Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR). (2011) 1985–1992
[19] Blei, D.M., Jordan, M.I.: Modeling annotated data. In: Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval. SIGIR ’03 (2003) 127–134
[20] Thomas, J., Cook, K.: Illuminating the Path: The Research and Development Agenda for Visual Analytics. IEEE Press (2005)
[21] Tenenbaum, J.B., de Silva, V., Langford, J.C.: A Global Geometric Framework for Nonlinear Dimensionality Reduction. Science 290 (2000) 2319–2323
[22] Globerson, A., Chechik, G., Pereira, F., Tishby, N.: Euclidean embedding of co-occurrence data. Journal of Machine Learning Research 8 (2007) 2265–2295
[23] Roweis, S.T., Saul, L.K.: Nonlinear dimensionality reduction by locally linear embedding. SCIENCE 290 (2000) 2323–2326 9