nips nips2013 nips2013-294 nips2013-294-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Soravit Changpinyo, Kuan Liu, Fei Sha
Abstract: Measuring similarity is crucial to many learning tasks. To this end, metric learning has been the dominant paradigm. However, similarity is a richer and broader notion than what metrics entail. For example, similarity can arise from the process of aggregating the decisions of multiple latent components, where each latent component compares data in its own way by focusing on a different subset of features. In this paper, we propose Similarity Component Analysis (SCA), a probabilistic graphical model that discovers those latent components from data. In SCA, a latent component generates a local similarity value, computed with its own metric, independently of other components. The final similarity measure is then obtained by combining the local similarity values with a (noisy-)OR gate. We derive an EM-based algorithm for fitting the model parameters with similarity-annotated data from pairwise comparisons. We validate the SCA model on synthetic datasets where SCA discovers the ground-truth about the latent components. We also apply SCA to a multiway classification task and a link prediction task. For both tasks, SCA attains significantly better prediction accuracies than competing methods. Moreover, we show how SCA can be instrumental in exploratory analysis of data, where we gain insights about the data by examining patterns hidden in its latent components’ local similarity values. 1
[1] NIPS0-12 dataset. http://www.stats.ox.ac.uk/˜teh/data.html.
[2] I. Abraham, S. Chechik, D. Kempe, and A. Slivkins. Low-distortion Inference of Latent Similarities from a Multiplex Social Network. CoRR, abs/1202.0922, 2012.
[3] E. M. Airoldi, D. M. Blei, S. E. Fienberg, and E. P. Xing. Mixed Membership Stochastic Blockmodels. Journal of Machine Learning Research, 9:1981–2014, June 2008.
[4] D. M. Blei, A. Y. Ng, and M. I. Jordan. Latent Dirichlet Allocation. Journal of Machine Learning Research, 3:993–1022, 2003.
[5] J. V. Davis, B. Kulis, P. Jain, S. Sra, and I. S. Dhillon. Information-theoretic Metric Learning. In ICML, 2007.
[6] S. E. Fienberg, M. M. Meyer, and S. S. Wasserman. Statistical Analysis of Multiple Sociometric Relations. Journal of the American Statistical Association, 80(389):51–67, March 1985.
[7] A. Globerson and S. Roweis. Metric Learning by Collapsing Classes. In NIPS, 2005.
[8] J. Goldberger, S. Roweis, G. Hinton, and R. Salakhutdinov. Neighbourhood Components Analysis. In NIPS, 2004.
[9] S. Hauberg, O. Freifeld, and M. Black. A Geometric take on Metric Learning. In NIPS, 2012.
[10] T. S. Jaakkola and M. I. Jordan. Variational Probabilistic Inference and the QMR-DT Network. Journal of Artificial Intelligence Research, 10(1):291–322, May 1999.
[11] P. Jain, B. Kulis, I. Dhillon, and K. Grauman. Online Metric Learning and Fast Similarity Search. In NIPS, 2008.
[12] D. Lin. An Information-Theoretic Definition of Similarity. In ICML, 1998.
[13] S. Parameswaran and K. Weinberger. Large Margin Multi-Task Metric Learning. In NIPS, 2010.
[14] J. Pearl. Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 1988.
[15] M. Szell, R. Lambiotte, and S. Thurner. Multirelational Organization of Large-scale Social Networks in an Online World. Proceedings of the National Academy of Sciences, 2010.
[16] L. van der Maaten and G. Hinton. Visualizing Non-Metric Similarities in Multiple Maps. Machine Learning, 33:33–55, 2012.
[17] J. Wang, A. Woznica, and A. Kalousis. Parametric Local Metric Learning for Nearest Neighbor Classification. In NIPS, 2012.
[18] K. Q. Weinberger and L. K. Saul. Distance Metric Learning for Large Margin Nearest Neighbor Classification. Journal of Machine Learning Research, 10:207–244, 2009.
[19] E. P. Xing, A. Y. Ng, M. I. Jordan, and S. Russell. Distance Metric Learning, with Application to Clustering with Side-information. In NIPS, 2002. 9