nips nips2001 nips2001-110 nips2001-110-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Alberto Paccanaro, Geoffrey E. Hinton
Abstract: We present Linear Relational Embedding (LRE), a new method of learning a distributed representation of concepts from data consisting of instances of relations between given concepts. Its final goal is to be able to generalize, i.e. infer new instances of these relations among the concepts. On a task involving family relationships we show that LRE can generalize better than any previously published method. We then show how LRE can be used effectively to find compact distributed representations for variable-sized recursive data structures, such as trees and lists. 1 Linear Relational Embedding Our aim is to take a large set of facts about a domain expressed as tuples of arbitrary symbols in a simple and rigid syntactic format and to be able to infer other “common-sense” facts without having any prior knowledge about the domain. Let us imagine a situation in which we have a set of concepts and a set of relations among these concepts, and that our data consists of few instances of these relations that hold among the concepts. We want to be able to infer other instances of these relations. For example, if the concepts are the people in a certain family, the relations are kinship relations, and we are given the facts ”Alberto has-father Pietro” and ”Pietro has-brother Giovanni”, we would like to be able to infer ”Alberto has-uncle Giovanni”. Our approach is to learn appropriate distributed representations of the entities in the data, and then exploit the generalization properties of the distributed representations [2] to make the inferences. In this paper we present a method, which we have called Linear Relational Embedding (LRE), which learns a distributed representation for the concepts by embedding them in a space where the relations between concepts are linear transformations of their distributed representations. Let us consider the case in which all the relations are binary, i.e. involve two concepts. , and the problem In this case our data consists of triplets we are trying to solve is to infer missing triplets when we are given only few of them. Inferring a triplet is equivalent to being able to complete it, that is to come up with one of its elements, given the other two. Here we shall always try to complete the third element of the triplets 1 . LRE will then represent each concept in the data as a learned vector in a 2 0 £ § ¥ £ § ¥ %
[1] Geoffrey E. Hinton. Learning distributed representations of concepts. In Proceedings of the Eighth Annual Conference of the Cognitive Science Society, pages 1–12. Erlbaum, NJ, 1986.
[2] Geoffrey E. Hinton, James L. McClelland, and David E. Rumelhart. Distributed representations. In David E. Rumelhart, James L. McClelland, and the PDP research Group, editors, Parallel Distributed Processing, volume 1, pages 77–109. The MIT Press, 1986.
[3] Randall C. O’Reilly. The LEABRA model of neural interactions and learning in the neocortex. PhD thesis, Department of Psychology, Carnegie Mellon University, 1996.
[4] Alberto Paccanaro. Learning Distributed Representations of Relational Data using Linear Relational Embedding. PhD thesis, Computer Science Department, University of Toronto, 2002.
[5] Alberto Paccanaro and Geoffrey E. Hinton. Learning distributed representations by mapping concepts and relations into a linear space. In Pat Langley, editor, Proceedings of ICML2000, pages 711–718. Morgan Kaufmann, Stanford University, 2000.
[6] Jordan B. Pollack. Recursive distributed representations. Artificial Intelligence, 46:77–105, 1990.
[7] J. R. Quinlan. Learning logical definitions from relations. Machine Learning, 5:239–266, 1990.