nips nips2006 nips2006-180 nips2006-180-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: T. F. Jaeger, Roger P. Levy
Abstract: If language users are rational, they might choose to structure their utterances so as to optimize communicative properties. In particular, information-theoretic and psycholinguistic considerations suggest that this may include maximizing the uniformity of information density in an utterance. We investigate this possibility in the context of syntactic reduction, where the speaker has the option of either marking a higher-order unit (a phrase) with an extra word, or leaving it unmarked. We demonstrate that speakers are more likely to reduce less information-dense phrases. In a second step, we combine a stochastic model of structured utterance production with a logistic-regression model of syntactic reduction to study which types of cues speakers employ when estimating the predictability of upcoming elements. We demonstrate that the trend toward predictability-sensitive syntactic reduction (Jaeger, 2006) is robust in the face of a wide variety of control variables, and present evidence that speakers use both surface and structural cues for predictability estimation.
Arnold, J. E., Wasow, T., Asudeh, A., and Alrenga, P. (2004). Avoiding attachment ambiguities: The role of constituent ordering. Journal of Memory and Language, 51:55–70. Aylett, M. (2000). Stochastic Suprasegmentals: Relationships between Redundancy, Prosodic Structure and Care of Articulation in Spontaneous Speech. PhD thesis, University of Edinburgh. Aylett, M. and Turk, A. (2004). The Smooth Signal Redundancy Hypothesis: A functional explanation for relationships between redundancy, prosodic prominence, and duration in spontaneous speech. Language and Speech, 47(1):31–56. Bell, A., Jurafsky, D., Fosler-Lussier, E., Girand, C., Gregory, M., and Gildea, D. (2003). Effects of disfluencies, predictability, and utterance position on word form variation in English conversation. Journal of the Acoustical Society of America, 113(2):1001–1024. Berger, A. L., Pietra, S. A. D., and Pietra, V. J. D. (1996). A Maximum Entropy approach to natural language processing. Computational Linguistics, 22(1):39–71. Collins, M. (2003). Head-driven statistical models for natural language parsing. Computational Linguistics, 29:589–637. Della Pietra, S., Della Pietra, V., and Lafferty, J. (1997). Inducing features of random fields. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(4):380–393. Genzel, D. and Charniak, E. (2002). Entropy rate constancy in text. In Proceedings of ACL. Hale, J. (2001). A probabilistic Earley parser as a psycholinguistic model. In Proceedings of NAACL, volume 2, pages 159–166. Jaeger, T. F. (2005). Optional that indicates production difficulty: evidence from disfluencies. In Proceedings of Disfluency in Spontaneous Speech Workshop. Jaeger, T. F. (2006). Redundancy and Syntactic Reduction in Spontaneous Speech. PhD thesis, Stanford University, Stanford, CA. Keller, F. (2004). The entropy rate principle as a predictor of processing effort: An evaluation against eyetracking data. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, pages 317–324, Barcelona. Levy, R. (2005). Probabilistic Models of Word Order and Syntactic Discontinuity. PhD thesis, Stanford University. Levy, R. (2006). Expectation-based syntactic comprehension. Ms., University of Edinburgh. Roark, B. (2001). Probabilistic top-down parsing and language modeling. Computational Linguistics, 27(2):249–276. Roland, D., Elman, J. L., and Ferreira, V. S. (2006). Why is that? Structural prediction and ambiguity resolution in a very large corpus of English sentences. Cognition, 98:245–272. Wasow, T., Jaeger, T. F., and Orr, D. (in press). Lexical variation in relativizer frequency. In Wiese, H. and Simon, H., editors, Proceedings of the Workshop on Expecting the unexpected: Exceptions in Grammar at the 27th Annual Meeting of the German Linguistic Association, University of Cologne, Germany. DGfS.