acl acl2011 acl2011-8 knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Mehdi Manshadi ; James Allen ; Mary Swift
Abstract: Previous work on quantifier scope annotation focuses on scoping sentences with only two quantified noun phrases (NPs), where the quantifiers are restricted to a predefined list. It also ignores negation, modal/logical operators, and other sentential adverbials. We present a comprehensive scope annotation scheme. We annotate the scope interaction between all scopal terms in the sentence from quantifiers to scopal adverbials, without putting any restriction on the number of scopal terms in a sentence. In addition, all NPs, explicitly quantified or not, with no restriction on the type of quantification, are investigated for possible scope interactions. 1
Reference: text
sentIndex sentText sentNum sentScore
1 edu , , Abstract Previous work on quantifier scope annotation focuses on scoping sentences with only two quantified noun phrases (NPs), where the quantifiers are restricted to a predefined list. [sent-3, score-1.563]
2 We annotate the scope interaction between all scopal terms in the sentence from quantifiers to scopal adverbials, without putting any restriction on the number of scopal terms in a sentence. [sent-6, score-1.609]
3 In addition, all NPs, explicitly quantified or not, with no restriction on the type of quantification, are investigated for possible scope interactions. [sent-7, score-0.616]
4 1 Introduction Since the early days of natural language understanding (NLU), quantifier scope disambiguation has been an extremely hard task. [sent-8, score-0.886]
5 Therefore, early NLU systems either devised some mechanism for leaving the semantic representation underspecified (Woods 1978, Hobbs and Shieber 1987), or tried to assign scoping to sentences based on heuristics (VanLehn 1978, Moran 1988, Alshawi 1992). [sent-9, score-0.495]
6 The motivation of most recent formalisms is to develop a constraint-based framework where you can incrementally add constraints to filter out unwanted scopings. [sent-13, score-0.067]
7 However, almost all of these formalisms are based on hard constraints, which have to be 141 satisfied in every reading of the sentence. [sent-14, score-0.119]
8 As a result, statistical methods seem to be well suited for scope disambiguation. [sent-18, score-0.487]
9 Surprisingly enough, after two decades of extensive work on statistical techniques in natural language processing, there has not been much work on scope disambiguation (see section 6 for a review). [sent-19, score-0.582]
10 It considers sentences with only two quantifiers, where the quantifiers are picked from a predefined list. [sent-21, score-0.271]
11 For example, it ignores definites, bare singulars/plurals, and proper nouns, as well as negations and other scopal operators. [sent-22, score-0.385]
12 A major reason for the lack of work on statistical scope disambiguation is the lack of a comprehensive scope-disambiguated corpus. [sent-23, score-0.68]
13 In fact, our own early effort to annotate part of the Penn Treebank with full scope information soon proved to be too ambitious. [sent-27, score-0.52]
14 Instead, we have picked a domain that covers many challenging phenomena in scope disambiguation, while keeping the scope disambiguation fairly intuitive. [sent-28, score-1.15]
15 This helps us to build the first moderately sized corpus of natural language text with full scope information. [sent-29, score-0.521]
16 By fully scoping a sentence, we mean to label the scope interaction between every two scopal elements in that senProceedings ofP thoer t4l9atnhd A, Onrnuegaoln M,e Jeuntineg 19 o-f2 t4h,e 2 A0s1s1o. [sent-30, score-1.322]
17 We scope all scope-bearing NPs (quantified or not), negations, logical/modal operators, and other sentential adverbials. [sent-33, score-0.534]
18 In addition, we label sentences with coreference relations because they affect the scope interaction between NPs. [sent-36, score-0.733]
19 2 Domain The domain is the description of tasks about editing plain text files; in other words, a natural language interface for text editors such as Linux SED, AWK, or EMACS programs. [sent-37, score-0.159]
20 This domain has several properties that make it a great choice for a first effort to build a comprehensive scopedisambiguated corpus. [sent-39, score-0.101]
21 As shown in the examples, the domain carries many quantified NPs. [sent-41, score-0.206]
22 Also, scopal operators such as negation, and logical operators occur pretty often in the domain. [sent-42, score-0.466]
23 Second, scope disambiguation is critical for deep understanding in this domain. [sent-43, score-0.582]
24 Third, scoping is fairly intuitive, because a conscious knowledge of scoping is required in order to be able to accomplish the explained task. [sent-44, score-0.814]
25 This is exactly the key property of this domain that makes building a comprehensive scope-disambiguated corpus feasible. [sent-45, score-0.135]
26 1 The core corpus The core part of the corpus has been gathered from three different resources, each making up roughly one third of the core corpus. [sent-47, score-0.326]
27 • One liners: These are help documents found on the web for Linux command-line text editors such as SED and AWK, giving a description of a task plus one line of code performing the task. [sent-48, score-0.066]
28 • Computer science graduate students: These are the sentences provided by CS graduate students describing some of the routine text editing tasks they often do. [sent-56, score-0.104]
29 2 Expanding corpus with crowd sourcing The core corpus was used to get more sentences using crowd sourcing. [sent-59, score-0.325]
30 We provided input/output (I/O) examples for each task in the core corpus, and asked the workers on Mechanical Turk to provide the description of the task based on the I/O example(s). [sent-60, score-0.196]
31 Figure (2) shows an example of two I/O pairs given to the workers in order to get the description of a single task. [sent-61, score-0.11]
32 The reason for using two I/O pairs (instead of only one) is that there is almost always a trivial description for a single I/O pair. [sent-62, score-0.066]
33 Even with two I/O pairs, we sometimes get the description of a different task, which happens to work for the both pairs. [sent-63, score-0.066]
34 For example the original description for the task given in figure (2) is: 1. [sent-64, score-0.066]
35 We can add these new tasks to the core corpus, label them with new I/O pairs and hence expand the corpus in a bootstrapping fashion. [sent-74, score-0.152]
36 The data acquired from Mechanical Turk is often quite noisy, therefore all sentences are reviewed manually and tagged with different categories (e. [sent-75, score-0.088]
37 3 Pre-processing the corpus The corpus is tokenized and parsed using the Stanford PCFG parser (Klein and Manning 2003). [sent-80, score-0.068]
38 Shallow NP chunks and negations are automatically extracted from the parse trees and indexed. [sent-82, score-0.2]
39 The resulting NP-chunked sentences are then reviewed manually, first to fix the chunking errors, hence providing gold standard chunks, and second, to add chunks for other scopal operators such as sentential adverbials since the above automated approach will not extract those. [sent-83, score-0.705]
40 As shown in these examples, NP chunks are indexed by numbers, negation by the letter ‘N’ fol- lowed by a number and all other scopal operators by the letter ‘O’ followed by a number. [sent-85, score-0.628]
41 4 Scope annotation The chunked sentences are given to the annotators for scope annotation. [sent-86, score-0.637]
42 Given a pair of chunks iand j,three kinds of relation could hold between them. [sent-87, score-0.142]
43 1 • No scope interaction: If a pair is left unscoped, it means that either there is no scope interaction between the chunks, or switching the order of the chunks results in a logically equivalent formula. [sent-93, score-1.258]
44 The overall scoping is represented as a list of semicolon-separated constraints. [sent-94, score-0.407]
45 The annotators 1 Bridging anaphora relations are simply represented as outscoping relations, because often there is not a clear distinction between the two. [sent-95, score-0.309]
46 However for theoretical purposes, an outscoping constraint (i>j), where iis not accessible to j, is being understood as a bridging anaphora relation. [sent-96, score-0.322]
47 intuitive scoping Our early experiments showed that a main source of inter-annotator disagreement are pairs of chunks for which, both orderings are logically equivalent (e. [sent-100, score-0.718]
48 two existentials or two universals), but an annotator may label them with outscoping constraints based on his/her intuition. [sent-102, score-0.207]
49 From a scope– disambiguation point of view, the main issue with plurals come from the fact that they carry two possible kinds of readings: collective vs. [sent-109, score-0.277]
50 We treat plurals as a set of individuals and assume that the index of a plural NP refers to the set (collective reading). [sent-111, score-0.184]
51 However, we also assume that every plural potentially carries an implicit universal quantifier ranging over all elements in the set. [sent-112, score-0.45]
52 We represent this implicit universal with id (‘d’ for distributive) where iis the index of the plural NP. [sent-113, score-0.088]
53 distributivity distinction at the sentence level, for us the right treatment is to make this distinction at the constraint level. [sent-115, score-0.121]
54 That is, a plural may have a collective reading in one constraint but a distributive reading in another, as shown in example 2 in figure (3). [sent-116, score-0.317]
55 3 Other challenges of scope annotation In spite of choosing a specific domain with fairly intuitive quantifier scoping, the scope annotation has been a very challenging job. [sent-118, score-1.394]
56 There are several major sources of difficulty in scope annotation. [sent-119, score-0.528]
57 First, there has not been much work on corpusbased study of quantifier scoping. [sent-120, score-0.271]
58 Most work on quantifier scoping focuses on scoping phenomena, which may be interesting from theoretical perspective, but do not occur very often in practice. [sent-121, score-1.123]
59 During annotation of the corpus, we encountered a lot of these phenomena, which we have tried to generalize and find a reasonable treatment for. [sent-123, score-0.069]
60 Second, other sources of ambiguity are likely to show up as scope disagreement. [sent-124, score-0.487]
61 Finally, very often the disagreement in scoping does not result from the different interpretations of the sentence, but the different representations of the same interpretation. [sent-125, score-0.407]
62 Technical details of the annotation scheme are beyond the scope of this paper. [sent-127, score-0.524]
63 5 Statistics The current corpus contains around 500 sentences in the core level and 2000 sentences acquired from crowd sourcing. [sent-129, score-0.288]
64 9, out of which 95% are NPs and the rest are scopal operators. [sent-131, score-0.278]
65 The core corpus has already been annotated, out of which a hundred sentences have been annotated by three annotators in order to measure the inter-annotator agreement (IAA). [sent-133, score-0.264]
66 1 Inter-annotator agreement Although coreference relations were labeled in the corpus, we do not incorporate them in calculating IAA. [sent-137, score-0.154]
67 This is because, annotating coreference relations is much easier than scope disambiguation, so incorporating them favors toward higher IAAs, which may be deceiving. [sent-138, score-0.578]
68 Furthermore previous work only considers scope relations and hence we do the same in order to have a fair comparison. [sent-139, score-0.526]
69 Corpus statistics We represent each scoping using a directed graph over the chunk indices. [sent-142, score-0.56]
70 For every outscoping relation i>j, node i is connected to node j by the directed edge (i,j). [sent-143, score-0.254]
71 For example, figure (4a) represents the scoping in (5). [sent-144, score-0.407]
72 Delete [1/ the first character] of [2/ every word] and [3/ the first word] of [4/ every line] in [5/ the file]. [sent-146, score-0.1]
73 (5>2>1 ;5>4>3) Note that the directed graph must be a DAG (directed acyclic graph), otherwise the scoping is not valid. [sent-147, score-0.519]
74 In order to be able to measure the similarity of two DAGs corresponding to two different scopings of a single sentence, we borrow the notion of transitive closure from graph theory. [sent-148, score-0.168]
75 The transitive closure (TC) of a directed graph G=(V,E) is the graph G+=(V,E+), where E+ is defined as follows: 6. [sent-149, score-0.196]
76 At constraint level, every pair of chunks in every sentence is considered one instance. [sent-152, score-0.287]
77 A sentence counts as a match if and only if every pair of chunks in the sentence has the same label in both scopings. [sent-154, score-0.224]
78 For example, if the target language is first order logic with generalized quantifiers, the relative scope of the chunks labeled NI does not affect the interpretation. [sent-159, score-0.728]
79 2 Therefore, we define a new version of observed agreement in which we consider a pair a match if it is labeled NI in one scoping or assigned the same label in both scopings. [sent-160, score-0.502]
80 6 Related work To the best of our knowledge, there have been three major efforts on building a scope- disambiguated corpus for statistical scope disambiguation, among which Higgins and Sadock (2003) is the most comprehensive. [sent-162, score-0.593]
81 Their corpus consists of 890 sentences from the Wall Street journal section of the Penn Treebank. [sent-163, score-0.089]
82 They pick sentences containing exactly two quantifiers from a predefined list. [sent-164, score-0.271]
83 Every sentence is labeled with one of the three labels corresponding to the first quantifier having widescope, the second quantifier having wide scope, or no scope interaction between the two. [sent-166, score-1.129]
84 The majority of sentences in their corpus (more than 60%) have been labeled with no scope interaction. [sent-168, score-0.608]
85 They pick a set of sentences from LSAT and GRE logic games, which again contain only two quantifiers from a limited list of quantifiers. [sent-170, score-0.303]
86 In around 70% of these sentences, 2 Note that any pair left unscoped is labeled NI. [sent-172, score-0.088]
87 Most of these pairs are those whose both orderings are logically equivalent (section 4. [sent-173, score-0.105]
88 Besides, we assume all the scopings are valid that is there is at least one interpretation satisfying them. [sent-175, score-0.084]
89 Inter-annotator agreement the first quantifier has wide scope. [sent-178, score-0.302]
90 A major problem with this data is that the sentences are artificially constructed for the LSAT and GRE tests. [sent-179, score-0.096]
91 In a recent work Srinivasan and Yates (2009) study the usage of pragmatic knowledge in finding the intended scoping of a sentence. [sent-180, score-0.454]
92 The corpus consists of short sentences with two specific quantifiers: Every and A. [sent-182, score-0.089]
93 In fact, they try to isolate the effect of pragmatic knowledge on scope disambiguation. [sent-184, score-0.534]
94 7 Summary and future work We have constructed a comprehensive scope– disambiguated corpus of English text within the domain of editing plain text files. [sent-185, score-0.215]
95 Previous work focuses on annotating the relative scope of two NPs per sentence, while ignoring the complex scope-bearing NPs such as definites and indefinites, and achieves the IAA of 52%. [sent-189, score-0.571]
96 The current corpus contains 2500 sentences, out of which 500 sentences have already been annotated. [sent-190, score-0.089]
97 Since world knowledge plays a major role in scope disambiguation, we believe that leveraging unlabeled domain specific data in order to extract lexical information is a promising approach for scope disambiguation. [sent-193, score-1.059]
98 We hope that availability of this corpus motivates more research on statistical scope disambiguation. [sent-194, score-0.521]
99 Quantifier scope disambiguation using extracted pragmatic knowledge: Preliminary results. [sent-261, score-0.629]
100 (1988) Determining the scope of English quantifiers, TR AI-TR-483, AI Lab, MIT. [sent-264, score-0.487]
wordName wordTfidf (topN-words)
[('scope', 0.487), ('scoping', 0.407), ('scopal', 0.278), ('quantifier', 0.271), ('quantifiers', 0.181), ('chunks', 0.142), ('outscoping', 0.139), ('plurals', 0.127), ('iaa', 0.121), ('disambiguation', 0.095), ('operators', 0.094), ('quantified', 0.09), ('core', 0.086), ('alshawi', 0.085), ('definites', 0.084), ('distributive', 0.084), ('scopings', 0.084), ('nps', 0.08), ('logically', 0.074), ('carries', 0.072), ('interaction', 0.068), ('logic', 0.067), ('description', 0.066), ('directed', 0.065), ('quantification', 0.064), ('annotators', 0.058), ('crowd', 0.058), ('negations', 0.058), ('plural', 0.057), ('comprehensive', 0.057), ('adverbials', 0.056), ('awk', 0.056), ('davies', 0.056), ('evel', 0.056), ('lsat', 0.056), ('outscopes', 0.056), ('sadock', 0.056), ('unscoped', 0.056), ('woods', 0.056), ('collective', 0.055), ('sentences', 0.055), ('coreference', 0.052), ('every', 0.05), ('editing', 0.049), ('bare', 0.049), ('egg', 0.049), ('indefinites', 0.049), ('sed', 0.049), ('vanlehn', 0.049), ('pragmatic', 0.047), ('sentential', 0.047), ('graph', 0.047), ('np', 0.046), ('nlu', 0.045), ('tutorials', 0.045), ('constraint', 0.045), ('domain', 0.044), ('workers', 0.044), ('bos', 0.042), ('higgins', 0.042), ('major', 0.041), ('chunk', 0.041), ('copestake', 0.04), ('crouch', 0.04), ('gre', 0.04), ('linux', 0.04), ('srinivasan', 0.04), ('negation', 0.04), ('restriction', 0.039), ('relations', 0.039), ('sort', 0.039), ('fleiss', 0.039), ('galen', 0.039), ('reading', 0.038), ('theoretical', 0.038), ('distinction', 0.038), ('phenomena', 0.037), ('letter', 0.037), ('closure', 0.037), ('hobbs', 0.037), ('readings', 0.037), ('annotation', 0.037), ('constraints', 0.036), ('predefined', 0.035), ('anaphora', 0.035), ('ni', 0.034), ('corpus', 0.034), ('bridging', 0.034), ('reviewed', 0.033), ('early', 0.033), ('label', 0.032), ('lot', 0.032), ('labeled', 0.032), ('agreement', 0.031), ('yates', 0.031), ('iis', 0.031), ('orderings', 0.031), ('intuitive', 0.031), ('formalisms', 0.031), ('disambiguated', 0.031)]
simIndex simValue paperId paperTitle
same-paper 1 1.0000001 8 acl-2011-A Corpus of Scope-disambiguated English Text
Author: Mehdi Manshadi ; James Allen ; Mary Swift
Abstract: Previous work on quantifier scope annotation focuses on scoping sentences with only two quantified noun phrases (NPs), where the quantifiers are restricted to a predefined list. It also ignores negation, modal/logical operators, and other sentential adverbials. We present a comprehensive scope annotation scheme. We annotate the scope interaction between all scopal terms in the sentence from quantifiers to scopal adverbials, without putting any restriction on the number of scopal terms in a sentence. In addition, all NPs, explicitly quantified or not, with no restriction on the type of quantification, are investigated for possible scope interactions. 1
2 0.24959819 50 acl-2011-Automatic Extraction of Lexico-Syntactic Patterns for Detection of Negation and Speculation Scopes
Author: Emilia Apostolova ; Noriko Tomuro ; Dina Demner-Fushman
Abstract: Detecting the linguistic scope of negated and speculated information in text is an important Information Extraction task. This paper presents ScopeFinder, a linguistically motivated rule-based system for the detection of negation and speculation scopes. The system rule set consists of lexico-syntactic patterns automatically extracted from a corpus annotated with negation/speculation cues and their scopes (the BioScope corpus). The system performs on par with state-of-the-art machine learning systems. Additionally, the intuitive and linguistically motivated rules will allow for manual adaptation of the rule set to new domains and corpora. 1 Motivation Information Extraction (IE) systems often face the problem of distinguishing between affirmed, negated, and speculative information in text. For example, sentiment analysis systems need to detect negation for accurate polarity classification. Similarly, medical IE systems need to differentiate between affirmed, negated, and speculated (possible) medical conditions. The importance of the task of negation and speculation (a.k.a. hedge) detection is attested by a number of research initiatives. The creation of the BioScope corpus (Vincze et al., 2008) assisted in the development and evaluation of several negation/hedge scope detection systems. The corpus consists of medical and biological texts annotated for negation, speculation, and their linguistic scope. The 2010 283 Noriko Tomuro Dina Demner-Fushman DePaul University Chicago, IL USA t omuro @ c s . depaul . edu National Library of Medicine Bethesda, MD USA ddemne r@mai l nih . gov . i2b2 NLP Shared Task1 included a track for detection of the assertion status of medical problems (e.g. affirmed, negated, hypothesized, etc.). The CoNLL2010 Shared Task (Farkas et al., 2010) focused on detecting hedges and their scopes in Wikipedia articles and biomedical texts. In this paper, we present a linguistically motivated rule-based system for the detection of negation and speculation scopes that performs on par with state-of-the-art machine learning systems. The rules used by the ScopeFinder system are automatically extracted from the BioScope corpus and encode lexico-syntactic patterns in a user-friendly format. While the system was developed and tested using a biomedical corpus, the rule extraction mechanism is not domain-specific. In addition, the linguistically motivated rule encoding allows for manual adaptation to new domains and corpora. 2 Task Definition Negation/Speculation detection is typically broken down into two sub-tasks - discovering a negation/speculation cue and establishing its scope. The following example from the BioScope corpus shows the annotated hedging cue (in bold) together with its associated scope (surrounded by curly brackets): Finally, we explored the {possible role of 5hydroxyeicosatetraenoic acid as a regulator of arachidonic acid liberation}. Typically, systems first identify negation/speculation cues and subsequently try to identify their associated cue scope. However, the two tasks are interrelated and both require 1https://www.i2b2.org/NLP/Relations/ Proceedings ofP thoer t4l9atnhd A, Onrnuegaoln M,e Jeuntineg 19 o-f2 t4h,e 2 A0s1s1o.c?i ac t2io0n11 fo Ar Cssoocmiaptuiotanti foonra Clo Lminpguutiast i ocns:aslh Loirntpgaupisetrics , pages 283–287, syntactic understanding. Consider the following two sentences from the BioScope corpus: 1) By contrast, {D-mib appears to be uniformly expre1ss)e Bdy yin c oimnatrgaisnta,l { dDis-mcsi }b. 2) Differentiation assays using water soluble phorbol esters reveal that differentiation becomes irreversible soon after AP-1 appears. Both sentences contain the word form appears, however in the first sentence the word marks a hedg- ing cue, while in the second sentence the word does not suggest speculation. Unlike previous work, we do not attempt to identify negation/speculation cues independently of their scopes. Instead, we concentrate on scope detection, simultaneously detecting corresponding cues. 3 Dataset We used the BioScope corpus (Vincze et al., 2008) to develop our system and evaluate its performance. To our knowledge, the BioScope corpus is the only publicly available dataset annotated with negation/speculation cues and their scopes. It consists of biomedical papers, abstracts, and clinical reports (corpus statistics are shown in Tables 1 and 2). Corpus Type Sentences Documents Mean Document Size Clinical752019543.85 Full Papers Paper Abstracts 3352 14565 9 1273 372.44 11.44 Table 1: Statistics of the BioScope corpus. Document sizes represent number of sentences. Corpus Type Negation Cues Speculation Cues Negation Speculation Clinical87211376.6%13.4% Full Papers Paper Abstracts 378 1757 682 2694 13.76% 13.45% 22.29% 17.69% Table 2: Statistics of the BioScope corpus. The 2nd and 3d columns show the total number of cues within the datasets; the 4th and 5th columns show the percentage of negated and speculative sentences. 70% ofthe corpus documents (randomly selected) were used to develop the ScopeFinder system (i.e. extract lexico-syntactic rules) and the remaining 30% were used to evaluate system performance. While the corpus focuses on the biomedical domain, our rule extraction method is not domain specific and in future work we are planning to apply our method on different types of corpora. 4 Method Intuitively, rules for detecting both speculation and negation scopes could be concisely expressed as a 284 Figure 1: Parse tree of the sentence ‘T cells {lack active NFkappa B } bPuatr express Sp1 as expected’ generated by cthtiev eS NtanF-fkoaprdp parser. Speculation scope ewxporedcste are gsehnoewrant eind ellipsis. tTanhecue word is shown in grey. The nearest common ancestor of all cue and scope leaf nodes is shown in a box. combination of lexical and syntactic patterns. example, BioScope O¨zg u¨r For and Radev (2009) examined sample sentences and developed hedging scope rules such as: The scope of a modal verb cue (e.g. may, might, could) is the verb phrase to which it is attached; The scope of a verb cue (e.g. appears, seems) followed by an infinitival clause extends to the whole sentence. Similar lexico-syntactic rules have been also manually compiled and used in a number of hedge scope detection systems, e.g. (Kilicoglu and Bergler, 2008), (Rei and Briscoe, 2010), (Velldal et al., 2010), (Kilicoglu and Bergler, 2010), (Zhou et al., 2010). However, manually creating a comprehensive set of such lexico-syntactic scope rules is a laborious and time-consuming process. In addition, such an approach relies heavily on the availability of accurately parsed sentences, which could be problematic for domains such as biomedical texts (Clegg and Shepherd, 2007; McClosky and Charniak, 2008). Instead, we attempted to automatically extract lexico-syntactic scope rules from the BioScope corpus, relying only on consistent (but not necessarily accurate) parse tree representations. We first parsed each sentence in the training dataset which contained a negation or speculation cue using the Stanford parser (Klein and Manning, 2003; De Marneffe et al., 2006). Figure 1 shows the parse tree of a sample sentence containing a negation cue and its scope. Next, for each cue-scope instance within the sen- tence, we identified the nearest common ancestor Figure 2: Lexico-syntactic pattern extracted from the sentence from Figure 1. The rule is equivalent to the following string representation: (VP (VBP lack) (NP (JJ *scope*) (NN *scope*) (NN *scope*))). which encompassed the cue word(s) and all words in the scope (shown in a box on Figure 1). The subtree rooted by this ancestor is the basis for the resulting lexico-syntactic rule. The leaf nodes of the resulting subtree were converted to a generalized representation: scope words were converted to *scope*; noncue and non-scope words were converted to *; cue words were converted to lower case. Figure 2 shows the resulting rule. This rule generation approach resulted in a large number of very specific rule patterns - 1,681 nega- tion scope rules and 3,043 speculation scope rules were extracted from the training dataset. To identify a more general set of rules (and increase recall) we next performed a simple transformation of the derived rule set. If all children of a rule tree node are of type *scope* or * (i.e. noncue words), the node label is replaced by *scope* or * respectively, and the node’s children are pruned from the rule tree; neighboring identical siblings of type *scope* or * are replaced by a single node of the corresponding type. Figure 3 shows an example of this transformation. (a)ThechildrenofnodesJ /N /N are(b)Thechildren pruned and their labels are replaced by of node NP are *scope*. pruned and its label is replaced by *scope*. Figure 3: Transformation of the tree shown in Figure 2. The final rule is equivalent to the following string representation: (VP (VBP lack) *scope* ) 285 The rule tree pruning described above reduced the negation scope rule patterns to 439 and the speculation rule patterns to 1,000. In addition to generating a set of scope finding rules, we also implemented a module that parses string representations of the lexico-syntactic rules and performs subtree matching. The ScopeFinder module2 identifies negation and speculation scopes in sentence parse trees using string-encoded lexicosyntactic patterns. Candidate sentence parse subtrees are first identified by matching the path of cue leafnodes to the root ofthe rule subtree pattern. Ifan identical path exists in the sentence, the root of the candidate subtree is thus also identified. The candidate subtree is evaluated for a match by recursively comparing all node children (starting from the root of the subtree) to the rule pattern subtree. Nodes of type *scope* and * match any number of nodes, similar to the semantics of Regex Kleene star (*). 5 Results As an informed baseline, we used a previously de- veloped rule-based system for negation and speculation scope discovery (Apostolova and Tomuro, 2010). The system, inspired by the NegEx algorithm (Chapman et al., 2001), uses a list of phrases split into subsets (preceding vs. following their scope) to identify cues using string matching. The cue scopes extend from the cue to the beginning or end of the sentence, depending on the cue type. Table 3 shows the baseline results. PSFCNalpueingpleciarPutcAlai opbtneisor tacsP6597C348o.r12075e4ctly6859RP203475r. 81e26d037icteF569784C52. 04u913e84s5F2A81905l.2786P14redictCus Table 3: Baseline system performance. P (Precision), R (Recall), and F (F1-score) are computed based on the sentence tokens of correctly predicted cues. The last column shows the F1-score for sentence tokens of all predicted cues (including erroneous ones). We used only the scopes of predicted cues (correctly predicted cues vs. all predicted cues) to mea- 2The rule sets and source code are publicly available at http://scopefinder.sourceforge.net/. sure the baseline system performance. The baseline system heuristics did not contain all phrase cues present in the dataset. The scopes of cues that are missing from the baseline system were not included in the results. As the baseline system was not penalized for missing cue phrases, the results represent the upper bound of the system. Table 4 shows the results from applying the full extracted rule set (1,681 negation scope rules and 3,043 speculation scope rules) on the test data. As expected, this rule set consisting of very specific scope matching rules resulted in very high precision and very low recall. Negation P R F A Clinical99.4734.3051.0117.58 Full Papers Paper Abstracts 95.23 87.33 25.89 05.78 40.72 10.84 28.00 07.85 Speculation Clinical96.5020.1233.3022.90 Full Papers Paper Abstracts 88.72 77.50 15.89 11.89 26.95 20.62 10.13 10.00 Table 4: Results from applying the full extracted rule set on the test data. Precision (P), Recall (R), and F1-score (F) are com- puted based the number of correctly identified scope tokens in each sentence. Accuracy (A) is computed for correctly identified full scopes (exact match). Table 5 shows the results from applying the rule set consisting of pruned pattern trees (439 negation scope rules and 1,000 speculation scope rules) on the test data. As shown, overall results improved significantly, both over the baseline and over the unpruned set of rules. Comparable results are shown in bold in Tables 3, 4, and 5. Negation P R F A Clinical85.5992.1588.7585.56 Full Papers 49.17 94.82 64.76 71.26 Paper Abstracts 61.48 92.64 73.91 80.63 Speculation Clinical67.2586.2475.5771.35 Full Papers 65.96 98.43 78.99 52.63 Paper Abstracts 60.24 95.48 73.87 65.28 Table 5: Results from applying the pruned rule set on the test data. Precision (P), Recall (R), and F1-score (F) are computed based on the number of correctly identified scope tokens in each sentence. Accuracy (A) is computed for correctly identified full scopes (exact match). 6 Related Work Interest in the task of identifying negation and spec- ulation scopes has developed in recent years. Rele286 vant research was facilitated by the appearance of a publicly available annotated corpus. All systems described below were developed and evaluated against the BioScope corpus (Vincze et al., 2008). O¨zg u¨r and Radev (2009) have developed a supervised classifier for identifying speculation cues and a manually compiled list of lexico-syntactic rules for identifying their scopes. For the performance of the rule based system on identifying speculation scopes, they report 61. 13 and 79.89 accuracy for BioScope full papers and abstracts respectively. Similarly, Morante and Daelemans (2009b) developed a machine learning system for identifying hedging cues and their scopes. They modeled the scope finding problem as a classification task that determines if a sentence token is the first token in a scope sequence, the last one, or neither. Results of the scope finding system with predicted hedge signals were reported as F1-scores of 38. 16, 59.66, 78.54 and for clinical texts, full papers, and abstracts respectively3. Accuracy (computed for correctly identified scopes) was reported as 26.21, 35.92, and 65.55 for clinical texts, papers, and abstracts respectively. Morante and Daelemans have also developed a metalearner for identifying the scope of negation (2009a). Results of the negation scope finding system with predicted cues are reported as F1-scores (computed on scope tokens) of 84.20, 70.94, and 82.60 for clinical texts, papers, and abstracts respectively. Accuracy (the percent of correctly identified exact scopes) is reported as 70.75, 41.00, and 66.07 for clinical texts, papers, and abstracts respectively. The top three best performers on the CoNLL2010 shared task on hedge scope detection (Farkas et al., 2010) report an F1-score for correctly identified hedge cues and their scopes ranging from 55.3 to 57.3. The shared task evaluation metrics used stricter matching criteria based on exact match of both cues and their corresponding scopes4. CoNLL-2010 shared task participants applied a variety of rule-based and machine learning methods 3F1-scores are computed based on scope tokens. Unlike our evaluation metric, scope token matches are computed for each cue within a sentence, i.e. a token is evaluated multiple times if it belongs to more than one cue scope. 4Our system does not focus on individual cue-scope pair de- tection (we instead optimized scope detection) and as a result performance metrics are not directly comparable. on the task - Morante et al. (2010) used a memorybased classifier based on the k-nearest neighbor rule to determine if a token is the first token in a scope sequence, the last, or neither; Rei and Briscoe (2010) used a combination of manually compiled rules, a CRF classifier, and a sequence of post-processing steps on the same task; Velldal et al (2010) manually compiled a set of heuristics based on syntactic information taken from dependency structures. 7 Discussion We presented a method for automatic extraction of lexico-syntactic rules for negation/speculation scopes from an annotated corpus. The developed ScopeFinder system, based on the automatically extracted rule sets, was compared to a baseline rule-based system that does not use syntactic information. The ScopeFinder system outperformed the baseline system in all cases and exhibited results comparable to complex feature-based, machine-learning systems. In future work, we will explore the use of statistically based methods for the creation of an optimum set of lexico-syntactic tree patterns and will evaluate the system performance on texts from different domains. References E. Apostolova and N. Tomuro. 2010. Exploring surfacelevel heuristics for negation and speculation discovery in clinical texts. In Proceedings of the 2010 Workshop on Biomedical Natural Language Processing, pages 81–82. Association for Computational Linguistics. W.W. Chapman, W. Bridewell, P. Hanbury, G.F. Cooper, and B.G. Buchanan. 2001. A simple algorithm for identifying negated findings and diseases in discharge summaries. Journal of biomedical informatics, 34(5):301–310. A.B. Clegg and A.J. Shepherd. 2007. Benchmarking natural-language parsers for biological applications using dependency graphs. BMC bioinformatics, 8(1):24. M.C. De Marneffe, B. MacCartney, and C.D. Manning. 2006. Generating typed dependency parses from phrase structure parses. In LREC 2006. Citeseer. R. Farkas, V. Vincze, G. M o´ra, J. Csirik, and G. Szarvas. 2010. The CoNLL-2010 Shared Task: Learning to Detect Hedges and their Scope in Natural Language Text. In Proceedings of the Fourteenth Conference on 287 Computational Natural Language Learning (CoNLL2010): Shared Task, pages 1–12. H. Kilicoglu and S. Bergler. 2008. Recognizing speculative language in biomedical research articles: a linguistically motivated perspective. BMC bioinformatics, 9(Suppl 11):S10. H. Kilicoglu and S. Bergler. 2010. A High-Precision Approach to Detecting Hedges and Their Scopes. CoNLL-2010: Shared Task, page 70. D. Klein and C.D. Manning. 2003. Fast exact inference with a factored model for natural language parsing. Advances in neural information processing systems, pages 3–10. D. McClosky and E. Charniak. 2008. Self-training for biomedical parsing. In Proceedings of the 46th Annual Meeting of the Association for Computational Linguistics on Human Language Technologies: Short Papers, pages 101–104. Association for Computational Linguistics. R. Morante and W. Daelemans. 2009a. A metalearning approach to processing the scope of negation. In Proceedings of the Thirteenth Conference on Computational Natural Language Learning, pages 21–29. Association for Computational Linguistics. R. Morante and W. Daelemans. 2009b. Learning the scope of hedge cues in biomedical texts. In Proceed- ings of the Workshop on BioNLP, pages 28–36. Association for Computational Linguistics. R. Morante, V. Van Asch, and W. Daelemans. 2010. Memory-based resolution of in-sentence scopes of hedge cues. CoNLL-2010: Shared Task, page 40. A. O¨zg u¨r and D.R. Radev. 2009. Detecting speculations and their scopes in scientific text. In Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing: Volume 3-Volume 3, pages 1398–1407. Association for Computational Linguistics. M. Rei and T. Briscoe. 2010. Combining manual rules and supervised learning for hedge cue and scope detection. In Proceedings of the 14th Conference on Natural Language Learning, pages 56–63. E. Velldal, L. Øvrelid, and S. Oepen. 2010. Resolving Speculation: MaxEnt Cue Classification and Dependency-Based Scope Rules. CoNLL-2010: Shared Task, page 48. V. Vincze, G. Szarvas, R. Farkas, G. M o´ra, and J. Csirik. 2008. The BioScope corpus: biomedical texts annotated for uncertainty, negation and their scopes. BMC bioinformatics, 9(Suppl 11):S9. H. Zhou, X. Li, D. Huang, Z. Li, and Y. Yang. 2010. Exploiting Multi-Features to Detect Hedges and Their Scope in Biomedical Texts. CoNLL-2010: Shared Task, page 106.
3 0.13317458 273 acl-2011-Semantic Representation of Negation Using Focus Detection
Author: Eduardo Blanco ; Dan Moldovan
Abstract: Negation is present in all human languages and it is used to reverse the polarity of part of statements that are otherwise affirmative by default. A negated statement often carries positive implicit meaning, but to pinpoint the positive part from the negative part is rather difficult. This paper aims at thoroughly representing the semantics of negation by revealing implicit positive meaning. The proposed representation relies on focus of negation detection. For this, new annotation over PropBank and a learning algorithm are proposed.
4 0.085359298 71 acl-2011-Coherent Citation-Based Summarization of Scientific Papers
Author: Amjad Abu-Jbara ; Dragomir Radev
Abstract: In citation-based summarization, text written by several researchers is leveraged to identify the important aspects of a target paper. Previous work on this problem focused almost exclusively on its extraction aspect (i.e. selecting a representative set of citation sentences that highlight the contribution of the target paper). Meanwhile, the fluency of the produced summaries has been mostly ignored. For example, diversity, readability, cohesion, and ordering of the sentences included in the summary have not been thoroughly considered. This resulted in noisy and confusing summaries. In this work, we present an approach for producing readable and cohesive citation-based summaries. Our experiments show that the pro- posed approach outperforms several baselines in terms of both extraction quality and fluency.
5 0.068469033 157 acl-2011-I Thou Thee, Thou Traitor: Predicting Formal vs. Informal Address in English Literature
Author: Manaal Faruqui ; Sebastian Pado
Abstract: In contrast to many languages (like Russian or French), modern English does not distinguish formal and informal (“T/V”) address overtly, for example by pronoun choice. We describe an ongoing study which investigates to what degree the T/V distinction is recoverable in English text, and with what textual features it correlates. Our findings are: (a) human raters can label English utterances as T or V fairly well, given sufficient context; (b), lexical cues can predict T/V almost at human level.
6 0.067931026 19 acl-2011-A Mobile Touchable Application for Online Topic Graph Extraction and Exploration of Web Content
7 0.065504149 85 acl-2011-Coreference Resolution with World Knowledge
8 0.059877224 7 acl-2011-A Corpus for Modeling Morpho-Syntactic Agreement in Arabic: Gender, Number and Rationality
9 0.059707724 72 acl-2011-Collecting Highly Parallel Data for Paraphrase Evaluation
10 0.055598404 281 acl-2011-Sentiment Analysis of Citations using Sentence Structure-Based Features
11 0.05420142 63 acl-2011-Bootstrapping coreference resolution using word associations
12 0.054084446 23 acl-2011-A Pronoun Anaphora Resolution System based on Factorial Hidden Markov Models
13 0.051859632 284 acl-2011-Simple Unsupervised Grammar Induction from Raw Text with Cascaded Finite State Models
14 0.051234599 170 acl-2011-In-domain Relation Discovery with Meta-constraints via Posterior Regularization
15 0.050975885 73 acl-2011-Collective Classification of Congressional Floor-Debate Transcripts
16 0.049369141 9 acl-2011-A Cross-Lingual ILP Solution to Zero Anaphora Resolution
17 0.048219837 206 acl-2011-Learning to Transform and Select Elementary Trees for Improved Syntax-based Machine Translations
18 0.04794823 288 acl-2011-Subjective Natural Language Problems: Motivations, Applications, Characterizations, and Implications
19 0.045174912 200 acl-2011-Learning Dependency-Based Compositional Semantics
20 0.045117691 296 acl-2011-Terminal-Aware Synchronous Binarization
topicId topicWeight
[(0, 0.141), (1, 0.021), (2, -0.043), (3, -0.022), (4, -0.02), (5, 0.037), (6, 0.03), (7, -0.005), (8, -0.057), (9, -0.076), (10, -0.046), (11, -0.045), (12, -0.054), (13, 0.03), (14, -0.023), (15, -0.048), (16, -0.023), (17, -0.035), (18, 0.032), (19, -0.051), (20, -0.017), (21, -0.015), (22, -0.016), (23, -0.012), (24, 0.054), (25, 0.084), (26, -0.143), (27, 0.115), (28, -0.144), (29, -0.07), (30, 0.009), (31, 0.048), (32, 0.083), (33, 0.069), (34, -0.133), (35, 0.04), (36, -0.009), (37, 0.006), (38, 0.171), (39, -0.11), (40, 0.022), (41, -0.071), (42, 0.018), (43, -0.107), (44, 0.081), (45, 0.093), (46, -0.109), (47, 0.121), (48, -0.01), (49, -0.021)]
simIndex simValue paperId paperTitle
same-paper 1 0.94270015 8 acl-2011-A Corpus of Scope-disambiguated English Text
Author: Mehdi Manshadi ; James Allen ; Mary Swift
Abstract: Previous work on quantifier scope annotation focuses on scoping sentences with only two quantified noun phrases (NPs), where the quantifiers are restricted to a predefined list. It also ignores negation, modal/logical operators, and other sentential adverbials. We present a comprehensive scope annotation scheme. We annotate the scope interaction between all scopal terms in the sentence from quantifiers to scopal adverbials, without putting any restriction on the number of scopal terms in a sentence. In addition, all NPs, explicitly quantified or not, with no restriction on the type of quantification, are investigated for possible scope interactions. 1
Author: Emilia Apostolova ; Noriko Tomuro ; Dina Demner-Fushman
Abstract: Detecting the linguistic scope of negated and speculated information in text is an important Information Extraction task. This paper presents ScopeFinder, a linguistically motivated rule-based system for the detection of negation and speculation scopes. The system rule set consists of lexico-syntactic patterns automatically extracted from a corpus annotated with negation/speculation cues and their scopes (the BioScope corpus). The system performs on par with state-of-the-art machine learning systems. Additionally, the intuitive and linguistically motivated rules will allow for manual adaptation of the rule set to new domains and corpora. 1 Motivation Information Extraction (IE) systems often face the problem of distinguishing between affirmed, negated, and speculative information in text. For example, sentiment analysis systems need to detect negation for accurate polarity classification. Similarly, medical IE systems need to differentiate between affirmed, negated, and speculated (possible) medical conditions. The importance of the task of negation and speculation (a.k.a. hedge) detection is attested by a number of research initiatives. The creation of the BioScope corpus (Vincze et al., 2008) assisted in the development and evaluation of several negation/hedge scope detection systems. The corpus consists of medical and biological texts annotated for negation, speculation, and their linguistic scope. The 2010 283 Noriko Tomuro Dina Demner-Fushman DePaul University Chicago, IL USA t omuro @ c s . depaul . edu National Library of Medicine Bethesda, MD USA ddemne r@mai l nih . gov . i2b2 NLP Shared Task1 included a track for detection of the assertion status of medical problems (e.g. affirmed, negated, hypothesized, etc.). The CoNLL2010 Shared Task (Farkas et al., 2010) focused on detecting hedges and their scopes in Wikipedia articles and biomedical texts. In this paper, we present a linguistically motivated rule-based system for the detection of negation and speculation scopes that performs on par with state-of-the-art machine learning systems. The rules used by the ScopeFinder system are automatically extracted from the BioScope corpus and encode lexico-syntactic patterns in a user-friendly format. While the system was developed and tested using a biomedical corpus, the rule extraction mechanism is not domain-specific. In addition, the linguistically motivated rule encoding allows for manual adaptation to new domains and corpora. 2 Task Definition Negation/Speculation detection is typically broken down into two sub-tasks - discovering a negation/speculation cue and establishing its scope. The following example from the BioScope corpus shows the annotated hedging cue (in bold) together with its associated scope (surrounded by curly brackets): Finally, we explored the {possible role of 5hydroxyeicosatetraenoic acid as a regulator of arachidonic acid liberation}. Typically, systems first identify negation/speculation cues and subsequently try to identify their associated cue scope. However, the two tasks are interrelated and both require 1https://www.i2b2.org/NLP/Relations/ Proceedings ofP thoer t4l9atnhd A, Onrnuegaoln M,e Jeuntineg 19 o-f2 t4h,e 2 A0s1s1o.c?i ac t2io0n11 fo Ar Cssoocmiaptuiotanti foonra Clo Lminpguutiast i ocns:aslh Loirntpgaupisetrics , pages 283–287, syntactic understanding. Consider the following two sentences from the BioScope corpus: 1) By contrast, {D-mib appears to be uniformly expre1ss)e Bdy yin c oimnatrgaisnta,l { dDis-mcsi }b. 2) Differentiation assays using water soluble phorbol esters reveal that differentiation becomes irreversible soon after AP-1 appears. Both sentences contain the word form appears, however in the first sentence the word marks a hedg- ing cue, while in the second sentence the word does not suggest speculation. Unlike previous work, we do not attempt to identify negation/speculation cues independently of their scopes. Instead, we concentrate on scope detection, simultaneously detecting corresponding cues. 3 Dataset We used the BioScope corpus (Vincze et al., 2008) to develop our system and evaluate its performance. To our knowledge, the BioScope corpus is the only publicly available dataset annotated with negation/speculation cues and their scopes. It consists of biomedical papers, abstracts, and clinical reports (corpus statistics are shown in Tables 1 and 2). Corpus Type Sentences Documents Mean Document Size Clinical752019543.85 Full Papers Paper Abstracts 3352 14565 9 1273 372.44 11.44 Table 1: Statistics of the BioScope corpus. Document sizes represent number of sentences. Corpus Type Negation Cues Speculation Cues Negation Speculation Clinical87211376.6%13.4% Full Papers Paper Abstracts 378 1757 682 2694 13.76% 13.45% 22.29% 17.69% Table 2: Statistics of the BioScope corpus. The 2nd and 3d columns show the total number of cues within the datasets; the 4th and 5th columns show the percentage of negated and speculative sentences. 70% ofthe corpus documents (randomly selected) were used to develop the ScopeFinder system (i.e. extract lexico-syntactic rules) and the remaining 30% were used to evaluate system performance. While the corpus focuses on the biomedical domain, our rule extraction method is not domain specific and in future work we are planning to apply our method on different types of corpora. 4 Method Intuitively, rules for detecting both speculation and negation scopes could be concisely expressed as a 284 Figure 1: Parse tree of the sentence ‘T cells {lack active NFkappa B } bPuatr express Sp1 as expected’ generated by cthtiev eS NtanF-fkoaprdp parser. Speculation scope ewxporedcste are gsehnoewrant eind ellipsis. tTanhecue word is shown in grey. The nearest common ancestor of all cue and scope leaf nodes is shown in a box. combination of lexical and syntactic patterns. example, BioScope O¨zg u¨r For and Radev (2009) examined sample sentences and developed hedging scope rules such as: The scope of a modal verb cue (e.g. may, might, could) is the verb phrase to which it is attached; The scope of a verb cue (e.g. appears, seems) followed by an infinitival clause extends to the whole sentence. Similar lexico-syntactic rules have been also manually compiled and used in a number of hedge scope detection systems, e.g. (Kilicoglu and Bergler, 2008), (Rei and Briscoe, 2010), (Velldal et al., 2010), (Kilicoglu and Bergler, 2010), (Zhou et al., 2010). However, manually creating a comprehensive set of such lexico-syntactic scope rules is a laborious and time-consuming process. In addition, such an approach relies heavily on the availability of accurately parsed sentences, which could be problematic for domains such as biomedical texts (Clegg and Shepherd, 2007; McClosky and Charniak, 2008). Instead, we attempted to automatically extract lexico-syntactic scope rules from the BioScope corpus, relying only on consistent (but not necessarily accurate) parse tree representations. We first parsed each sentence in the training dataset which contained a negation or speculation cue using the Stanford parser (Klein and Manning, 2003; De Marneffe et al., 2006). Figure 1 shows the parse tree of a sample sentence containing a negation cue and its scope. Next, for each cue-scope instance within the sen- tence, we identified the nearest common ancestor Figure 2: Lexico-syntactic pattern extracted from the sentence from Figure 1. The rule is equivalent to the following string representation: (VP (VBP lack) (NP (JJ *scope*) (NN *scope*) (NN *scope*))). which encompassed the cue word(s) and all words in the scope (shown in a box on Figure 1). The subtree rooted by this ancestor is the basis for the resulting lexico-syntactic rule. The leaf nodes of the resulting subtree were converted to a generalized representation: scope words were converted to *scope*; noncue and non-scope words were converted to *; cue words were converted to lower case. Figure 2 shows the resulting rule. This rule generation approach resulted in a large number of very specific rule patterns - 1,681 nega- tion scope rules and 3,043 speculation scope rules were extracted from the training dataset. To identify a more general set of rules (and increase recall) we next performed a simple transformation of the derived rule set. If all children of a rule tree node are of type *scope* or * (i.e. noncue words), the node label is replaced by *scope* or * respectively, and the node’s children are pruned from the rule tree; neighboring identical siblings of type *scope* or * are replaced by a single node of the corresponding type. Figure 3 shows an example of this transformation. (a)ThechildrenofnodesJ /N /N are(b)Thechildren pruned and their labels are replaced by of node NP are *scope*. pruned and its label is replaced by *scope*. Figure 3: Transformation of the tree shown in Figure 2. The final rule is equivalent to the following string representation: (VP (VBP lack) *scope* ) 285 The rule tree pruning described above reduced the negation scope rule patterns to 439 and the speculation rule patterns to 1,000. In addition to generating a set of scope finding rules, we also implemented a module that parses string representations of the lexico-syntactic rules and performs subtree matching. The ScopeFinder module2 identifies negation and speculation scopes in sentence parse trees using string-encoded lexicosyntactic patterns. Candidate sentence parse subtrees are first identified by matching the path of cue leafnodes to the root ofthe rule subtree pattern. Ifan identical path exists in the sentence, the root of the candidate subtree is thus also identified. The candidate subtree is evaluated for a match by recursively comparing all node children (starting from the root of the subtree) to the rule pattern subtree. Nodes of type *scope* and * match any number of nodes, similar to the semantics of Regex Kleene star (*). 5 Results As an informed baseline, we used a previously de- veloped rule-based system for negation and speculation scope discovery (Apostolova and Tomuro, 2010). The system, inspired by the NegEx algorithm (Chapman et al., 2001), uses a list of phrases split into subsets (preceding vs. following their scope) to identify cues using string matching. The cue scopes extend from the cue to the beginning or end of the sentence, depending on the cue type. Table 3 shows the baseline results. PSFCNalpueingpleciarPutcAlai opbtneisor tacsP6597C348o.r12075e4ctly6859RP203475r. 81e26d037icteF569784C52. 04u913e84s5F2A81905l.2786P14redictCus Table 3: Baseline system performance. P (Precision), R (Recall), and F (F1-score) are computed based on the sentence tokens of correctly predicted cues. The last column shows the F1-score for sentence tokens of all predicted cues (including erroneous ones). We used only the scopes of predicted cues (correctly predicted cues vs. all predicted cues) to mea- 2The rule sets and source code are publicly available at http://scopefinder.sourceforge.net/. sure the baseline system performance. The baseline system heuristics did not contain all phrase cues present in the dataset. The scopes of cues that are missing from the baseline system were not included in the results. As the baseline system was not penalized for missing cue phrases, the results represent the upper bound of the system. Table 4 shows the results from applying the full extracted rule set (1,681 negation scope rules and 3,043 speculation scope rules) on the test data. As expected, this rule set consisting of very specific scope matching rules resulted in very high precision and very low recall. Negation P R F A Clinical99.4734.3051.0117.58 Full Papers Paper Abstracts 95.23 87.33 25.89 05.78 40.72 10.84 28.00 07.85 Speculation Clinical96.5020.1233.3022.90 Full Papers Paper Abstracts 88.72 77.50 15.89 11.89 26.95 20.62 10.13 10.00 Table 4: Results from applying the full extracted rule set on the test data. Precision (P), Recall (R), and F1-score (F) are com- puted based the number of correctly identified scope tokens in each sentence. Accuracy (A) is computed for correctly identified full scopes (exact match). Table 5 shows the results from applying the rule set consisting of pruned pattern trees (439 negation scope rules and 1,000 speculation scope rules) on the test data. As shown, overall results improved significantly, both over the baseline and over the unpruned set of rules. Comparable results are shown in bold in Tables 3, 4, and 5. Negation P R F A Clinical85.5992.1588.7585.56 Full Papers 49.17 94.82 64.76 71.26 Paper Abstracts 61.48 92.64 73.91 80.63 Speculation Clinical67.2586.2475.5771.35 Full Papers 65.96 98.43 78.99 52.63 Paper Abstracts 60.24 95.48 73.87 65.28 Table 5: Results from applying the pruned rule set on the test data. Precision (P), Recall (R), and F1-score (F) are computed based on the number of correctly identified scope tokens in each sentence. Accuracy (A) is computed for correctly identified full scopes (exact match). 6 Related Work Interest in the task of identifying negation and spec- ulation scopes has developed in recent years. Rele286 vant research was facilitated by the appearance of a publicly available annotated corpus. All systems described below were developed and evaluated against the BioScope corpus (Vincze et al., 2008). O¨zg u¨r and Radev (2009) have developed a supervised classifier for identifying speculation cues and a manually compiled list of lexico-syntactic rules for identifying their scopes. For the performance of the rule based system on identifying speculation scopes, they report 61. 13 and 79.89 accuracy for BioScope full papers and abstracts respectively. Similarly, Morante and Daelemans (2009b) developed a machine learning system for identifying hedging cues and their scopes. They modeled the scope finding problem as a classification task that determines if a sentence token is the first token in a scope sequence, the last one, or neither. Results of the scope finding system with predicted hedge signals were reported as F1-scores of 38. 16, 59.66, 78.54 and for clinical texts, full papers, and abstracts respectively3. Accuracy (computed for correctly identified scopes) was reported as 26.21, 35.92, and 65.55 for clinical texts, papers, and abstracts respectively. Morante and Daelemans have also developed a metalearner for identifying the scope of negation (2009a). Results of the negation scope finding system with predicted cues are reported as F1-scores (computed on scope tokens) of 84.20, 70.94, and 82.60 for clinical texts, papers, and abstracts respectively. Accuracy (the percent of correctly identified exact scopes) is reported as 70.75, 41.00, and 66.07 for clinical texts, papers, and abstracts respectively. The top three best performers on the CoNLL2010 shared task on hedge scope detection (Farkas et al., 2010) report an F1-score for correctly identified hedge cues and their scopes ranging from 55.3 to 57.3. The shared task evaluation metrics used stricter matching criteria based on exact match of both cues and their corresponding scopes4. CoNLL-2010 shared task participants applied a variety of rule-based and machine learning methods 3F1-scores are computed based on scope tokens. Unlike our evaluation metric, scope token matches are computed for each cue within a sentence, i.e. a token is evaluated multiple times if it belongs to more than one cue scope. 4Our system does not focus on individual cue-scope pair de- tection (we instead optimized scope detection) and as a result performance metrics are not directly comparable. on the task - Morante et al. (2010) used a memorybased classifier based on the k-nearest neighbor rule to determine if a token is the first token in a scope sequence, the last, or neither; Rei and Briscoe (2010) used a combination of manually compiled rules, a CRF classifier, and a sequence of post-processing steps on the same task; Velldal et al (2010) manually compiled a set of heuristics based on syntactic information taken from dependency structures. 7 Discussion We presented a method for automatic extraction of lexico-syntactic rules for negation/speculation scopes from an annotated corpus. The developed ScopeFinder system, based on the automatically extracted rule sets, was compared to a baseline rule-based system that does not use syntactic information. The ScopeFinder system outperformed the baseline system in all cases and exhibited results comparable to complex feature-based, machine-learning systems. In future work, we will explore the use of statistically based methods for the creation of an optimum set of lexico-syntactic tree patterns and will evaluate the system performance on texts from different domains. References E. Apostolova and N. Tomuro. 2010. Exploring surfacelevel heuristics for negation and speculation discovery in clinical texts. In Proceedings of the 2010 Workshop on Biomedical Natural Language Processing, pages 81–82. Association for Computational Linguistics. W.W. Chapman, W. Bridewell, P. Hanbury, G.F. Cooper, and B.G. Buchanan. 2001. A simple algorithm for identifying negated findings and diseases in discharge summaries. Journal of biomedical informatics, 34(5):301–310. A.B. Clegg and A.J. Shepherd. 2007. Benchmarking natural-language parsers for biological applications using dependency graphs. BMC bioinformatics, 8(1):24. M.C. De Marneffe, B. MacCartney, and C.D. Manning. 2006. Generating typed dependency parses from phrase structure parses. In LREC 2006. Citeseer. R. Farkas, V. Vincze, G. M o´ra, J. Csirik, and G. Szarvas. 2010. The CoNLL-2010 Shared Task: Learning to Detect Hedges and their Scope in Natural Language Text. In Proceedings of the Fourteenth Conference on 287 Computational Natural Language Learning (CoNLL2010): Shared Task, pages 1–12. H. Kilicoglu and S. Bergler. 2008. Recognizing speculative language in biomedical research articles: a linguistically motivated perspective. BMC bioinformatics, 9(Suppl 11):S10. H. Kilicoglu and S. Bergler. 2010. A High-Precision Approach to Detecting Hedges and Their Scopes. CoNLL-2010: Shared Task, page 70. D. Klein and C.D. Manning. 2003. Fast exact inference with a factored model for natural language parsing. Advances in neural information processing systems, pages 3–10. D. McClosky and E. Charniak. 2008. Self-training for biomedical parsing. In Proceedings of the 46th Annual Meeting of the Association for Computational Linguistics on Human Language Technologies: Short Papers, pages 101–104. Association for Computational Linguistics. R. Morante and W. Daelemans. 2009a. A metalearning approach to processing the scope of negation. In Proceedings of the Thirteenth Conference on Computational Natural Language Learning, pages 21–29. Association for Computational Linguistics. R. Morante and W. Daelemans. 2009b. Learning the scope of hedge cues in biomedical texts. In Proceed- ings of the Workshop on BioNLP, pages 28–36. Association for Computational Linguistics. R. Morante, V. Van Asch, and W. Daelemans. 2010. Memory-based resolution of in-sentence scopes of hedge cues. CoNLL-2010: Shared Task, page 40. A. O¨zg u¨r and D.R. Radev. 2009. Detecting speculations and their scopes in scientific text. In Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing: Volume 3-Volume 3, pages 1398–1407. Association for Computational Linguistics. M. Rei and T. Briscoe. 2010. Combining manual rules and supervised learning for hedge cue and scope detection. In Proceedings of the 14th Conference on Natural Language Learning, pages 56–63. E. Velldal, L. Øvrelid, and S. Oepen. 2010. Resolving Speculation: MaxEnt Cue Classification and Dependency-Based Scope Rules. CoNLL-2010: Shared Task, page 48. V. Vincze, G. Szarvas, R. Farkas, G. M o´ra, and J. Csirik. 2008. The BioScope corpus: biomedical texts annotated for uncertainty, negation and their scopes. BMC bioinformatics, 9(Suppl 11):S9. H. Zhou, X. Li, D. Huang, Z. Li, and Y. Yang. 2010. Exploiting Multi-Features to Detect Hedges and Their Scope in Biomedical Texts. CoNLL-2010: Shared Task, page 106.
3 0.83030128 273 acl-2011-Semantic Representation of Negation Using Focus Detection
Author: Eduardo Blanco ; Dan Moldovan
Abstract: Negation is present in all human languages and it is used to reverse the polarity of part of statements that are otherwise affirmative by default. A negated statement often carries positive implicit meaning, but to pinpoint the positive part from the negative part is rather difficult. This paper aims at thoroughly representing the semantics of negation by revealing implicit positive meaning. The proposed representation relies on focus of negation detection. For this, new annotation over PropBank and a learning algorithm are proposed.
4 0.42038184 165 acl-2011-Improving Classification of Medical Assertions in Clinical Notes
Author: Youngjun Kim ; Ellen Riloff ; Stephane Meystre
Abstract: We present an NLP system that classifies the assertion type of medical problems in clinical notes used for the Fourth i2b2/VA Challenge. Our classifier uses a variety of linguistic features, including lexical, syntactic, lexicosyntactic, and contextual features. To overcome an extremely unbalanced distribution of assertion types in the data set, we focused our efforts on adding features specifically to improve the performance of minority classes. As a result, our system reached 94. 17% micro-averaged and 79.76% macro-averaged F1-measures, and showed substantial recall gains on the minority classes. 1
5 0.38406876 281 acl-2011-Sentiment Analysis of Citations using Sentence Structure-Based Features
Author: Awais Athar
Abstract: Sentiment analysis of citations in scientific papers and articles is a new and interesting problem due to the many linguistic differences between scientific texts and other genres. In this paper, we focus on the problem of automatic identification of positive and negative sentiment polarity in citations to scientific papers. Using a newly constructed annotated citation sentiment corpus, we explore the effectiveness of existing and novel features, including n-grams, specialised science-specific lexical features, dependency relations, sentence splitting and negation features. Our results show that 3-grams and dependencies perform best in this task; they outperform the sentence splitting, science lexicon and negation based features.
6 0.3792668 294 acl-2011-Temporal Evaluation
7 0.36554924 138 acl-2011-French TimeBank: An ISO-TimeML Annotated Reference Corpus
9 0.35841167 71 acl-2011-Coherent Citation-Based Summarization of Scientific Papers
10 0.35400641 209 acl-2011-Lexically-Triggered Hidden Markov Models for Clinical Document Coding
11 0.35009906 74 acl-2011-Combining Indicators of Allophony
12 0.34737808 63 acl-2011-Bootstrapping coreference resolution using word associations
13 0.34348455 85 acl-2011-Coreference Resolution with World Knowledge
14 0.34111214 120 acl-2011-Even the Abstract have Color: Consensus in Word-Colour Associations
15 0.33783689 9 acl-2011-A Cross-Lingual ILP Solution to Zero Anaphora Resolution
16 0.33542144 215 acl-2011-MACAON An NLP Tool Suite for Processing Word Lattices
17 0.32811695 252 acl-2011-Prototyping virtual instructors from human-human corpora
18 0.32247764 28 acl-2011-A Statistical Tree Annotator and Its Applications
19 0.32081735 157 acl-2011-I Thou Thee, Thou Traitor: Predicting Formal vs. Informal Address in English Literature
20 0.32027397 42 acl-2011-An Interface for Rapid Natural Language Processing Development in UIMA
topicId topicWeight
[(5, 0.051), (17, 0.059), (26, 0.028), (31, 0.024), (37, 0.077), (39, 0.029), (41, 0.05), (53, 0.016), (55, 0.033), (59, 0.056), (72, 0.047), (91, 0.037), (93, 0.286), (96, 0.128)]
simIndex simValue paperId paperTitle
1 0.79630601 294 acl-2011-Temporal Evaluation
Author: Naushad UzZaman ; James Allen
Abstract: In this paper we propose a new method for evaluating systems that extract temporal information from text. It uses temporal closure1 to reward relations that are equivalent but distinct. Our metric measures the overall performance of systems with a single score, making comparison between different systems straightforward. Our approach is easy to implement, intuitive, accurate, scalable and computationally inexpensive. 1
same-paper 2 0.7586655 8 acl-2011-A Corpus of Scope-disambiguated English Text
Author: Mehdi Manshadi ; James Allen ; Mary Swift
Abstract: Previous work on quantifier scope annotation focuses on scoping sentences with only two quantified noun phrases (NPs), where the quantifiers are restricted to a predefined list. It also ignores negation, modal/logical operators, and other sentential adverbials. We present a comprehensive scope annotation scheme. We annotate the scope interaction between all scopal terms in the sentence from quantifiers to scopal adverbials, without putting any restriction on the number of scopal terms in a sentence. In addition, all NPs, explicitly quantified or not, with no restriction on the type of quantification, are investigated for possible scope interactions. 1
3 0.74235606 191 acl-2011-Knowledge Base Population: Successful Approaches and Challenges
Author: Heng Ji ; Ralph Grishman
Abstract: In this paper we give an overview of the Knowledge Base Population (KBP) track at the 2010 Text Analysis Conference. The main goal of KBP is to promote research in discovering facts about entities and augmenting a knowledge base (KB) with these facts. This is done through two tasks, Entity Linking linking names in context to entities in the KB and Slot Filling – adding information about an entity to the KB. A large source collection of newswire and web documents is provided from which systems are to discover information. Attributes (“slots”) derived from Wikipedia infoboxes are used to create the reference KB. In this paper we provide an overview of the techniques which can serve as a basis for a good KBP system, lay out the – – remaining challenges by comparison with traditional Information Extraction (IE) and Question Answering (QA) tasks, and provide some suggestions to address these challenges. 1
4 0.54658329 32 acl-2011-Algorithm Selection and Model Adaptation for ESL Correction Tasks
Author: Alla Rozovskaya ; Dan Roth
Abstract: We consider the problem of correcting errors made by English as a Second Language (ESL) writers and address two issues that are essential to making progress in ESL error correction - algorithm selection and model adaptation to the first language of the ESL learner. A variety of learning algorithms have been applied to correct ESL mistakes, but often comparisons were made between incomparable data sets. We conduct an extensive, fair comparison of four popular learning methods for the task, reversing conclusions from earlier evaluations. Our results hold for different training sets, genres, and feature sets. A second key issue in ESL error correction is the adaptation of a model to the first language ofthe writer. Errors made by non-native speakers exhibit certain regularities and, as we show, models perform much better when they use knowledge about error patterns of the nonnative writers. We propose a novel way to adapt a learned algorithm to the first language of the writer that is both cheaper to implement and performs better than other adaptation methods.
5 0.54574805 190 acl-2011-Knowledge-Based Weak Supervision for Information Extraction of Overlapping Relations
Author: Raphael Hoffmann ; Congle Zhang ; Xiao Ling ; Luke Zettlemoyer ; Daniel S. Weld
Abstract: Information extraction (IE) holds the promise of generating a large-scale knowledge base from the Web’s natural language text. Knowledge-based weak supervision, using structured data to heuristically label a training corpus, works towards this goal by enabling the automated learning of a potentially unbounded number of relation extractors. Recently, researchers have developed multiinstance learning algorithms to combat the noisy training data that can come from heuristic labeling, but their models assume relations are disjoint — for example they cannot extract the pair Founded ( Jobs Apple ) and CEO-o f ( Jobs Apple ) . , , This paper presents a novel approach for multi-instance learning with overlapping relations that combines a sentence-level extrac- , tion model with a simple, corpus-level component for aggregating the individual facts. We apply our model to learn extractors for NY Times text using weak supervision from Freebase. Experiments show that the approach runs quickly and yields surprising gains in accuracy, at both the aggregate and sentence level.
6 0.54296827 324 acl-2011-Unsupervised Semantic Role Induction via Split-Merge Clustering
7 0.54228592 311 acl-2011-Translationese and Its Dialects
8 0.54182744 327 acl-2011-Using Bilingual Parallel Corpora for Cross-Lingual Textual Entailment
9 0.54159552 170 acl-2011-In-domain Relation Discovery with Meta-constraints via Posterior Regularization
10 0.53973246 86 acl-2011-Coreference for Learning to Extract Relations: Yes Virginia, Coreference Matters
11 0.53950381 274 acl-2011-Semi-Supervised Frame-Semantic Parsing for Unknown Predicates
12 0.53940296 3 acl-2011-A Bayesian Model for Unsupervised Semantic Parsing
13 0.53804719 235 acl-2011-Optimal and Syntactically-Informed Decoding for Monolingual Phrase-Based Alignment
14 0.53781605 133 acl-2011-Extracting Social Power Relationships from Natural Language
15 0.53726131 126 acl-2011-Exploiting Syntactico-Semantic Structures for Relation Extraction
16 0.536695 119 acl-2011-Evaluating the Impact of Coder Errors on Active Learning
17 0.536107 65 acl-2011-Can Document Selection Help Semi-supervised Learning? A Case Study On Event Extraction
18 0.53608155 36 acl-2011-An Efficient Indexer for Large N-Gram Corpora
19 0.53604156 246 acl-2011-Piggyback: Using Search Engines for Robust Cross-Domain Named Entity Recognition
20 0.5344553 145 acl-2011-Good Seed Makes a Good Crop: Accelerating Active Learning Using Language Modeling