acl acl2011 acl2011-209 knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Svetlana Kiritchenko ; Colin Cherry
Abstract: The automatic coding of clinical documents is an important task for today’s healthcare providers. Though it can be viewed as multi-label document classification, the coding problem has the interesting property that most code assignments can be supported by a single phrase found in the input document. We propose a Lexically-Triggered Hidden Markov Model (LT-HMM) that leverages these phrases to improve coding accuracy. The LT-HMM works in two stages: first, a lexical match is performed against a term dictionary to collect a set of candidate codes for a document. Next, a discriminative HMM selects the best subset of codes to assign to the document by tagging candidates as present or absent. By confirming codes proposed by a dictionary, the LT-HMM can share features across codes, enabling strong performance even on rare codes. In fact, we are able to recover codes that do not occur in the training set at all. Our approach achieves the best ever performance on the 2007 Medical NLP Challenge test set, with an F-measure of 89.84.
Reference: text
sentIndex sentText sentNum sentScore
1 ca Abstract The automatic coding of clinical documents is an important task for today’s healthcare providers. [sent-5, score-0.613]
2 Though it can be viewed as multi-label document classification, the coding problem has the interesting property that most code assignments can be supported by a single phrase found in the input document. [sent-6, score-0.903]
3 We propose a Lexically-Triggered Hidden Markov Model (LT-HMM) that leverages these phrases to improve coding accuracy. [sent-7, score-0.275]
4 The LT-HMM works in two stages: first, a lexical match is performed against a term dictionary to collect a set of candidate codes for a document. [sent-8, score-0.896]
5 Next, a discriminative HMM selects the best subset of codes to assign to the document by tagging candidates as present or absent. [sent-9, score-0.824]
6 By confirming codes proposed by a dictionary, the LT-HMM can share features across codes, enabling strong performance even on rare codes. [sent-10, score-0.64]
7 In fact, we are able to recover codes that do not occur in the training set at all. [sent-11, score-0.622]
8 Converting clinical narratives into a structured form would support essential activities such as administrative reporting, quality control, biosurveillance and biomedical research (Meystre et al. [sent-17, score-0.257]
9 One way of representing a document is to code the patient’s conditions and the performed procedures into a nomenclature of clinical codes. [sent-19, score-0.857]
10 The International Classification of Diseases, 9th and 10th revisions, Clinical Modification (ICD9-CM, ICD-10-CM) are the official administrative coding schemes for healthcare organizations in several countries, including the US and Canada. [sent-20, score-0.435]
11 Typically, coding is performed by trained coding professionals, but this process can be both costly and errorprone. [sent-21, score-0.55]
12 Automated methods can speed-up the coding process, improve the accuracy and consistency of internal documentation, and even result in higher reimbursement for the healthcare organization (Benson, 2006). [sent-22, score-0.366]
13 Traditionally, statistical document coding is viewed as multi-class multi-label document classification, where each clinical free-text document is labelled with one or several codes from a pre-defined, possibly very large set of codes (Patrick et al. [sent-23, score-2.156]
14 One classification model is learned for each code, and then all models are applied in turn to a new document to determine which codes should be assigned to the document. [sent-26, score-0.791]
15 This paper presents a novel approach to document coding that simultaneously models code-specific as well as general patterns in the data. [sent-28, score-0.43]
16 Ac s2s0o1ci1a Atiosnso fcoirat Cioonm foprut Caotimonpaulta Lti nognuails Lti cnsg,u piasgteics 742–751, us to predict any code label, even codes for which no training data is available. [sent-31, score-1.09]
17 Our approach, the lexically-triggered HMM (LT-HMM), is based on the fact that a code assignment is often indicated by short lexical triggers in the text. [sent-32, score-0.585]
18 First, the LT-HMM identifies candidate codes by matching terms from a medical terminology dictionary. [sent-34, score-0.91]
19 In this architecture, low-frequency codes can still be matched and confirmed using general characteristics of their trigger’s local context, leading to better prediction performance on these codes. [sent-36, score-0.597]
20 2 Document Coding and Lexical Triggers Document coding is a special case of multi-class multi-label text classification. [sent-37, score-0.275]
21 Given a fixed set of possible codes, the ultimate goal is to assign a set of codes to documents, based on their content. [sent-38, score-0.621]
22 Furthermore, we observe that for each code assigned to a document, there is generally at least one corresponding trigger term in the text that accounts for the code’s assignment. [sent-39, score-0.823]
23 For example, if an ICD-9-CM coding professional were to see “allergic bronchitis” somewhere in a clinical narrative, he or she would immediately consider adding code 493. [sent-40, score-0.947]
24 The presence of these trigger terms separates document coding from text classification tasks, such as topic or genre classification, where evidence for a particular label is built up throughout a document. [sent-42, score-0.884]
25 However, this does not make document coding a term recognition task, concerned only with the detection of triggers. [sent-43, score-0.528]
26 Codes are assigned to a document as a whole, and code assignment decisions within a document may interact. [sent-44, score-0.815]
27 Formally, we define the document coding task as follows: given a set of documents X and a set of available codes C, assign to each document xi a subset of codes Ci ⊂ C. [sent-46, score-1.828]
28 In particular, we will assume that an (incomplete) dictionary D(c) exists for each code c ∈ C, which lists specific code terms asso743 ciated with c. [sent-49, score-1.023]
29 Each code can have several corresponding terms while each term indicates the presence of exactly one code. [sent-52, score-0.586]
30 A candidate code c is proposed each time a term from D(c) is found in a document. [sent-53, score-0.663]
31 1 From triggers to codes The presence of a term from D(c) does not automatically imply the assignment of code c to a document. [sent-55, score-1.281]
32 Even with extremely precise dictionaries, there are three main reasons why a candidate code may not appear in a document’s code subset. [sent-56, score-1.072]
33 The context of the trigger term might indicate the irrelevancy of the code. [sent-58, score-0.437]
34 In the clinical domain, such irrelevancy can be specified by a negative or speculative statement (e. [sent-59, score-0.327]
35 There can be several closely related candidate codes; yet only one, the best fitted code should be assigned to the document. [sent-66, score-0.621]
36 00) may both appear in the same clinical report, but only the most specific code, 789. [sent-69, score-0.242]
37 For example, the ICD-9-CM coding rules state that no symptom codes should be given to a document if a definite diagnosis is present. [sent-73, score-1.123]
38 On the other hand, if the diagnosis is uncertain, then codes for the symptoms should be assigned. [sent-75, score-0.656]
39 This suggests a paradigm where suggested by a detected trigger in terms of both its local context presence of other candidate codes a candidate code, term, is assessed (item 1) and the for the document (items 2 and 3). [sent-76, score-1.551]
40 1Note that dictionary-based trigger detection could be replaced by tagging approaches similar to those used in namedentity-recognition or information extraction. [sent-77, score-0.335]
41 2 ICD-9-CM Coding As a specific application we have chosen the task of assigning ICD-9-CM codes to free-form clinical narratives. [sent-79, score-0.819]
42 The reports were annotated with ICD-9-CM codes by three coding companies, and the majority codes were selected as a gold standard. [sent-83, score-1.469]
43 In particular, the ICD-9-CM coding guidelines come with an index file that contains hundreds of thousands of terms mapped to corresponding codes. [sent-87, score-0.368]
44 As mentioned above, the ICD-9-CM coding rules create strong code dependencies: codes are assigned to a document as a set and not individually. [sent-93, score-1.516]
45 Furthermore, the code distribution throughout the CMC training documents has a very heavy tail; that is, there are a few heavily-used codes and a large number of codes that are used only occasionally. [sent-94, score-1.716]
46 3 Related work Automated clinical coding has received much attention in the medical informatics literature. [sent-96, score-0.603]
47 reviewed 113 studies on automated coding published in the last 40 years (Stanfill et al. [sent-98, score-0.293]
48 The authors conclude that there exists a variety of tools 744 covering different purposes, healthcare specialties, and clinical document types; however, these tools are not generalizable and neither are their evaluation results. [sent-100, score-0.468]
49 This approach is feasible for the small code set used in the challenge, but it is questionable in real-life settings where thousands of codes need to be considered. [sent-114, score-1.047]
50 Strong competition systems had good answers for dealing with negative and speculative contexts, taking advantage of the competition’s limited set of possible code combinations, and handling of low-frequency codes. [sent-123, score-0.542]
51 We combine a symbolic component that matches lexical strings of a document against a medical dictionary to determine possible codes (Lussier et al. [sent-125, score-1.018]
52 , 2000; Kevers and Medori, 2010) and a statistical component that finalizes the assignment of codes to the document. [sent-126, score-0.654]
53 (2007), in that we train a single model for all codes with codespecific and generic features. [sent-128, score-0.635]
54 (2007) did not employ our lexical trigger step or our sequence-modeling formulation. [sent-130, score-0.293]
55 In fact, they considered all possible code subsets, which can be infeasible in real-life settings. [sent-131, score-0.45]
56 Lexically match text to the dictionary to get a set of candidate codes; 2. [sent-133, score-0.238]
57 Using features derived from the candidates and the document, select the best code subset. [sent-134, score-0.517]
58 In the first stage, dictionary terms are detected in the document using exact string matching. [sent-135, score-0.362]
59 All codes corresponding to matches become candidate codes, and no other codes can be proposed for this document. [sent-136, score-1.346]
60 In the second stage, a single classifier is trained to select the best code subset from the matched candidates. [sent-137, score-0.472]
61 The LT-HMM allows features learned from a document coded with ci to transfer at test time to predict code cj, provided their respective triggers appear in similar contexts. [sent-140, score-0.846]
62 Training one common classifier improves our chances to reliably predict codes that have few training instances, and even codes that do not appear at all in the training data. [sent-141, score-1.304]
63 1 Trigger Detection We have manually assembled a dictionary of terms for each of the 45 codes used in the CMC challenge. [sent-143, score-0.72]
64 2 The dictionaries were built by collecting relevant medical terminology from UMLS, the ICD-9CM coding guidelines, and the CMC training data. [sent-144, score-0.454]
65 If a trigger term is missing from the dictionary and, as the result, the code is not selected as a candidate code, it will not be recovered in the following stage, resulting in a false negative. [sent-156, score-1.042]
66 2 Sequence Construction After trigger detection, we view the input document as a sequence of candidate codes, each correspond- ing to a detected trigger (see Figure 1). [sent-162, score-1.044]
67 By tagging these candidates in sequence, we can label each candidate code as present or absent and use previous tagging decisions to model code interactions. [sent-163, score-1.189]
68 The final code subset is constructed by collecting all candidate codes tagged as present. [sent-164, score-1.199]
69 Our training data consists of [document, code set] pairs, augmented with the trigger terms detected through dictionary matching. [sent-165, score-0.975]
70 We transform this into a sequence to be tagged using the following steps: Ordering: The candidate code sequence is presented in reverse chronological order, according to when their corresponding trigger terms appear in the document. [sent-166, score-1.04]
71 That is, the last candidate to be detected by the dictionary will be the first code to appear in our candidate sequence. [sent-167, score-0.944]
72 Reverse order was chosen because clinical documents often close with a final (and informative) diagnosis. [sent-168, score-0.247]
73 Merging: Each detected trigger corresponds to exactly one code; however, several triggers may be detected for the same code throughout a document. [sent-169, score-1.032]
74 If a code has several triggers, we keep only the last occurrence. [sent-170, score-0.45]
75 Labelling: Each candidate code is assigned a binary label (present or absent) based on whether it appears in the gold-standard code set. [sent-172, score-1.111]
76 The top binary layer is the correct output tag sequence, which confirms or rejects the presence of candidate codes. [sent-174, score-0.295]
77 The bottom layer shows the candidate code sequence derived from the text, with corresponding trigger phrases and some prominent features. [sent-175, score-0.96]
78 process can not introduce gold-standard codes that were not proposed by the dictionary. [sent-176, score-0.597]
79 To the left, we have an input text with underlined trigger phrases, as detected by our dictionary. [sent-178, score-0.4]
80 This implies an input sequence (bottom right), which consists of detected codes and their corresponding trigger phrases. [sent-179, score-1.041]
81 The gold-standard code set for the document is used to infer a gold-standard label sequence for these codes (top right). [sent-180, score-1.266]
82 A discriminative HMM has two major categories of features: emission features, which characterize a candidate’s tag in terms of the input document x, and transition features, which characterize a tag in terms of the tags that have come before it. [sent-189, score-0.454]
83 Transition Features The transition features are modeled as simple indicators over n-grams of present codes, for values of n up to 10, the largest number of codes proposed by 746 our dictionary in the training set. [sent-192, score-0.789]
84 3 This allows the system to learn sequences of codes that are (and are not) likely to occur in the gold-standard data. [sent-193, score-0.618]
85 We found it useful to pad our n-grams with “beginning of document” tokens for sequences when fewer than n codes have been labelled as present, but found it harmful to include an end-of-document tag once labelling is complete. [sent-194, score-0.678]
86 Emission Features The vast majority of our training signal comes from emission features, which carefully model both the trigger term’s local context and the document as a whole. [sent-196, score-0.558]
87 For each candidate code, three types of features are generated: document features, ConText features, and code-semantics features (Table 1). [sent-197, score-0.393]
88 These n-gram features have the candidate code appended to them, making them similar to features traditionally used in multiclass document categorization. [sent-199, score-0.863]
89 ConText is publicly available software that determines the presence of negated, hypothetical, historical, and family-related context for a given phrase in a clinical text (Harkema et al. [sent-201, score-0.303]
90 Regardless of its simplicity, the algorithm has shown very good performance on a variety of clinical document types. [sent-212, score-0.377]
91 We run ConText for each trigger term located in the text and produce two types of features: features related to the candidate code in question and features related to other candidate codes of the document. [sent-213, score-1.791]
92 Code Semantics: We include features that indicate if the code itself corresponds to a disease or a symptom. [sent-217, score-0.493]
93 Like the ConText features, code features come in two types: those regarding the candidate code in question and those regarding other candidate codes from the same document. [sent-219, score-1.874]
94 Generic features are concerned with classifying any candidate as present or absent based on characteristics of its trigger or semantics. [sent-221, score-0.552]
95 Code-specific features append the candidate code to the feature. [sent-222, score-0.645]
96 For example, the feature context=po s represents that the current candidate has a trigger term in a positive context, while context=pos : 4 8 6 adds the information that the code in question is 4 8 6. [sent-223, score-0.974]
97 Note that n-grams features are only code-specific, as they are not connected to any specific trigger term. [sent-224, score-0.336]
98 To an extent, code-specific features allow us to replicate the traditional classification approach, which focuses on one code at a time. [sent-225, score-0.513]
99 Using these features, the classifier is free to build complex submodels for a particular code, provided that this code has enough training examples. [sent-226, score-0.497]
100 In this way, even in the extreme case of having zero training examples for a particular code, the model can still potentially assign the code to new documents, provided it is detected by our dictionary. [sent-228, score-0.583]
wordName wordTfidf (topN-words)
[('codes', 0.597), ('code', 0.45), ('trigger', 0.293), ('coding', 0.275), ('clinical', 0.222), ('document', 0.155), ('candidate', 0.152), ('medical', 0.106), ('cmc', 0.099), ('triggers', 0.099), ('healthcare', 0.091), ('dictionary', 0.086), ('detected', 0.084), ('term', 0.061), ('diagnosis', 0.059), ('symbolic', 0.053), ('umls', 0.052), ('absent', 0.045), ('documentation', 0.045), ('challenge', 0.044), ('sequence', 0.044), ('context', 0.043), ('features', 0.043), ('emission', 0.042), ('allergic', 0.04), ('bronchitis', 0.04), ('irrelevancy', 0.04), ('pneumonia', 0.04), ('stanfill', 0.04), ('suominen', 0.04), ('presence', 0.038), ('coded', 0.038), ('hmm', 0.038), ('generic', 0.038), ('terms', 0.037), ('tag', 0.037), ('wt', 0.036), ('assignment', 0.036), ('speculative', 0.035), ('administrative', 0.035), ('organizations', 0.034), ('professionals', 0.032), ('obstacle', 0.03), ('dictionaries', 0.03), ('procedures', 0.03), ('negative', 0.03), ('come', 0.03), ('pain', 0.027), ('rejects', 0.027), ('competition', 0.027), ('crammer', 0.027), ('patient', 0.026), ('hypothetical', 0.026), ('guidelines', 0.026), ('negated', 0.025), ('training', 0.025), ('documents', 0.025), ('teams', 0.025), ('assign', 0.024), ('goldstein', 0.024), ('separates', 0.024), ('candidates', 0.024), ('tagging', 0.024), ('margin', 0.024), ('input', 0.023), ('labelling', 0.023), ('ci', 0.023), ('classifier', 0.022), ('throughout', 0.022), ('narrative', 0.022), ('sequences', 0.021), ('component', 0.021), ('layer', 0.021), ('transition', 0.02), ('classification', 0.02), ('label', 0.02), ('appear', 0.02), ('versions', 0.02), ('binary', 0.02), ('rules', 0.02), ('traditionally', 0.02), ('assigned', 0.019), ('gradient', 0.019), ('concerned', 0.019), ('stage', 0.018), ('automated', 0.018), ('positive', 0.018), ('characterize', 0.018), ('predict', 0.018), ('detection', 0.018), ('terminology', 0.018), ('indicators', 0.018), ('alterations', 0.017), ('odfe', 0.017), ('radiology', 0.017), ('symptom', 0.017), ('afford', 0.017), ('stimulate', 0.017), ('kiritchenko', 0.017), ('meystre', 0.017)]
simIndex simValue paperId paperTitle
same-paper 1 1.0000001 209 acl-2011-Lexically-Triggered Hidden Markov Models for Clinical Document Coding
Author: Svetlana Kiritchenko ; Colin Cherry
Abstract: The automatic coding of clinical documents is an important task for today’s healthcare providers. Though it can be viewed as multi-label document classification, the coding problem has the interesting property that most code assignments can be supported by a single phrase found in the input document. We propose a Lexically-Triggered Hidden Markov Model (LT-HMM) that leverages these phrases to improve coding accuracy. The LT-HMM works in two stages: first, a lexical match is performed against a term dictionary to collect a set of candidate codes for a document. Next, a discriminative HMM selects the best subset of codes to assign to the document by tagging candidates as present or absent. By confirming codes proposed by a dictionary, the LT-HMM can share features across codes, enabling strong performance even on rare codes. In fact, we are able to recover codes that do not occur in the training set at all. Our approach achieves the best ever performance on the 2007 Medical NLP Challenge test set, with an F-measure of 89.84.
Author: Deyi Xiong ; Min Zhang ; Haizhou Li
Abstract: In this paper, with a belief that a language model that embraces a larger context provides better prediction ability, we present two extensions to standard n-gram language models in statistical machine translation: a backward language model that augments the conventional forward language model, and a mutual information trigger model which captures long-distance dependencies that go beyond the scope of standard n-gram language models. We integrate the two proposed models into phrase-based statistical machine translation and conduct experiments on large-scale training data to investigate their effectiveness. Our experimental results show that both models are able to significantly improve transla- , tion quality and collectively achieve up to 1 BLEU point over a competitive baseline.
3 0.14954014 165 acl-2011-Improving Classification of Medical Assertions in Clinical Notes
Author: Youngjun Kim ; Ellen Riloff ; Stephane Meystre
Abstract: We present an NLP system that classifies the assertion type of medical problems in clinical notes used for the Fourth i2b2/VA Challenge. Our classifier uses a variety of linguistic features, including lexical, syntactic, lexicosyntactic, and contextual features. To overcome an extremely unbalanced distribution of assertion types in the data set, we focused our efforts on adding features specifically to improve the performance of minority classes. As a result, our system reached 94. 17% micro-averaged and 79.76% macro-averaged F1-measures, and showed substantial recall gains on the minority classes. 1
4 0.14589064 260 acl-2011-Recognizing Authority in Dialogue with an Integer Linear Programming Constrained Model
Author: Elijah Mayfield ; Carolyn Penstein Rose
Abstract: We present a novel computational formulation of speaker authority in discourse. This notion, which focuses on how speakers position themselves relative to each other in discourse, is first developed into a reliable coding scheme (0.71 agreement between human annotators). We also provide a computational model for automatically annotating text using this coding scheme, using supervised learning enhanced by constraints implemented with Integer Linear Programming. We show that this constrained model’s analyses of speaker authority correlates very strongly with expert human judgments (r2 coefficient of 0.947).
5 0.14313932 65 acl-2011-Can Document Selection Help Semi-supervised Learning? A Case Study On Event Extraction
Author: Shasha Liao ; Ralph Grishman
Abstract: Annotating training data for event extraction is tedious and labor-intensive. Most current event extraction tasks rely on hundreds of annotated documents, but this is often not enough. In this paper, we present a novel self-training strategy, which uses Information Retrieval (IR) to collect a cluster of related documents as the resource for bootstrapping. Also, based on the particular characteristics of this corpus, global inference is applied to provide more confident and informative data selection. We compare this approach to self-training on a normal newswire corpus and show that IR can provide a better corpus for bootstrapping and that global inference can further improve instance selection. We obtain gains of 1.7% in trigger labeling and 2.3% in role labeling through IR and an additional 1.1% in trigger labeling and 1.3% in role labeling by applying global inference. 1
6 0.074800156 52 acl-2011-Automatic Labelling of Topic Models
7 0.07251855 135 acl-2011-Faster and Smaller N-Gram Language Models
8 0.071703561 328 acl-2011-Using Cross-Entity Inference to Improve Event Extraction
9 0.069556572 139 acl-2011-From Bilingual Dictionaries to Interlingual Document Representations
10 0.053891875 117 acl-2011-Entity Set Expansion using Topic information
11 0.053092066 128 acl-2011-Exploring Entity Relations for Named Entity Disambiguation
12 0.051075257 104 acl-2011-Domain Adaptation for Machine Translation by Mining Unseen Words
13 0.050810002 50 acl-2011-Automatic Extraction of Lexico-Syntactic Patterns for Detection of Negation and Speculation Scopes
14 0.049245819 11 acl-2011-A Fast and Accurate Method for Approximate String Search
15 0.048877824 150 acl-2011-Hierarchical Text Classification with Latent Concepts
16 0.04843723 158 acl-2011-Identification of Domain-Specific Senses in a Machine-Readable Dictionary
17 0.048048254 293 acl-2011-Template-Based Information Extraction without the Templates
18 0.047912288 251 acl-2011-Probabilistic Document Modeling for Syntax Removal in Text Summarization
19 0.047911048 140 acl-2011-Fully Unsupervised Word Segmentation with BVE and MDL
20 0.04775463 132 acl-2011-Extracting Paraphrases from Definition Sentences on the Web
topicId topicWeight
[(0, 0.134), (1, 0.039), (2, -0.063), (3, 0.029), (4, 0.017), (5, 0.032), (6, -0.025), (7, 0.012), (8, 0.049), (9, 0.024), (10, -0.046), (11, 0.012), (12, 0.025), (13, 0.032), (14, 0.03), (15, 0.018), (16, 0.005), (17, -0.001), (18, 0.019), (19, -0.041), (20, 0.073), (21, -0.075), (22, 0.056), (23, -0.036), (24, -0.043), (25, 0.038), (26, 0.022), (27, 0.052), (28, -0.072), (29, 0.021), (30, 0.073), (31, 0.022), (32, 0.128), (33, 0.043), (34, -0.14), (35, 0.05), (36, 0.013), (37, -0.06), (38, 0.12), (39, 0.002), (40, 0.035), (41, 0.062), (42, 0.012), (43, -0.005), (44, 0.026), (45, -0.079), (46, -0.013), (47, -0.002), (48, -0.098), (49, 0.196)]
simIndex simValue paperId paperTitle
same-paper 1 0.94504887 209 acl-2011-Lexically-Triggered Hidden Markov Models for Clinical Document Coding
Author: Svetlana Kiritchenko ; Colin Cherry
Abstract: The automatic coding of clinical documents is an important task for today’s healthcare providers. Though it can be viewed as multi-label document classification, the coding problem has the interesting property that most code assignments can be supported by a single phrase found in the input document. We propose a Lexically-Triggered Hidden Markov Model (LT-HMM) that leverages these phrases to improve coding accuracy. The LT-HMM works in two stages: first, a lexical match is performed against a term dictionary to collect a set of candidate codes for a document. Next, a discriminative HMM selects the best subset of codes to assign to the document by tagging candidates as present or absent. By confirming codes proposed by a dictionary, the LT-HMM can share features across codes, enabling strong performance even on rare codes. In fact, we are able to recover codes that do not occur in the training set at all. Our approach achieves the best ever performance on the 2007 Medical NLP Challenge test set, with an F-measure of 89.84.
2 0.6998021 165 acl-2011-Improving Classification of Medical Assertions in Clinical Notes
Author: Youngjun Kim ; Ellen Riloff ; Stephane Meystre
Abstract: We present an NLP system that classifies the assertion type of medical problems in clinical notes used for the Fourth i2b2/VA Challenge. Our classifier uses a variety of linguistic features, including lexical, syntactic, lexicosyntactic, and contextual features. To overcome an extremely unbalanced distribution of assertion types in the data set, we focused our efforts on adding features specifically to improve the performance of minority classes. As a result, our system reached 94. 17% micro-averaged and 79.76% macro-averaged F1-measures, and showed substantial recall gains on the minority classes. 1
Author: Deyi Xiong ; Min Zhang ; Haizhou Li
Abstract: In this paper, with a belief that a language model that embraces a larger context provides better prediction ability, we present two extensions to standard n-gram language models in statistical machine translation: a backward language model that augments the conventional forward language model, and a mutual information trigger model which captures long-distance dependencies that go beyond the scope of standard n-gram language models. We integrate the two proposed models into phrase-based statistical machine translation and conduct experiments on large-scale training data to investigate their effectiveness. Our experimental results show that both models are able to significantly improve transla- , tion quality and collectively achieve up to 1 BLEU point over a competitive baseline.
4 0.5215987 80 acl-2011-ConsentCanvas: Automatic Texturing for Improved Readability in End-User License Agreements
Author: Oliver Schneider ; Alex Garnett
Abstract: We present ConsentCanvas, a system which structures and “texturizes” End-User License Agreement (EULA) documents to be more readable. The system aims to help users better understand the terms under which they are providing their informed consent. ConsentCanvas receives unstructured text documents as input and uses unsupervised natural language processing methods to embellish the source document using a linked stylesheet. Unlike similar usable security projects which employ summarization techniques, our system preserves the contents of the source document, minimizing the cognitive and legal burden for both the end user and the licensor. Our system does not require a corpus for training. 1
5 0.4682588 42 acl-2011-An Interface for Rapid Natural Language Processing Development in UIMA
Author: Balaji Soundrarajan ; Thomas Ginter ; Scott DuVall
Abstract: This demonstration presents the Annotation Librarian, an application programming interface that supports rapid development of natural language processing (NLP) projects built in Apache Unstructured Information Management Architecture (UIMA). The flexibility of UIMA to support all types of unstructured data – images, audio, and text – increases the complexity of some of the most common NLP development tasks. The Annotation Librarian interface handles these common functions and allows the creation and management of annotations by mirroring Java methods used to manipulate Strings. The familiar syntax and NLP-centric design allows developers to adopt and rapidly develop NLP algorithms in UIMA. The general functionality of the interface is described in relation to the use cases that necessitated its creation. 1
6 0.45718977 50 acl-2011-Automatic Extraction of Lexico-Syntactic Patterns for Detection of Negation and Speculation Scopes
7 0.45430779 74 acl-2011-Combining Indicators of Allophony
9 0.41836411 301 acl-2011-The impact of language models and loss functions on repair disfluency detection
10 0.41823792 135 acl-2011-Faster and Smaller N-Gram Language Models
11 0.41292563 24 acl-2011-A Scalable Probabilistic Classifier for Language Modeling
12 0.41227716 199 acl-2011-Learning Condensed Feature Representations from Large Unsupervised Data Sets for Supervised Learning
13 0.40853518 278 acl-2011-Semi-supervised condensed nearest neighbor for part-of-speech tagging
14 0.4042722 65 acl-2011-Can Document Selection Help Semi-supervised Learning? A Case Study On Event Extraction
15 0.4033381 273 acl-2011-Semantic Representation of Negation Using Focus Detection
16 0.4022561 11 acl-2011-A Fast and Accurate Method for Approximate String Search
17 0.3997803 8 acl-2011-A Corpus of Scope-disambiguated English Text
18 0.38674396 260 acl-2011-Recognizing Authority in Dialogue with an Integer Linear Programming Constrained Model
19 0.38592121 139 acl-2011-From Bilingual Dictionaries to Interlingual Document Representations
20 0.38427705 125 acl-2011-Exploiting Readymades in Linguistic Creativity: A System Demonstration of the Jigsaw Bard
topicId topicWeight
[(5, 0.058), (17, 0.056), (26, 0.034), (31, 0.013), (37, 0.066), (39, 0.071), (41, 0.097), (46, 0.234), (55, 0.042), (59, 0.03), (61, 0.013), (72, 0.037), (91, 0.039), (96, 0.102), (97, 0.013)]
simIndex simValue paperId paperTitle
1 0.82014781 297 acl-2011-That's What She Said: Double Entendre Identification
Author: Chloe Kiddon ; Yuriy Brun
Abstract: Humor identification is a hard natural language understanding problem. We identify a subproblem — the “that’s what she said” problem with two distinguishing characteristics: (1) use of nouns that are euphemisms for sexually explicit nouns and (2) structure common in the erotic domain. We address this problem in a classification approach that includes features that model those two characteristics. Experiments on web data demonstrate that our approach improves precision by 12% over baseline techniques that use only word-based features. —
same-paper 2 0.78697312 209 acl-2011-Lexically-Triggered Hidden Markov Models for Clinical Document Coding
Author: Svetlana Kiritchenko ; Colin Cherry
Abstract: The automatic coding of clinical documents is an important task for today’s healthcare providers. Though it can be viewed as multi-label document classification, the coding problem has the interesting property that most code assignments can be supported by a single phrase found in the input document. We propose a Lexically-Triggered Hidden Markov Model (LT-HMM) that leverages these phrases to improve coding accuracy. The LT-HMM works in two stages: first, a lexical match is performed against a term dictionary to collect a set of candidate codes for a document. Next, a discriminative HMM selects the best subset of codes to assign to the document by tagging candidates as present or absent. By confirming codes proposed by a dictionary, the LT-HMM can share features across codes, enabling strong performance even on rare codes. In fact, we are able to recover codes that do not occur in the training set at all. Our approach achieves the best ever performance on the 2007 Medical NLP Challenge test set, with an F-measure of 89.84.
3 0.7044763 238 acl-2011-P11-2093 k2opt.pdf
Author: empty-author
Abstract: We present a pointwise approach to Japanese morphological analysis (MA) that ignores structure information during learning and tagging. Despite the lack of structure, it is able to outperform the current state-of-the-art structured approach for Japanese MA, and achieves accuracy similar to that of structured predictors using the same feature set. We also find that the method is both robust to outof-domain data, and can be easily adapted through the use of a combination of partial annotation and active learning.
4 0.59940439 316 acl-2011-Unary Constraints for Efficient Context-Free Parsing
Author: Nathan Bodenstab ; Kristy Hollingshead ; Brian Roark
Abstract: We present a novel pruning method for context-free parsing that increases efficiency by disallowing phrase-level unary productions in CKY chart cells spanning a single word. Our work is orthogonal to recent work on “closing” chart cells, which has focused on multi-word constituents, leaving span-1 chart cells unpruned. We show that a simple discriminative classifier can learn with high accuracy which span-1 chart cells to close to phrase-level unary productions. Eliminating these unary productions from the search can have a large impact on downstream processing, depending on implementation details of the search. We apply our method to four parsing architectures and demonstrate how it is complementary to the cell-closing paradigm, as well as other pruning methods such as coarse-to-fine, agenda, and beam-search pruning.
5 0.59849274 58 acl-2011-Beam-Width Prediction for Efficient Context-Free Parsing
Author: Nathan Bodenstab ; Aaron Dunlop ; Keith Hall ; Brian Roark
Abstract: Efficient decoding for syntactic parsing has become a necessary research area as statistical grammars grow in accuracy and size and as more NLP applications leverage syntactic analyses. We review prior methods for pruning and then present a new framework that unifies their strengths into a single approach. Using a log linear model, we learn the optimal beam-search pruning parameters for each CYK chart cell, effectively predicting the most promising areas of the model space to explore. We demonstrate that our method is faster than coarse-to-fine pruning, exemplified in both the Charniak and Berkeley parsers, by empirically comparing our parser to the Berkeley parser using the same grammar and under identical operating conditions.
6 0.59386265 119 acl-2011-Evaluating the Impact of Coder Errors on Active Learning
7 0.59197074 65 acl-2011-Can Document Selection Help Semi-supervised Learning? A Case Study On Event Extraction
8 0.59033298 196 acl-2011-Large-Scale Cross-Document Coreference Using Distributed Inference and Hierarchical Models
9 0.58871919 126 acl-2011-Exploiting Syntactico-Semantic Structures for Relation Extraction
10 0.5858708 145 acl-2011-Good Seed Makes a Good Crop: Accelerating Active Learning Using Language Modeling
11 0.58537215 202 acl-2011-Learning Hierarchical Translation Structure with Linguistic Annotations
12 0.58527631 324 acl-2011-Unsupervised Semantic Role Induction via Split-Merge Clustering
13 0.58466941 269 acl-2011-Scaling up Automatic Cross-Lingual Semantic Role Annotation
14 0.58404076 137 acl-2011-Fine-Grained Class Label Markup of Search Queries
15 0.58276457 135 acl-2011-Faster and Smaller N-Gram Language Models
17 0.58186299 300 acl-2011-The Surprising Variance in Shortest-Derivation Parsing
18 0.58142948 246 acl-2011-Piggyback: Using Search Engines for Robust Cross-Domain Named Entity Recognition
20 0.58049572 236 acl-2011-Optimistic Backtracking - A Backtracking Overlay for Deterministic Incremental Parsing