acl acl2013 acl2013-193 knowledge-graph by maker-knowledge-mining

193 acl-2013-Improving Chinese Word Segmentation on Micro-blog Using Rich Punctuations

Source: pdf

Author: Longkai Zhang ; Li Li ; Zhengyan He ; Houfeng Wang ; Ni Sun

Abstract: Micro-blog is a new kind of medium which is short and informal. While no segmented corpus of micro-blogs is available to train Chinese word segmentation model, existing Chinese word segmentation tools cannot perform equally well as in ordinary news texts. In this paper we present an effective yet simple approach to Chinese word segmentation of micro-blog. In our approach, we incorporate punctuation information of unlabeled micro-blog data by introducing characters behind or ahead of punctuations, for they indicate the beginning or end of words. Meanwhile a self-training framework to incorporate confident instances is also used, which prove to be helpful. Ex- periments on micro-blog data show that our approach improves performance, especially in OOV-recall. 1 INTRODUCTION Micro-blog (also known as tweets in English) is a new kind of broadcast medium in the form of blogging. A micro-blog differs from a traditional blog in that it is typically smaller in size. Furthermore, texts in micro-blogs tend to be informal and new words occur more frequently. These new features of micro-blogs make the Chinese Word Segmentation (CWS) models trained on the source domain, such as news corpus, fail to perform equally well when transferred to texts from micro-blogs. For example, the most widely used Chinese segmenter ”ICTCLAS” yields 0.95 f-score in news corpus, only gets 0.82 f-score on micro-blog data. The poor segmentation results will hurt subsequent analysis on micro-blog text. ∗Corresponding author Manually labeling the texts of micro-blog is time consuming. Luckily, punctuations provide useful information because they are used as indicators of the end of previous sentence and the beginning of the next one, which also indicate the start and the end of a word. These ”natural boundaries” appear so frequently in micro-blog texts that we can easily make good use of them. TABLE 1 shows some statistics of the news corpus vs. the micro-blogs. Besides, English letters and digits are also more than those in news corpus. They all are natural delimiters of Chinese characters and we treat them just the same as punctuations. We propose a method to enlarge the training corpus by using punctuation information. We build a semi-supervised learning (SSL) framework which can iteratively incorporate newly labeled instances from unlabeled micro-blog data during the training process. We test our method on microblog texts and experiments show good results. This paper is organized as follows. In section 1 we introduce the problem. Section 2 gives detailed description of our approach. We show the experi- ment and analyze the results in section 3. Section 4 gives the related works and in section 5 we conclude the whole work. 2 Our method 2.1 Punctuations Chinese word segmentation problem might be treated as a character labeling problem which gives each character a label indicating its position in one word. To be simple, one can use label ’B’ to indicate a character is the beginning of a word, and use ’N’ to indicate a character is not the beginning of a word. We also use the 2-tag in our work. Other tag sets like the ’BIES’ tag set are not suiteable because the puctuation information cannot decide whether a character after punctuation should be labeled as ’B’ or ’S’(word with Single 177 ProceedingSsof oifa, th Beu 5l1gsarti Aan,An uuaglu Mste 4e-ti9n2g 0 o1f3 t.he ?c A2s0s1o3ci Aatsiosonc fioartio Cno fmorpu Ctoamtiopnuatalt Lioinngauli Lsitnicgsu,i psatgicess 177–182, micNreow-bslogC68h56i. n73e%%seE10n1.g6.8l%i%shN20u. m76%%berPu1n13c9.t u03a%%tion Table 1: Percentage of Chinese, English, number, punctuation in the news corpus vs. the micro-blogs. character). Punctuations can serve as implicit labels for the characters before and after them. The character right after punctuations must be the first character of a word, meanwhile the character right before punctuations must be the last character of a word. An example is given in TABLE 2. 2.2 Algorithm Our algorithm “ADD-N” is shown in TABLE 3. The initially selected character instances are those right after punctuations. By definition they are all labeled with ’B’ . In this case, the number of training instances with label ’B’ is increased while the number with label ’N’ remains unchanged. Because of this, the model trained on this unbalanced corpus tends to be biased. This problem can become even worse when there is inexhaustible supply of texts from the target domain. We assume that labeled corpus of the source domain can be treated as a balanced reflection of different labels. Therefore we choose to estimate the balanced point by counting characters labeling ’B’ and ’N’ and calculate the ratio which we denote as η . We assume the enlarged corpus is also balanced if and only if the ratio of ’B’ to ’N’ is just the same to η of the source domain. Our algorithm uses data from source domain to make the labels balanced. When enlarging corpus using characters behind punctuations from texts in target domain, only characters labeling ’B’ are added. We randomly reuse some characters labeling ’N’ from labeled data until ratio η is reached. We do not use characters ahead of punctuations, because the single-character words ahead of punctuations take the label of ’B’ instead of ’N’ . In summary our algorithm tackles the problem by duplicating labeled data in source domain. We denote our algorithm as ”ADD-N”. We also use baseline feature templates include the features described in previous works (Sun and Xu, 2011; Sun et al., 2012). Our algorithm is not necessarily limited to a specific tagger. For simplicity and reliability, we use a simple MaximumEntropy tagger. 3 Experiment 3.1 Data set We evaluate our method using the data from weibo.com, which is the biggest micro-blog service in China. We use the API provided by weibo.com1 to crawl 500,000 micro-blog texts of weibo.com, which contains 24,243,772 characters. To keep the experiment tractable, we first randomly choose 50,000 of all the texts as unlabeled data, which contain 2,420,037 characters. We manually segment 2038 randomly selected microblogs.We follow the segmentation standard as the PKU corpus. In micro-blog texts, the user names and URLs have fixed format. User names start with ’ @ ’, followed by Chinese characters, English letters, numbers and ’ ’, and terminated when meeting punctuations or blanks. URLs also match fixed patterns, which are shortened using ”http : / /t . cn /” plus six random English letters or numbers. Thus user names and URLs can be pre-processed separately. We follow this principle in following experiments. We use the benchmark datasets provided by the second International Chinese Word Segmentation Bakeoff2 as the labeled data. We choose the PKU data in our experiment because our baseline methods use the same segmentation standard. We compare our method with three baseline methods. The first two are both famous Chinese word segmentation tools: ICTCLAS3 and Stanford Chinese word segmenter4, which are widely used in NLP related to word segmentation. Stanford Chinese word segmenter is a CRF-based segmentation tool and its segmentation standard is chosen as the PKU standard, which is the same to ours. ICTCLAS, on the other hand, is a HMMbased Chinese word segmenter. Another baseline is Li and Sun (2009), which also uses punctuation in their semi-supervised framework. F-score 1http : / / open . we ibo .com/wiki 2http : / /www . s ighan .org/bakeo f f2 0 0 5 / 3http : / / i c l .org/ ct as 4http : / / nlp . st an ford . edu /pro j ect s / chine s e-nlp . shtml \ # cws 178 评B论-是-风-格-，-评B论-是-能-力-。- BNBBNBBNBBNB Table 2: The first line represents the original text. The second line indicates whether each character is the Beginning of sentence. The third line is the tag sequence using ”BN” tag set. is used as the accuracy measure. The recall of out-of-vocabulary is also taken into consideration, which measures the ability of the model to correctly segment out of vocabulary words. 3.2 Main results methods on the development data. TABLE 4 summarizes the segmentation results. In TABLE 4, Li-Sun is the method in Li and Sun (2009). Maxent only uses the PKU data for training, with neither punctuation information nor self-training framework incorporated. The next 4 methods all require a 100 iteration of self-training. No-punc is the method that only uses self-training while no punctuation information is added. Nobalance is similar to ADD N. The only difference between No-balance and ADD-N is that the former does not balance label ’B’ and label ’N’ . The comparison of Maxent and No-punctuation shows that naively adding confident unlabeled instances does not guarantee to improve performance. The writing style and word formation of the source domain is different from target domain. When segmenting texts of the target domain using models trained on source domain, the performance will be hurt with more false segmented instances added into the training set. The comparison of Maxent, No-balance and ADD-N shows that considering punctuation as well as self-training does improve performance. Both the f-score and OOV-recall increase. By comparing No-balance and ADD-N alone we can find that we achieve relatively high f-score if we ignore tag balance issue, while slightly hurt the OOV-Recall. However, considering it will improve OOV-Recall by about +1.6% and the fscore +0.2%. We also experimented on different size of unlabeled data to evaluate the performance when adding unlabeled target domain data. TABLE 5 shows different f-scores and OOV-Recalls on different unlabeled data set. We note that when the number of texts changes from 0 to 50,000, the f-score and OOV both are improved. However, when unlabeled data changes to 200,000, the performance is a bit decreased, while still better than not using unlabeled data. This result comes from the fact that the method ’ADD-N’ only uses characters behind punctua179 Tabl152S0eiz 0:Segm0.8nP67ta245ion0p.8Rer6745f9om0a.8nF57c6e1witOh0 .d7Vi65f-2394Rernt size of unlabeled data tions from target domain. Taking more texts into consideration means selecting more characters labeling ’N’ from source domain to simulate those in target domain. If too many ’N’s are introduced, the training data will be biased against the true distribution of target domain. 3.3 Characters ahead of punctuations In the ”BN” tagging method mentioned above, we incorporate characters after punctuations from texts in micro-blog to enlarge training set.We also try an opposite approach, ”EN” tag, which uses ’E’ to represent ”End of word”, and ’N’ to rep- resent ”Not the end of word”. In this contrasting method, we only use charactersjust ahead ofpunctuations. We find that the two methods show similar results. Experiment results with ADD-N are shown in TABLE 6 . 5DU0an0lt a b0Tsiealzbe lde6:0.C8Fo7”m5BNpa”rO0itsOa.o7gVn7-3oRfBN0.8aFn”7E0dNEN”Ot0.aO.g7V6-3R 4 Related Work Recent studies show that character sequence labeling is an effective formulation of Chinese word segmentation (Low et al., 2005; Zhao et al., 2006a,b; Chen et al., 2006; Xue, 2003). These supervised methods show good results, however, are unable to incorporate information from new domain, where OOV problem is a big challenge for the research community. On the other hand unsupervised word segmentation Peng and Schuurmans (2001); Goldwater et al. (2006); Jin and Tanaka-Ishii (2006); Feng et al. (2004); Maosong et al. (1998) takes advantage of the huge amount of raw text to solve Chinese word segmentation problems. However, they usually are less accurate and more complicated than supervised ones. Meanwhile semi-supervised methods have been applied into NLP applications. Bickel et al. (2007) learns a scaling factor from data of source domain and use the distribution to resemble target domain distribution. Wu et al. (2009) uses a Domain adaptive bootstrapping (DAB) framework, which shows good results on Named Entity Recognition. Similar semi-supervised applications include Shen et al. (2004); Daum e´ III and Marcu (2006); Jiang and Zhai (2007); Weinberger et al. (2006). Besides, Sun and Xu (201 1) uses a sequence labeling framework, while unsupervised statistics are used as discrete features in their model, which prove to be effective in Chinese word segmentation. There are previous works using punctuations as implicit annotations. Riley (1989) uses it in sentence boundary detection. Li and Sun (2009) proposed a compromising solution to by using a clas- sifier to select the most confident characters. We do not follow this approach because the initial errors will dramatically harm the performance. Instead, we only add the characters after punctuations which are sure to be the beginning of words (which means labeling ’B’) into our training set. Sun and Xu (201 1) uses punctuation information as discrete feature in a sequence labeling framework, which shows improvement compared to the pure sequence labeling approach. Our method is different from theirs. We use characters after punctuations directly. 5 Conclusion In this paper we have presented an effective yet simple approach to Chinese word segmentation on micro-blog texts. In our approach, punctuation information of unlabeled micro-blog data is used, as well as a self-training framework to incorporate confident instances. Experiments show that our approach improves performance, especially in OOV-recall. Both the punctuation information and the self-training phase contribute to this improve- ment. Acknowledgments This research was partly supported by National High Technology Research and Development Program of China (863 Program) (No. 2012AA01 1101), National Natural Science Foundation of China (No.91024009) and Major National Social Science Fund of China(No. 12&ZD227;). 180 References Bickel, S., Br¨ uckner, M., and Scheffer, T. (2007). Discriminative learning for differing training and test distributions. In Proceedings ofthe 24th international conference on Machine learning, pages 81–88. ACM. Chen, W., Zhang, Y., and Isahara, H. (2006). Chinese named entity recognition with conditional random fields. In 5th SIGHAN Workshop on Chinese Language Processing, Australia. Daum e´ III, H. and Marcu, D. (2006). Domain adaptation for statistical classifiers. Journal of Artificial Intelligence Research, 26(1): 101–126. Feng, H., Chen, K., Deng, X., and Zheng, W. (2004). Accessor variety criteria for chinese word extraction. Computational Linguistics, 30(1):75–93. Goldwater, S., Griffiths, T., and Johnson, M. (2006). Contextual dependencies in unsupervised word segmentation. In Proceedings of the 21st International Conference on Computational Linguistics and the 44th annual meeting of the Association for Computational Linguistics, pages 673–680. Association for Computational Linguistics. Jiang, J. and Zhai, C. (2007). Instance weighting for domain adaptation in nlp. In Annual Meeting-Association For Computational Linguistics, volume 45, page 264. Jin, Z. and Tanaka-Ishii, K. (2006). Unsupervised segmentation of chinese text by use of branching entropy. In Proceedings of the COLING/ACL on Main conference poster sessions, pages 428–435. Association for Computational Linguistics. Li, Z. and Sun, M. (2009). Punctuation as implicit annotations for chinese word segmentation. Computational Linguistics, 35(4):505– 512. Low, J., Ng, H., and Guo, W. (2005). A maximum entropy approach to chinese word segmentation. In Proceedings of the Fourth SIGHAN Workshop on Chinese Language Processing, volume 164. Jeju Island, Korea. Maosong, S., Dayang, S., and Tsou, B. (1998). Chinese word segmentation without using lexicon and hand-crafted training data. In Proceedings of the 1 7th international conference on Computational linguistics-Volume 2, pages 1265–1271 . Association for Computational Linguistics. Pan, S. and Yang, Q. (2010). A survey on transfer learning. Knowledge and Data Engineering, IEEE Transactions on, 22(10): 1345–1359. Peng, F. and Schuurmans, D. (2001). Selfsupervised chinese word segmentation. Advances in Intelligent Data Analysis, pages 238– 247. Riley, M. (1989). Some applications of tree-based modelling to speech and language. In Proceedings of the workshop on Speech and Natural Language, pages 339–352. Association for Computational Linguistics. Shen, D., Zhang, J., Su, J., Zhou, G., and Tan, C. (2004). Multi-criteria-based active learning for named entity recognition. In Proceedings of the 42nd Annual Meeting on Association for Computational Linguistics, page 589. Association for Computational Linguistics. Sun, W. and Xu, J. (201 1). Enhancing chinese word segmentation using unlabeled data. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, pages 970–979. Association for Computational Linguistics. Sun, X., Wang, H., and Li, W. (2012). Fast online training with frequency-adaptive learning rates for chinese word segmentation and new word detection. In Proceedings of the 50th Annual Meeting of the Association for Compu- tational Linguistics (Volume 1: Long Papers), pages 253–262, Jeju Island, Korea. Association for Computational Linguistics. Weinberger, K., Blitzer, J., and Saul, L. (2006). Distance metric learning for large margin nearest neighbor classification. In In NIPS. Citeseer. Wu, D., Lee, W., Ye, N., and Chieu, H. (2009). Domain adaptive bootstrapping for named entity recognition. In Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing: Volume 3-Volume 3, pages 1523–1532. Association for Computational Linguistics. Xue, N. (2003). Chinese word segmentation as character tagging. Computational Linguistics and Chinese Language Processing, 8(1):29–48. Zhao, H., Huang, C., and Li, M. (2006a). An improved chinese word segmentation system with 181 conditional random field. In Proceedings of the Fifth SIGHAN Workshop on Chinese Language Processing, volume 117. Sydney: July. Zhao, H., Huang, C., Li, M., and Lu, B. (2006b). Effective tag set selection in chinese word segmentation via conditional random field modeling. In Proceedings pages of PACLIC, volume 20, 87–94. 182

Reference: text

Summary: the most important sentenses genereted by tfidf model

sentIndex sentText sentNum sentScore

1 Abstract Micro-blog is a new kind of medium which is short and informal. [sent-11, score-0.051]

2 While no segmented corpus of micro-blogs is available to train Chinese word segmentation model, existing Chinese word segmentation tools cannot perform equally well as in ordinary news texts. [sent-12, score-0.799]

3 In this paper we present an effective yet simple approach to Chinese word segmentation of micro-blog. [sent-13, score-0.354]

4 In our approach, we incorporate punctuation information of unlabeled micro-blog data by introducing characters behind or ahead of punctuations, for they indicate the beginning or end of words. [sent-14, score-0.896]

5 Meanwhile a self-training framework to incorporate confident instances is also used, which prove to be helpful. [sent-15, score-0.244]

6 1 INTRODUCTION Micro-blog (also known as tweets in English) is a new kind of broadcast medium in the form of blogging. [sent-17, score-0.051]

7 Furthermore, texts in micro-blogs tend to be informal and new words occur more frequently. [sent-19, score-0.114]

8 These new features of micro-blogs make the Chinese Word Segmentation (CWS) models trained on the source domain, such as news corpus, fail to perform equally well when transferred to texts from micro-blogs. [sent-20, score-0.204]

9 For example, the most widely used Chinese segmenter ”ICTCLAS” yields 0. [sent-21, score-0.045]

10 The poor segmentation results will hurt subsequent analysis on micro-blog text. [sent-24, score-0.392]

11 ∗Corresponding author Manually labeling the texts of micro-blog is time consuming. [sent-25, score-0.222]

12 Luckily, punctuations provide useful information because they are used as indicators of the end of previous sentence and the beginning of the next one, which also indicate the start and the end of a word. [sent-26, score-0.539]

13 These ”natural boundaries” appear so frequently in micro-blog texts that we can easily make good use of them. [sent-27, score-0.114]

14 TABLE 1 shows some statistics of the news corpus vs. [sent-28, score-0.052]

15 Besides, English letters and digits are also more than those in news corpus. [sent-30, score-0.103]

16 They all are natural delimiters of Chinese characters and we treat them just the same as punctuations. [sent-31, score-0.245]

17 We propose a method to enlarge the training corpus by using punctuation information. [sent-32, score-0.239]

18 We build a semi-supervised learning (SSL) framework which can iteratively incorporate newly labeled instances from unlabeled micro-blog data during the training process. [sent-33, score-0.326]

19 We test our method on microblog texts and experiments show good results. [sent-34, score-0.114]

20 1 Punctuations Chinese word segmentation problem might be treated as a character labeling problem which gives each character a label indicating its position in one word. [sent-41, score-0.92]

21 To be simple, one can use label ’B’ to indicate a character is the beginning of a word, and use ’N’ to indicate a character is not the beginning of a word. [sent-42, score-0.624]

22 Other tag sets like the ’BIES’ tag set are not suiteable because the puctuation information cannot decide whether a character after punctuation should be labeled as ’B’ or ’S’(word with Single 177 ProceedingSsof oifa, th Beu 5l1gsarti Aan,An uuaglu Mste 4e-ti9n2g 0 o1f3 t. [sent-44, score-0.578]

23 t u03a%%tion Table 1: Percentage of Chinese, English, number, punctuation in the news corpus vs. [sent-51, score-0.236]

24 Punctuations can serve as implicit labels for the characters before and after them. [sent-54, score-0.258]

25 The character right after punctuations must be the first character of a word, meanwhile the character right before punctuations must be the last character of a word. [sent-55, score-1.787]

26 The initially selected character instances are those right after punctuations. [sent-59, score-0.253]

27 In this case, the number of training instances with label ’B’ is increased while the number with label ’N’ remains unchanged. [sent-61, score-0.154]

28 Because of this, the model trained on this unbalanced corpus tends to be biased. [sent-62, score-0.032]

29 This problem can become even worse when there is inexhaustible supply of texts from the target domain. [sent-63, score-0.157]

30 We assume that labeled corpus of the source domain can be treated as a balanced reflection of different labels. [sent-64, score-0.268]

31 Therefore we choose to estimate the balanced point by counting characters labeling ’B’ and ’N’ and calculate the ratio which we denote as η . [sent-65, score-0.416]

32 We assume the enlarged corpus is also balanced if and only if the ratio of ’B’ to ’N’ is just the same to η of the source domain. [sent-66, score-0.167]

33 Our algorithm uses data from source domain to make the labels balanced. [sent-67, score-0.167]

34 When enlarging corpus using characters behind punctuations from texts in target domain, only characters labeling ’B’ are added. [sent-68, score-1.224]

35 We randomly reuse some characters labeling ’N’ from labeled data until ratio η is reached. [sent-69, score-0.403]

36 We do not use characters ahead of punctuations, because the single-character words ahead of punctuations take the label of ’B’ instead of ’N’ . [sent-70, score-1.013]

37 In summary our algorithm tackles the problem by duplicating labeled data in source domain. [sent-71, score-0.157]

38 To keep the experiment tractable, we first randomly choose 50,000 of all the texts as unlabeled data, which contain 2,420,037 characters. [sent-83, score-0.277]

39 In micro-blog texts, the user names and URLs have fixed format. [sent-86, score-0.038]

40 User names start with ’ @ ’, followed by Chinese characters, English letters, numbers and ’ ’, and terminated when meeting punctuations or blanks. [sent-87, score-0.526]

41 cn /” plus six random English letters or numbers. [sent-89, score-0.097]

42 Thus user names and URLs can be pre-processed separately. [sent-90, score-0.038]

43 We use the benchmark datasets provided by the second International Chinese Word Segmentation Bakeoff2 as the labeled data. [sent-92, score-0.047]

44 We choose the PKU data in our experiment because our baseline methods use the same segmentation standard. [sent-93, score-0.309]

45 The first two are both famous Chinese word segmentation tools: ICTCLAS3 and Stanford Chinese word segmenter4, which are widely used in NLP related to word segmentation. [sent-95, score-0.444]

46 Stanford Chinese word segmenter is a CRF-based segmentation tool and its segmentation standard is chosen as the PKU standard, which is the same to ours. [sent-96, score-0.708]

47 ICTCLAS, on the other hand, is a HMMbased Chinese word segmenter. [sent-97, score-0.045]

48 Another baseline is Li and Sun (2009), which also uses punctuation in their semi-supervised framework. [sent-98, score-0.222]

49 shtml \ # cws 178 评B论-是-风-格-，-评B论-是-能-力-。- BNBBNBBNBBNB Table 2: The first line represents the original text. [sent-107, score-0.061]

50 The second line indicates whether each character is the Beginning of sentence. [sent-108, score-0.203]

51 The third line is the tag sequence using ”BN” tag set. [sent-109, score-0.18]

52 The recall of out-of-vocabulary is also taken into consideration, which measures the ability of the model to correctly segment out of vocabulary words. [sent-111, score-0.035]

53 Maxent only uses the PKU data for training, with neither punctuation information nor self-training framework incorporated. [sent-116, score-0.222]

54 No-punc is the method that only uses self-training while no punctuation information is added. [sent-118, score-0.222]

55 The only difference between No-balance and ADD-N is that the former does not balance label ’B’ and label ’N’ . [sent-120, score-0.14]

56 The comparison of Maxent and No-punctuation shows that naively adding confident unlabeled instances does not guarantee to improve performance. [sent-121, score-0.307]

57 The writing style and word formation of the source domain is different from target domain. [sent-122, score-0.217]

58 When segmenting texts of the target domain using models trained on source domain, the performance will be hurt with more false segmented instances added into the training set. [sent-123, score-0.458]

59 The comparison of Maxent, No-balance and ADD-N shows that considering punctuation as well as self-training does improve performance. [sent-124, score-0.184]

60 By comparing No-balance and ADD-N alone we can find that we achieve relatively high f-score if we ignore tag balance issue, while slightly hurt the OOV-Recall. [sent-126, score-0.191]

61 We also experimented on different size of unlabeled data to evaluate the performance when adding unlabeled target domain data. [sent-130, score-0.46]

62 TABLE 5 shows different f-scores and OOV-Recalls on different unlabeled data set. [sent-131, score-0.163]

63 We note that when the number of texts changes from 0 to 50,000, the f-score and OOV both are improved. [sent-132, score-0.114]

64 However, when unlabeled data changes to 200,000, the performance is a bit decreased, while still better than not using unlabeled data. [sent-133, score-0.326]

65 This result comes from the fact that the method ’ADD-N’ only uses characters behind punctua179 Tabl152S0eiz 0:Segm0. [sent-134, score-0.291]

66 d7Vi65f-2394Rernt size of unlabeled data tions from target domain. [sent-138, score-0.206]

67 Taking more texts into consideration means selecting more characters labeling ’N’ from source domain to simulate those in target domain. [sent-139, score-0.642]

68 If too many ’N’s are introduced, the training data will be biased against the true distribution of target domain. [sent-140, score-0.043]

69 3 Characters ahead of punctuations In the ”BN” tagging method mentioned above, we incorporate characters after punctuations from texts in micro-blog to enlarge training set. [sent-142, score-1.505]

70 We also try an opposite approach, ”EN” tag, which uses ’E’ to represent ”End of word”, and ’N’ to rep- resent ”Not the end of word”. [sent-143, score-0.038]

71 In this contrasting method, we only use charactersjust ahead ofpunctuations. [sent-144, score-0.179]

72 g7V6-3R 4 Related Work Recent studies show that character sequence labeling is an effective formulation of Chinese word segmentation (Low et al. [sent-152, score-0.701]

73 These supervised methods show good results, however, are unable to incorporate information from new domain, where OOV problem is a big challenge for the research community. [sent-156, score-0.066]

74 On the other hand unsupervised word segmentation Peng and Schuurmans (2001); Goldwater et al. [sent-157, score-0.354]

75 (1998) takes advantage of the huge amount of raw text to solve Chinese word segmentation problems. [sent-160, score-0.354]

76 (2007) learns a scaling factor from data of source domain and use the distribution to resemble target domain distribution. [sent-164, score-0.295]

77 (2009) uses a Domain adaptive bootstrapping (DAB) framework, which shows good results on Named Entity Recognition. [sent-166, score-0.107]

78 Besides, Sun and Xu (201 1) uses a sequence labeling framework, while unsupervised statistics are used as discrete features in their model, which prove to be effective in Chinese word segmentation. [sent-170, score-0.298]

79 There are previous works using punctuations as implicit annotations. [sent-171, score-0.503]

80 Li and Sun (2009) proposed a compromising solution to by using a clas- sifier to select the most confident characters. [sent-173, score-0.169]

81 Instead, we only add the characters after punctuations which are sure to be the beginning of words (which means labeling ’B’) into our training set. [sent-175, score-0.858]

82 Sun and Xu (201 1) uses punctuation information as discrete feature in a sequence labeling framework, which shows improvement compared to the pure sequence labeling approach. [sent-176, score-0.547]

83 5 Conclusion In this paper we have presented an effective yet simple approach to Chinese word segmentation on micro-blog texts. [sent-179, score-0.354]

84 In our approach, punctuation information of unlabeled micro-blog data is used, as well as a self-training framework to incorporate confident instances. [sent-180, score-0.507]

85 Both the punctuation information and the self-training phase contribute to this improve- ment. [sent-182, score-0.184]

86 In Annual Meeting-Association For Computational Linguistics, volume 45, page 264. [sent-223, score-0.037]

87 Unsupervised segmentation of chinese text by use of branching entropy. [sent-227, score-0.663]

88 In Proceedings of the Fourth SIGHAN Workshop on Chinese Language Processing, volume 164. [sent-240, score-0.037]

89 Chinese word segmentation without using lexicon and hand-crafted training data. [sent-246, score-0.354]

90 Fast online training with frequency-adaptive learning rates for chinese word segmentation and new word detection. [sent-283, score-0.753]

91 An improved chinese word segmentation system with 181 conditional random field. [sent-309, score-0.708]

92 In Proceedings of the Fifth SIGHAN Workshop on Chinese Language Processing, volume 117. [sent-310, score-0.037]

93 Effective tag set selection in chinese word segmentation via conditional random field modeling. [sent-317, score-0.78]

94 In Proceedings pages of PACLIC, volume 20, 87–94. [sent-318, score-0.037]

similar papers computed by tfidf model

tfidf for this paper:

wordName wordTfidf (topN-words)

[('punctuations', 0.456), ('chinese', 0.354), ('segmentation', 0.309), ('characters', 0.211), ('character', 0.203), ('punctuation', 0.184), ('unlabeled', 0.163), ('pku', 0.161), ('sun', 0.158), ('ahead', 0.147), ('texts', 0.114), ('labeling', 0.108), ('confident', 0.094), ('domain', 0.091), ('sighan', 0.085), ('beginning', 0.083), ('hurt', 0.083), ('ictclas', 0.077), ('zhengyan', 0.077), ('tag', 0.072), ('riley', 0.071), ('weinberger', 0.071), ('urls', 0.07), ('schuurmans', 0.067), ('incorporate', 0.066), ('xu', 0.064), ('bickel', 0.064), ('li', 0.063), ('meanwhile', 0.063), ('maxent', 0.062), ('cws', 0.061), ('maosong', 0.061), ('balanced', 0.06), ('enlarge', 0.055), ('news', 0.052), ('label', 0.052), ('letters', 0.051), ('bn', 0.051), ('medium', 0.051), ('instances', 0.05), ('china', 0.049), ('zhao', 0.048), ('peng', 0.047), ('labeled', 0.047), ('implicit', 0.047), ('cn', 0.046), ('word', 0.045), ('segmenter', 0.045), ('oov', 0.045), ('jin', 0.044), ('goldwater', 0.044), ('zhai', 0.044), ('daum', 0.043), ('target', 0.043), ('behind', 0.042), ('tsou', 0.039), ('dayang', 0.039), ('luckily', 0.039), ('compromising', 0.039), ('enlarging', 0.039), ('houfeng', 0.039), ('ibo', 0.039), ('longkai', 0.039), ('maximumentropy', 0.039), ('segmented', 0.039), ('feng', 0.039), ('names', 0.038), ('xue', 0.038), ('uses', 0.038), ('source', 0.038), ('ratio', 0.037), ('consideration', 0.037), ('discrete', 0.037), ('volume', 0.037), ('balance', 0.036), ('named', 0.036), ('adaptive', 0.036), ('sequence', 0.036), ('saul', 0.036), ('tackles', 0.036), ('sunny', 0.036), ('chieu', 0.036), ('sifier', 0.036), ('duplicating', 0.036), ('segment', 0.035), ('entity', 0.034), ('marcu', 0.034), ('prove', 0.034), ('delimiters', 0.034), ('scheffer', 0.034), ('ssl', 0.034), ('isahara', 0.034), ('bootstrapping', 0.033), ('contrasting', 0.032), ('enlarged', 0.032), ('reflection', 0.032), ('terminated', 0.032), ('unbalanced', 0.032), ('resemble', 0.032), ('hmmbased', 0.032)]

similar papers list:

simIndex simValue paperId paperTitle

same-paper 1 1.0000007 193 acl-2013-Improving Chinese Word Segmentation on Micro-blog Using Rich Punctuations

Author: Longkai Zhang ; Li Li ; Zhengyan He ; Houfeng Wang ; Ni Sun

2 0.37642047 123 acl-2013-Discriminative Learning with Natural Annotations: Word Segmentation as a Case Study

Author: Wenbin Jiang ; Meng Sun ; Yajuan Lu ; Yating Yang ; Qun Liu

Abstract: Structural information in web text provides natural annotations for NLP problems such as word segmentation and parsing. In this paper we propose a discriminative learning algorithm to take advantage of the linguistic knowledge in large amounts of natural annotations on the Internet. It utilizes the Internet as an external corpus with massive (although slight and sparse) natural annotations, and enables a classifier to evolve on the large-scaled and real-time updated web text. With Chinese word segmentation as a case study, experiments show that the segmenter enhanced with the Chinese wikipedia achieves sig- nificant improvement on a series of testing sets from different domains, even with a single classifier and local features.

3 0.35178223 80 acl-2013-Chinese Parsing Exploiting Characters

Author: Meishan Zhang ; Yue Zhang ; Wanxiang Che ; Ting Liu

Abstract: Characters play an important role in the Chinese language, yet computational processing of Chinese has been dominated by word-based approaches, with leaves in syntax trees being words. We investigate Chinese parsing from the character-level, extending the notion of phrase-structure trees by annotating internal structures of words. We demonstrate the importance of character-level information to Chinese processing by building a joint segmentation, part-of-speech (POS) tagging and phrase-structure parsing system that integrates character-structure features. Our joint system significantly outperforms a state-of-the-art word-based baseline on the standard CTB5 test, and gives the best published results for Chinese parsing.

4 0.33639258 82 acl-2013-Co-regularizing character-based and word-based models for semi-supervised Chinese word segmentation

Author: Xiaodong Zeng ; Derek F. Wong ; Lidia S. Chao ; Isabel Trancoso

Abstract: This paper presents a semi-supervised Chinese word segmentation (CWS) approach that co-regularizes character-based and word-based models. Similarly to multi-view learning, the “segmentation agreements” between the two different types of view are used to overcome the scarcity of the label information on unlabeled data. The proposed approach trains a character-based and word-based model on labeled data, respectively, as the initial models. Then, the two models are constantly updated using unlabeled examples, where the learning objective is maximizing their segmentation agreements. The agreements are regarded as a set of valuable constraints for regularizing the learning of both models on unlabeled data. The segmentation for an input sentence is decoded by using a joint scoring function combining the two induced models. The evaluation on the Chinese tree bank reveals that our model results in better gains over the state-of-the-art semi-supervised models reported in the literature.

5 0.28227365 164 acl-2013-FudanNLP: A Toolkit for Chinese Natural Language Processing

Author: Xipeng Qiu ; Qi Zhang ; Xuanjing Huang

Abstract: The growing need for Chinese natural language processing (NLP) is largely in a range of research and commercial applications. However, most of the currently Chinese NLP tools or components still have a wide range of issues need to be further improved and developed. FudanNLP is an open source toolkit for Chinese natural language processing (NLP) , which uses statistics-based and rule-based methods to deal with Chinese NLP tasks, such as word segmentation, part-ofspeech tagging, named entity recognition, dependency parsing, time phrase recognition, anaphora resolution and so on.

6 0.25527054 288 acl-2013-Punctuation Prediction with Transition-based Parsing

7 0.2487032 173 acl-2013-Graph-based Semi-Supervised Model for Joint Chinese Word Segmentation and Part-of-Speech Tagging

8 0.2449604 7 acl-2013-A Lattice-based Framework for Joint Chinese Word Segmentation, POS Tagging and Parsing

9 0.18099882 243 acl-2013-Mining Informal Language from Chinese Microtext: Joint Word Recognition and Segmentation

10 0.1774701 44 acl-2013-An Empirical Examination of Challenges in Chinese Parsing

11 0.14054574 99 acl-2013-Crowd Prefers the Middle Path: A New IAA Metric for Crowdsourcing Reveals Turker Biases in Query Segmentation

12 0.12893867 140 acl-2013-Evaluating Text Segmentation using Boundary Edit Distance

13 0.11392299 34 acl-2013-Accurate Word Segmentation using Transliteration and Language Model Projection

14 0.11131318 16 acl-2013-A Novel Translation Framework Based on Rhetorical Structure Theory

15 0.11053358 220 acl-2013-Learning Latent Personas of Film Characters

16 0.10185826 97 acl-2013-Cross-lingual Projections between Languages from Different Families

17 0.10163268 204 acl-2013-Iterative Transformation of Annotation Guidelines for Constituency Parsing

18 0.10058272 155 acl-2013-Fast and Accurate Shift-Reduce Constituent Parsing

19 0.098769002 112 acl-2013-Dependency Parser Adaptation with Subtrees from Auto-Parsed Target Domain Data

20 0.093743801 71 acl-2013-Bootstrapping Entity Translation on Weakly Comparable Corpora

similar papers computed by lsi model

lsi for this paper:

topicId topicWeight

[(0, 0.24), (1, -0.103), (2, -0.275), (3, 0.111), (4, 0.4), (5, -0.102), (6, -0.145), (7, 0.048), (8, -0.027), (9, 0.173), (10, -0.056), (11, 0.069), (12, 0.04), (13, -0.007), (14, -0.019), (15, -0.021), (16, 0.066), (17, -0.02), (18, -0.014), (19, 0.065), (20, 0.038), (21, -0.023), (22, 0.001), (23, -0.007), (24, 0.004), (25, -0.014), (26, 0.029), (27, 0.027), (28, 0.03), (29, 0.016), (30, -0.023), (31, 0.018), (32, 0.039), (33, 0.045), (34, 0.01), (35, 0.033), (36, 0.003), (37, -0.037), (38, -0.013), (39, 0.042), (40, -0.011), (41, -0.008), (42, -0.053), (43, 0.011), (44, -0.016), (45, -0.043), (46, 0.052), (47, -0.041), (48, -0.038), (49, 0.004)]

similar papers list:

simIndex simValue paperId paperTitle

same-paper 1 0.96533597 193 acl-2013-Improving Chinese Word Segmentation on Micro-blog Using Rich Punctuations

Author: Longkai Zhang ; Li Li ; Zhengyan He ; Houfeng Wang ; Ni Sun

2 0.92374516 123 acl-2013-Discriminative Learning with Natural Annotations: Word Segmentation as a Case Study

Author: Wenbin Jiang ; Meng Sun ; Yajuan Lu ; Yating Yang ; Qun Liu

3 0.90961224 82 acl-2013-Co-regularizing character-based and word-based models for semi-supervised Chinese word segmentation

Author: Xiaodong Zeng ; Derek F. Wong ; Lidia S. Chao ; Isabel Trancoso

4 0.84902996 243 acl-2013-Mining Informal Language from Chinese Microtext: Joint Word Recognition and Segmentation

Author: Aobo Wang ; Min-Yen Kan

Abstract: We address the problem of informal word recognition in Chinese microblogs. A key problem is the lack of word delimiters in Chinese. We exploit this reliance as an opportunity: recognizing the relation between informal word recognition and Chinese word segmentation, we propose to model the two tasks jointly. Our joint inference method significantly outperforms baseline systems that conduct the tasks individually or sequentially.

5 0.82000417 7 acl-2013-A Lattice-based Framework for Joint Chinese Word Segmentation, POS Tagging and Parsing

Author: Zhiguo Wang ; Chengqing Zong ; Nianwen Xue

Abstract: For the cascaded task of Chinese word segmentation, POS tagging and parsing, the pipeline approach suffers from error propagation while the joint learning approach suffers from inefficient decoding due to the large combined search space. In this paper, we present a novel lattice-based framework in which a Chinese sentence is first segmented into a word lattice, and then a lattice-based POS tagger and a lattice-based parser are used to process the lattice from two different viewpoints: sequential POS tagging and hierarchical tree building. A strategy is designed to exploit the complementary strengths of the tagger and parser, and encourage them to predict agreed structures. Experimental results on Chinese Treebank show that our lattice-based framework significantly improves the accuracy of the three sub-tasks. 1

6 0.81583279 80 acl-2013-Chinese Parsing Exploiting Characters

7 0.77938986 173 acl-2013-Graph-based Semi-Supervised Model for Joint Chinese Word Segmentation and Part-of-Speech Tagging

8 0.7572791 164 acl-2013-FudanNLP: A Toolkit for Chinese Natural Language Processing

9 0.67666847 140 acl-2013-Evaluating Text Segmentation using Boundary Edit Distance

10 0.6249215 128 acl-2013-Does Korean defeat phonotactic word segmentation?

11 0.61178643 44 acl-2013-An Empirical Examination of Challenges in Chinese Parsing

12 0.57962173 99 acl-2013-Crowd Prefers the Middle Path: A New IAA Metric for Crowdsourcing Reveals Turker Biases in Query Segmentation

13 0.53384155 34 acl-2013-Accurate Word Segmentation using Transliteration and Language Model Projection

14 0.53216046 50 acl-2013-An improved MDL-based compression algorithm for unsupervised word segmentation

15 0.49758673 288 acl-2013-Punctuation Prediction with Transition-based Parsing

16 0.48191765 220 acl-2013-Learning Latent Personas of Film Characters

17 0.45921582 204 acl-2013-Iterative Transformation of Annotation Guidelines for Constituency Parsing

18 0.44744974 356 acl-2013-Transfer Learning Based Cross-lingual Knowledge Extraction for Wikipedia

19 0.41604549 155 acl-2013-Fast and Accurate Shift-Reduce Constituent Parsing

20 0.4004544 370 acl-2013-Unsupervised Transcription of Historical Documents

similar papers computed by lda model

lda for this paper:

topicId topicWeight

[(0, 0.043), (6, 0.032), (11, 0.082), (15, 0.011), (16, 0.22), (24, 0.059), (26, 0.065), (35, 0.072), (42, 0.06), (48, 0.04), (70, 0.078), (77, 0.011), (88, 0.029), (90, 0.025), (95, 0.103)]

similar papers list:

simIndex simValue paperId paperTitle

1 0.81794286 123 acl-2013-Discriminative Learning with Natural Annotations: Word Segmentation as a Case Study

Author: Wenbin Jiang ; Meng Sun ; Yajuan Lu ; Yating Yang ; Qun Liu

2 0.80536109 157 acl-2013-Fast and Robust Compressive Summarization with Dual Decomposition and Multi-Task Learning

Author: Miguel Almeida ; Andre Martins

Abstract: We present a dual decomposition framework for multi-document summarization, using a model that jointly extracts and compresses sentences. Compared with previous work based on integer linear programming, our approach does not require external solvers, is significantly faster, and is modular in the three qualities a summary should have: conciseness, informativeness, and grammaticality. In addition, we propose a multi-task learning framework to take advantage of existing data for extractive summarization and sentence compression. Experiments in the TAC2008 dataset yield the highest published ROUGE scores to date, with runtimes that rival those of extractive summarizers.

same-paper 3 0.80464792 193 acl-2013-Improving Chinese Word Segmentation on Micro-blog Using Rich Punctuations

Author: Longkai Zhang ; Li Li ; Zhengyan He ; Houfeng Wang ; Ni Sun

4 0.79761451 191 acl-2013-Improved Bayesian Logistic Supervised Topic Models with Data Augmentation

Author: Jun Zhu ; Xun Zheng ; Bo Zhang

Abstract: Supervised topic models with a logistic likelihood have two issues that potentially limit their practical use: 1) response variables are usually over-weighted by document word counts; and 2) existing variational inference methods make strict mean-field assumptions. We address these issues by: 1) introducing a regularization constant to better balance the two parts based on an optimization formulation of Bayesian inference; and 2) developing a simple Gibbs sampling algorithm by introducing auxiliary Polya-Gamma variables and collapsing out Dirichlet variables. Our augment-and-collapse sampling algorithm has analytical forms of each conditional distribution without making any restricting assumptions and can be easily parallelized. Empirical results demonstrate significant improvements on prediction performance and time efficiency.

5 0.6920805 82 acl-2013-Co-regularizing character-based and word-based models for semi-supervised Chinese word segmentation

Author: Xiaodong Zeng ; Derek F. Wong ; Lidia S. Chao ; Isabel Trancoso

6 0.67925417 155 acl-2013-Fast and Accurate Shift-Reduce Constituent Parsing

7 0.67897499 80 acl-2013-Chinese Parsing Exploiting Characters

8 0.67390829 132 acl-2013-Easy-First POS Tagging and Dependency Parsing with Beam Search

9 0.6737693 134 acl-2013-Embedding Semantic Similarity in Tree Kernels for Domain Adaptation of Relation Extraction

10 0.67101085 288 acl-2013-Punctuation Prediction with Transition-based Parsing

11 0.66642362 169 acl-2013-Generating Synthetic Comparable Questions for News Articles

12 0.66602743 174 acl-2013-Graph Propagation for Paraphrasing Out-of-Vocabulary Words in Statistical Machine Translation

13 0.66577405 164 acl-2013-FudanNLP: A Toolkit for Chinese Natural Language Processing

14 0.6644693 343 acl-2013-The Effect of Higher-Order Dependency Features in Discriminative Phrase-Structure Parsing

15 0.66413879 333 acl-2013-Summarization Through Submodularity and Dispersion

16 0.66391218 212 acl-2013-Language-Independent Discriminative Parsing of Temporal Expressions

17 0.66297543 318 acl-2013-Sentiment Relevance

18 0.66202205 154 acl-2013-Extracting bilingual terminologies from comparable corpora

19 0.66137469 272 acl-2013-Paraphrase-Driven Learning for Open Question Answering

20 0.6611889 81 acl-2013-Co-Regression for Cross-Language Review Rating Prediction