自然语言处理nltk分词断句（2）

75 阅读 0 评论 50 点赞

我是靠谱客的博主伶俐大叔，这篇文章主要介绍自然语言处理nltk分词断句（2），现在分享给大家，希望可以做个参考。

复制代码

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
from nltk.corpus import gutenberg
import nltk


# file = gutenberg.words('melville-moby_dick.txt')
# text = nltk.Text(file)
# print(file)
# print(text)
# print(text.findall('<a> (<.*>) <man>'))
# 对Text对象使用findall的特殊方式
# this is not a list


# test_string = '''Over the last 10 years, I have used and tweaked Vim. 
# This configuration is the ultimate vimrc 
# (or at least my version of it).'''
# print(nltk.word_tokenize(test_string))


# 词干提取
# porter = nltk.PorterStemmer()
# lancaster = nltk.LancasterStemmer()
# test_list = gutenberg.words('melville-moby_dick.txt')[:10]
# list_by_porter = [porter.stem(word) for word in test_list]
# list_by_lancaster = [lancaster.stem(word) for word in test_list]
# print('the oringin:',test_list)
# print('porter stem:',list_by_porter)
# print('lancaster stem:',list_by_lancaster)


# test_list = gutenberg.words('melv