Answers for "word_tokenize(s)"

0

import word_tokenize

import nltk
from nltk import word_tokenize
Posted by: Guest on July-06-2020
0

word_tokenize

>>> from nltk.tokenize import word_tokenize
>>> s = '''Good muffins cost $3.88nin New York.  Please buy me
... two of them.nnThanks.'''
>>> word_tokenize(s)
['Good', 'muffins', 'cost', '$', '3.88', 'in', 'New', 'York', '.',
'Please', 'buy', 'me', 'two', 'of', 'them', '.', 'Thanks', '.']
Posted by: Guest on December-05-2021

Python Answers by Framework

Browse Popular Code Answers by Language