基于NLTK的中文文本内容抽取方法

NLTK是Python中用于自然语言处理的第三方模块,但处理中文文本具有一定局限性.利用NLTK对中文文本中的信息内容进行抽取与挖掘,采用同语境词提取、双连词搭配提取、概率统计以及篇章分析等方法,得到一个适用于中文文本的NLTK文本内容抽取框架,及其具体的实现方法.经实证分析表明,在抽取结果中可以找到反映文本特点的语料内容,得到抽取结果与文本主题具有较强相关性的结论.     

Multilingual Sentiment Mining System to Prognosticate Governance

In the age of the internet, social media are connecting us all at the tip of our fingers. People are linkedthrough different social media. The social network, Twitter, allows people to tweet their thoughts on any particular event or a specific political body which provides us with a diverse range of political insights. This paper serves the purpose of text processing of a multilingual dataset including Urdu, English, and Roman Urdu. Explore machine learning solutions for sentiment analysis and train models, collect the data on government from Twitter, apply sentiment analysis, and provide a python library that classifies text sentiment. Training data contained tweets in three languages: English: 200k, Urdu: 200k and Roman Urdu: 11k. Five different classification models are applied to determine sentiments, and eventually, the use of ensemble technique to move forward with the acquired results is explored. The Logistic Regression model performed best with an accuracy of 75%, followed by the Linear Support Vector classifier and Stochastic Gradient Descent model, both having 74% accuracy. Lastly, Multinomial Naïve Bayes and Complement Naïve Bayes models both achieved 73% accuracy.