Evaluating preprocessing by Turing Machine in text categorization

By developing the World Wide Web, text categorization becomes a key technology to deal with and organize a large number of documents. Automatic text categorization is amethod to contrast a massive data. The basic phases of text categorization include preprocessing, extracting relevant featuresagainst the features in a database, and finally categorizing a set of documents into predefined categories. In this article, we proposea new preprocessing method by Turing Machine. All of four steps in preprocessing such as sentence segmentation, tokenization, stopword removal, and word stemming are done by Turing Machine.Aiming to access the importance of the preprocessing by Turing Machine on the text classification problem, we applied the supportvector machine paradigm to the Reuters and PAGOD dataset. Searching for the best document representation, we evaluated and analyzed some known feature reduction, feature subset selection and term weighting. Experiments show that proposed method is more accurate than other methods

کلیدواژه ها:

Preprocessing ، Turing Machine ، text categorization ، Support Vector Machines