Generation of Syntax Parser on South Indian Language using Bottom-Up Parsing Technique and PCFG

In our research, we provide a statistical syntax parsing method experimented on Kannada texts, which is an official language of Karnataka, India. The dataset is downloaded from TDIL website. Using the Cocke-Younger-Kasami (CYK) parsing technique, we generated Kannada Treebank dataset from ۱۰۰۰ annotated sentences in the first stage. The Treebank generated in this stage contains ۱۰۰۰ syntactically structured sentences and it is used as input to train the syntax parser model in the second stage. We have adopted Probabilistic Context Free Grammar (PCFG) while training the parser model and extracting the Chmosky Normal Form (CNF) grammar from a Treebank dataset. The developed syntax parser model is tested on ۱۵۰ raw Kannada sentences. It outputs with the most likely parse tree for each sentence and this is verified with golden Treebank. The syntax parser model generated ۷۴.۲% precision, ۷۹.۴% recall, and ۷۵.۳% F۱-score respectively. The similar technique may be adopted for other low resource languages.

کلیدواژه ها:

Natural language processing ، Artificial Intelligence ، Syntax Parser ، CYK Parsing Algorithm ، Probabilistic Context Free Grammar