Ahmad Estiri - Web Technology Laboratory Ferdowsi University of Mashhad Mashhad, Iran
Mohsen Kahani - Web Technology Laboratory Ferdowsi University of Mashhad Mashhad, Iran
Hadi Ghaemi - Web Technology Laboratory Ferdowsi University of Mashhad Mashhad, Iran
Mohsen Abasi - Web Technology Laboratory Ferdowsi University of Mashhad Mashhad, Iran

in recent years, high increase in the amount of published web elements and the need to store, classify, restore, and process them have intensified the importance of naturallanguage processing and its related tools such as automatic summarizers and machine translators. In this paper, a novelapproach for evaluating automatic abstractive summarization system is proposed which can also be used in the other Natural Language Processing and Information Retrieval Applications. By comparing auto-abstracts (abstracts created by machine) with human abstracts (ideal abstracts created by human), the metricsintroduced in the proposed tool can automatically measure the quality of auto-abstracts. Evidently, we can’t semanticallycompare texts of abstractive summaries by comparison of just their words’ appearance. So it is necessary to use a lexicaldatabase such as WordNet. We use FerdowsNet with a properidea for Farsi language and it notably improves the evaluation results. This tool has been assessed by linguistic experts. This toolcontains metric for determining the quality of summaries automatically by comparing them with summaries generated byhumans (Ideal summaries). Evidently, we can’t semantically compare texts of abstractive summaries by comparison of justtheir words’ appearance and it is necessary to use a lexical database. We use this database with a proper idea together with Farsi parser in order to identify groups forming sentences and the results of evaluation improve significantly.

Farsi Natural Language Processing (NLP);Semantics; Evaluation; Automatic Abstractive Summarizer; Sentences groups; Parse tree; parser;