Evaluating Free Online Machine Translators in Rendering Persian News Headlines into English: Hutchins and Somers’ Criteria in Focus

Nowadays, multiple translations are done using the free online machine translation (MT), such as Google Translate, Bing translator, and Babylon. Since these machines are updated regularly by their designers, it seems that evaluating machine translators at regular intervals can be very significant either to the end users or the designers themselves. This study was aimed at examining the translation quality of 3 free online machine translators which offer Persian into English translation services, including Google Translate, Bing Translator, and Babylon. Forty Persian news headlines were randomly selected from Islamic Republic News Agency – IRNA. The news headlines were translated by the three MT systems. A questionnaire was designed and assigned to 30 professionals to compare the outputs with human translations and evaluate the MT systems to determine which system was more effective in translating the news headlines. The evaluation was based on the criteria proposed by Hutchins and Somers (1992): intelligibility, fidelity, and style. The findings indicated that the best machine translator for translating Persian news headlines into English was Google Translate with 91.25% of intelligibility, 84.25% of fidelity, and 87% of style. After Google Translate the best translation was delivered by Babylon with 59.25% of intelligibility, 62.5% of fidelity, and 66.5% of style. Additionally, Bing Translator delivered the lowest scores of 58.25% of intelligibility, 57.75% of fidelity, and 52.25% of style. As implication, the results of this evaluation would help Iranian users make, in advance, an informed choice as to the free online machine translator that meets their needs best when translating Persian news headlines into English.