Evaluation of narrative and expository text summaries using latent semantic analysis

Research output: Chapter in Book/Report/Conference proceedingChapterpeer-review

1 Scopus citations

Abstract

In this chapter I approach three automatic methods for the evaluation of summaries from narrative and expository texts in Spanish. The task consisted of correlating the evaluation made by three raters for 373 summaries with results provided by latent semantic analysis. Scores assigned by latent semantic analysis were obtained by means of the following three methods: 1) Comparison of summaries with the source text, 2) Comparison of summaries with a summary approved by consensus, and 3) Comparison of summaries with three summaries constructed by three language teachers. The most relevant results are a) a high positive correlation between the evaluation made by the raters (r= 0.642); b) a high positive correlation between the computer methods (r= 0.810); and c) a moderate-high positive correlation between the evaluations of raters and the second and third LSA methods (r= 0.585 and 0,604), in summaries from narrative texts. Both methods did not differ significantly in statistical terms from the correlation among raters when the texts evaluated were predominantly narrative. These results allow us to assert that at least two holistic LSA-based methods are useful for assessing reading comprehension of narrative texts written in Spanish.

Original languageEnglish
Title of host publicationApplied Natural Language Processing
Subtitle of host publicationIdentification, Investigation and Resolution
PublisherIGI Global
Pages531-544
Number of pages14
ISBN (Print)9781609607418
DOIs
StatePublished - 1 Dec 2011

Fingerprint Dive into the research topics of 'Evaluation of narrative and expository text summaries using latent semantic analysis'. Together they form a unique fingerprint.

Cite this