关键词: 自然语言处理, 文本摘要, 对比学习, 模型微调

Abstract: To address the problems of the traditional text summarization models trained based on cross-entropy loss functions，such as degraded performance during inference，low generalization，serious exposure bias phenomenon during generation，and low similarity between the generated summary and the reference summary text，a novel training approach is proposed in this paper.On the one hand，the model itself generates a candidate set using beam search and selects positive and negative samples based on the evaluation scores of the candidate summaries.Two sets of contrastive loss functions are built using“argmax-greedy search probability values”and“label probability values”within the output candidate set.On the other hand，a time-series recursive function designed to operate on the candidate set’s sentences guides the model to ensure temporal accuracy when outputting each individual candidate summary and mitigates exposure bias.Our experiments show the method significantly improves the generalization performance on the CNN／Daily Mail and Xsum public datasets.The Rouge and Bert Score reach 47.54 and 88.51 respectively on CNN／Daily Mail while they reach 48.75 and 92.61 on Xsum.