AMPD: an Analects-Mandarin parallel dataset for bidirectional translation

ABSTRACT

In the field of natural language processing, there is no specialized dataset for the Analects, which makes it difficult to assess whether language models can find the semantic relevance between the Analects and modern Mandarin. To address this issue, this paper proposes a dataset named AMPD (Analects-Mandarin Parallel Dataset), which includes the Analects and its corresponding modern Mandarin, keywords and their annotations in the Analects, as well as sentiment. Additionally, we propose four baseline tasks and benchmark them by implementing currently popular algorithms respectively.

GRAPHICAL ABSTRACT

KEYWORDS:

• The Analects
• dataset
• translation
• sentiment analysis

Disclosure statement

No potential conflict of interest was reported by the author(s).

Notes

1 https://lxs.sicnu.edu.cn/p/0/?StId=st_app_news_i_x638173469346478226

2 https://github.com/NiuTrans/Classical-Modern

3 https://huggingface.co/ClueAI/ChatYuan-large-v1

4 https://huggingface.co/ClueAI/ChatYuan-large-v2

5 https://openai.com/blog/chatgpt

6 Due to the limitation of the graphics memory, we are unable to conduct experiments with longer sequences.

Additional information

Funding

This work was partially supported by Sichuan Science and Technology Program [Grant No:2023YFQ0044].