Do Multilingual Language Models Think Better in English?

Julen Etxaniz, Gorka Azkune, Aitor Soroa, Oier Lopez de Lacalle, Mikel Artetxe
11-02-2023 Natural Language Processing, Large Language Models, Deep Learning, Multilinguality
PDF Citar Código fuente Datos arXiv

Resumen

Translate-test is a popular technique to improve the performance of multilingual language models. This approach works by translating the input into English using an external machine translation system, and running inference over the translated input. However, these improvements can be attributed to the use of a separate translation system, which is typically trained on large amounts of parallel data not seen by the language model. In this work, we introduce a new approach called self-translate, which overcomes the need of an external translation system by leveraging the few-shot translation capabilities of multilingual language models. Experiments over 5 tasks show that self-translate consistently outperforms direct inference, demonstrating that language models are unable to leverage their full multilingual potential when prompted in non-English languages. Our code is available at https://github.com/juletx/self-translate.

Tipo
Prepublicación
Publicación
arXiv
Natural Language Processing Large Language Models Deep Learning Multilinguality
Julen Etxaniz
Julen Etxaniz
Estudiante de Doctorado en Análisis y Procesamiento del Lenguaje

Estudiante de Doctorado en Análisis y Procesamiento del Lenguaje en la Facultad de Informática de la Universidad del País Vasco (UPV/EHU). Graduado en Ingeniería Informática con especialidad en Ingeniería del Software.