The primary goal of the subproject is to enhance the understanding and translation of sign language through the development of Vision-Language Models (VLMs). This initiative is part of an interdisciplinary collaboration aimed at overcoming current limitations in integrating visual and textual data, improving key aspects such as visual grounding, compositional representation, and knowledge transfer across modalities.
The project addresses critical challenges in the development of VLMs and their application to automatic sign language translation, an area of particular difficulty due to the scarcity of annotated datasets and the inherent complexity of multimodal signals. To tackle these challenges, advanced visual encoders will be developed, and training methodologies will be improved. Thanks to this, MOLVI aims to build more robust and accurate models capable of adapting to multiple sign languages, such as Spanish Sign Language (LSE).
The project's impact is extensive. On one hand, the results are expected to contribute to the social inclusion of deaf communities by facilitating communication in key sectors such as healthcare, education, and public services. On the other hand, MOLVI will establish new scientific foundations by exploring how to effectively integrate visual and textual data into deep learning models. This approach not only enhances the capabilities of existing models but also enables new applications for other low-resource languages.
The implementation of the technologies developed in MOLVI also includes practical strategies, such as medical language translation and the evaluation of sign language learning. By adopting a user-centered and collaborative approach, the subproject ensures that technological developments are culturally sensitive and responsive to the real needs of deaf communities, promoting accessibility and reducing inequalities.
In summary, MOLVI is a comprehensive and pioneering effort to empower deaf communities through technology, opening new possibilities for interaction and understanding between users of different languages and communication systems.
Organization:  Ministerio de Ciencia, Innovación y Universidades (MCIU)
Main researcher: Gorka Labaka
Participants: 
Eneko Agirre, Jon Alkorta, Ekain Arrieta, Gorka Azkune, Nerea Ezeiza, Gorka Labaka, Mikel Lersundi, Oier López de Lacalle , Imanol Miranda, Adrián Núñez, Paula Ontalvilla, Olatz Perez de Viñaspre, Ander Soraluze, Benjamin Suter