Despite recent tendencies towards building large "monolithic" neural models, fine-tuned expert models and parameter-efficient specialised modules still offer gains over large monoliths in specific tasks and for specific data distributions (e.g., low-resource languages or specialised domains). Moreover, such modularisation of skills and expertise into dedicated models or modules allows for asynchronous, decentralised, and more efficient continuous model development, as well as module reusability. However, a central question remains: how to combine and compose these modules to enable positive transfer, sample-efficient learning, and improved out-of-domain generalisation. In this talk, after discussing the key advantages of modularisation and modular specialisation, I will provide an overview of prominent module and model composition strategies. I will focus on composition at the parameter level (model merging) and functional level (model MoErging), and then illustrate the usefulness of these techniques across several applications.

Ivan Vulić is currently a Research Scientist at Google DeepMind in Zurich after spending a year there as a Visiting Researcher. Before that he was a Research Professor and a Royal Society University Research Fellow in the Language Technology Lab, University of Cambridge, where he spent 10 years across different research roles. From January 2018 until November 2024 he was also a Senior Scientist at PolyAI in London. Ivan holds a PhD in Computer Science from KU Leuven awarded summa cum laude. In 2021 he was awarded the annual Karen Spärck Jones Award from the British Computing Society for his research contributions to Natural Language Processing and Information Retrieval. His core expertise and research interests span, among others, cross-lingual, multilingual and multi-modal representation learning, modularity and composability of ML models, sample-efficient, parameter-efficient and few-shot ML, conversational AI, data-centric ML.

Large Language Models (LLMs) are a breakthrough technology with broad social and economic effects. However, the development of leading models is currently concentrated in a few technology hubs primarily in the U.S. and China, leaving smaller languages behind and making Europe dependent on external technologies. To secure its digital sovereignty and ensure that its languages are fully represented, it is essential for Europe to have the capacity to build its own foundation models. In this talk, I will present a line of LLM work ranging from early monolingual models for Finnish to current efforts to create fully open foundation models for all European languages in the OpenEuroLLM project.

Sampo Pyysalo is one of the leads of the TurkuNLP group (https://turkunlp.org/) in the University of Turku, Finland. His work focuses on machine learning for natural language processing, with particular emphasis on scientific text mining, Finnish language technology, and large language models. He received his PhD thesis from the University of Turku and held researcher positions at the University of Tokyo, University of Manchester and University of Cambridge before returning to the University of Turku in 2019. He is currently PI in the HPLT (https://hplt-project.org/) and OpenEuroLLM (https://openeurollm.eu/) projects, where he leads efforts to train multilingual language models.

Speech is our most natural way of communication and the loss of the ability to speak is therefore devastating to patients. A speech neuroprostheses that directly reconstructs speech processes from neural activity could provide a new means of communications to these severely affected patients. In this presentation, I will present some approaches to reconstruct different representations of speech from intracranial recordings and highlight how they can be used to build a speech neuroprosthesis. The decoding of speech processes is particularly challenging, as not only the neural, but also the target signal has complex, nonlinear dynamics. I will stress the use of interpretable machine learning models for this task to ensure that meaningful activity is decoded and scientific insights might be generated as a side product.

Dr. Christian Herff is an assistant professor in the School for Mental Health and Neuroscience at Maastricht University where he leads the invasive BCI research line. His research interest lays in the application of machine learning technology to neurophysiological data for Brain-Computer Interfaces and neuroscience research. With a particular focus on the decoding of speech processes from intracranial data, he tries to improve the lives of severely paralyzed patients while simultaneously improving our understanding of complex higher order cognition. He emphasizes the ability to achieve interpretable results based on computational models. In particular, visualization of complex dynamic models, such as deep neural networks, is of interest to him.

Today, LLMs are Swiss knives and MT one of their tools. Is this the end of MT research? In this talk, I argue that the connection between LLM and MT research is two-way. I present some of our recent work advancing multilingual LLMs, tools to estimate their quality, and how the two can be combined for test-time scaling. First, I present xCOMET, an open-source learned metric which integrates sentence-level evaluation and error span detection, exhibiting state-of-the-art performance across all types of meta-evaluation (sentence-level, system-level, and error span detection). Moreover, it does so while highlighting and categorizing error spans, thus enriching the quality assessment. Then, I present Tower, a suite of open multilingual LLMs for translation-related tasks. Tower models are created through continued pretraining on a carefully curated multilingual mixture of monolingual and parallel data. The combination of Tower with COMET reranking obtained the best results in 8 out of 11 language pairs in the WMT General Translation shared task, according to human evaluation. Finally, I describe EuroLLM, an ongoing EU-made project whose goal is to train an open multilingual LLM from scratch using the European HPC infrastructure (EuroHPC). The last release (EuroLLM-9B) supports 35 languages, including all 24 official EU languages, and it achieves strong results in various benchmarks, comparable or better than the best existing models of similar size.

xCOMET: https://huggingface.co/collections/Unbabel/xcomet-659eca973b3be2ae4ac023bb

Tower: https://huggingface.co/collections/Unbabel/tower-659eaedfe36e6dd29eb1805c

EuroLLM: https://huggingface.co/blog/eurollm-team/eurollm-9b

André F. T. Martins (PhD 2012, Carnegie Mellon University and Instituto Superior Técnico; https://andre-martins.github.io/) is an Associate Professor at Instituto Superior Técnico, University of Lisbon, researcher at Instituto de Telecomunicações, and the VP of AI Research at Unbabel. His research, funded by a ERC Starting Grant (DeepSPIN) and Consolidator Grant (DECOLLAGE), among other grants, include machine translation, quality estimation, structure and interpretability in deep learning systems for NLP. His work has received several paper awards at ACL conferences. He co-founded and co-organizes the Lisbon Machine Learning School (LxMLS), and he is a Fellow of the ELLIS society and co-director of the ELLIS Program in Natural Language Processing. He is a member of the R&I advisory group of EuroHPC, the European infrastructure for supercomputing.