Maravilla Labs Starts Maraestra: Research Into a Model That Learns to Teach Smaller Models

Maraestra is a teacher model built on Apertus 8B, the open Swiss language model. Given a description of a task, a few examples, its constraints and a way to measure success, its job is to prepare everything a much smaller student model needs to learn that task, and to keep adjusting the lessons until the student gets it right. The Swiss National Supercomputing Centre (CSCS) supports the training with GPU compute on its Alps supercomputer.

Small Models, Expensive Lessons

Language models with half a billion to a few billion parameters can run offline on a laptop, a phone or an industrial gateway. That makes them attractive for organizations that cannot send their data to a cloud service, such as hospitals, public administrations and manufacturers. But a small model only becomes useful once it does one job reliably, and getting it there is still largely manual work.

Teams collect or generate training data, fine-tune, evaluate, discover where the model falls short, and start over. Every new task repeats the cycle, and every cycle depends on people who know which examples will actually help. The knowledge of how to teach a small model stays in their heads and in one-off scripts.

Teaching as the Specialization

Maraestra is not meant to be an expert in medicine, law or finance. Its intended specialization is the teaching process itself. Starting from a capability definition, the project explores a loop in which the teacher model:

1. analyses the capability and identifies the concepts and edge cases a student has to master,
2. builds a curriculum that moves from simple examples to ambiguous, adversarial and rare cases,
3. generates labelled training data and, kept strictly separate, evaluation data,
4. inspects where the trained student still fails,
5. writes targeted remediation data and teaches again.

A model cannot change another model's weights just by producing text. The optimization itself is done by a conventional training engine built on PyTorch, using parameter-efficient fine-tuning such as LoRA. Maraestra decides what to teach; the training engine does the learning. A successful run ends with a standalone small model, its adapter merged back into the weights.

To learn this behaviour, Maraestra is itself trained on teaching episodes: records of a student's failures, the data a teacher produced in response, and how much the student improved afterwards. Gathered across many different tasks, these episodes are meant to capture which teaching strategies work. That is closer to learning how to teach than to learning facts.

Measured by the Student

The project judges the teacher by what happens to its students. The central questions are how much a student improves on a task it has never seen, and how many training examples it needs to reach a target level of performance, compared with generic synthetic data and with teacher models that were not trained for teaching.

The first experiments focus on tasks with clear, automatic measures of success: classification and routing, structured extraction into fixed schemas, and policy guardrails such as redacting personal data. Reasoning-heavy tasks come later. The work builds on established research in knowledge distillation, curriculum learning and synthetic data generation; its open question is whether a dedicated teacher can learn a general teaching policy that holds across very different tasks.

Built on Apertus

Maraestra uses Apertus, the fully open language model family developed by EPFL, ETH Zurich and CSCS within the Swiss AI Initiative. Its weights and training recipes are published along with documentation of its training data. This matters for research that has to be reproducible, and for models that organizations want to be able to audit.

Apertus also fits the research question itself. Its smallest models, with 0.5 to 4 billion parameters, were created by distillation from Apertus 8B. Maraestra starts from that same 8B model as the teacher and uses the small Apertus models as students, adding a layer on top: not how to compress a model in general, but how to teach it one specific capability efficiently.

Compute From CSCS

Training teachers and students requires GPU capacity that a small lab does not operate on its own. The Swiss National Supercomputing Centre in Lugano supports the project with GPU compute on Alps, where the training runs use NVIDIA GH200 Grace Hopper nodes. Maravilla Labs is grateful to CSCS for this support.

Running the project on Swiss infrastructure fits its broader aim: small, open models that organizations can train and run under their own control.

First Research Phase

The first research phase started on 1 September 2026 and is expected to conclude at the end of December 2026. It is planned to cover:

• baseline measurements of small Apertus models on a set of bounded tasks,
• a first version of the teacher–student training loop and its evaluation harness,
• first teaching episodes and an early comparison against generic synthetic data.

Maraestra is early research. Its direction is still open, and there are no results to report yet. Maravilla Labs plans to share its findings once the first phase is complete.

About the Name

Maraestra is an invented word. It combines “mara” from maravilla, Spanish for wonder, with “estra” from maestra, Spanish for teacher.

About Maravilla Labs

Maravilla Labs is a software research laboratory based in Basel, Switzerland focused on building original developer infrastructure. The lab develops systems for source control, software distribution, hosting, and runtime execution with an emphasis on simplicity, security, and open architecture.

Projects developed by Maravilla Labs include systems for git hosting, package registries, CI and CD execution, edge runtime infrastructure and a versioned content database, as well as research into small, sovereign language models.