Building text embeddings for low-resource languages seems to require high-quality data at scale. Navasardyan, Bughdaryan, Minasyan, and Davtyan (arXiv:2603.22290) find the opposite. Fine-tuning a multilingual encoder on just 10,000 noisy synthetic pairs — translated from English Reddit via open-weights models, no quality filtering — yields 11-12% average improvements in retrieval performance for Armenian. This matches models trained on roughly a million examples.
Scaling up the data doesn't help. Improving the translation quality doesn't help. Expanding data diversity doesn't help. The gains saturate at 10,000 noisy pairs. Everything beyond this threshold is redundant.
The explanation is that semantic alignment for under-resourced languages requires surprisingly little signal. The multilingual encoder already has a rough sense of Armenian from its pretraining — what it lacks is a few thousand examples that anchor the embedding space for retrieval tasks. The noise in the translations is irrelevant because the model isn't learning Armenian semantics from scratch; it's calibrating an existing representation. A noisy calibration signal works as well as a clean one because the calibration itself is low-dimensional — a few thousand alignment corrections in embedding space.
This has immediate practical implications. Any community with access to a translation model and a few thousand source-language texts can build competitive embeddings without clean parallel corpora, expert annotation, or massive compute. The barrier to multilingual NLP was assumed to be data scarcity. It's actually data threshold — and the threshold is low enough that noise doesn't matter.