Voyant Compass

2. How a model earns a place

A model may offer options in a language only if it scores at least chrF++ 30.0 for that language. chrF++ compares a model's output with a human translation of the same sentence, character by character and word by word, and 100 would mean identical. The human translations come from three places, in this order of trust. First come the team's calibration answers, then the published FLORES-200 and MAFAND-MT test sentences, and last the strings the lead has signed off. FLORES and MAFAND are news sentences rather than interface labels, so a calibration set decides wherever one exists. At most three models offer options in a language, with one size of each model family.

"Wrote their own" is the share of volunteers' decisions in which they wrote a translation instead of choosing an option, with the number of decisions in brackets. When it falls, the options are getting better.

3. How the models are asked

• Strings from one screen of Voyant are sent together, so that the same term reads the same way across the screen.
• Each request names the Voyant tool, the kind of string (a title, a button, a tooltip or a help text) and other strings on the same screen.
• It also gives the language's agreed glossary terms, its style notes, the team's own earlier translations of similar strings, and similar strings from the borrowed sources.
• Tags and placeholders such as {count} are protected. NLLB and MADLAD receive only the words between them, so they cannot break the markup.

4. Checks before an option is shown

An option is hidden if a placeholder was lost or added, a tag or brace was broken, a drafting marker was left behind, it is English with the words reordered, it is in the wrong script, or the model talked about the task instead of translating. The reason is recorded and nothing is deleted. Yorùbá text with no tone marks or underdots is shown with a warning, because the option may still be the best one. All text is stored in one Unicode form (NFC), so the same letters with tone marks always count as the same word.

5. What a volunteer sees

Each string shows up to three options. Options someone has already voted for come first. Next come options from models that pass the bar and the borrowed human translations, best first. Near misses, from models scoring between 20 and the bar, fill any place left. There is one option per model family. The options appear in a shuffled order, and the page does not say where each came from, so a vote judges the words and not their source. This also keeps the record of choices fair for research.

6. Who decides

• Reviewers vote for the best option, or say that none is right.
• Translators vote too, and they can write a new translation.
• Each language has a lead, who builds the team, gives out strings, keeps the glossary and style notes, and signs off. What a lead signs off appears on the experimental Voyant server within a minute.
• The project lead gives the final approval. A language goes to the main Voyant site once every string of its main screen is approved.

The main screen comes first. Its 254 strings, which every Voyant user meets, lead every queue. The long help texts and the Spyral notebooks come last.

7. Terminology and style

Each language keeps a glossary of agreed terms and a page of style notes, such as how to address the user and how to write tone marks. The hub suggests core terms, taken from Mozilla's terminology, from an earlier open glossary such as Kilinux's for Swahili, or from the best model, and a term counts only once the lead agrees it. Agreed terms and the style notes appear beside every string and are given to the models.

Leads coin new terms with the methods that Adegbola, Owolabi and Odejobi (2011) used for Yorùbá Windows. Build a term from a base form, so that related terms stay related. Extend the meaning of an existing word, as àká, a barn, became the word for a buffer. Describe a thing by what it does. Coin a word, translate the parts of a phrase, or borrow the English word and fit it to the language's sounds, as in kọ̀mpútà. They also warn that terms need people who know both the language and computing.

8. How the work learns

• Every choice, written translation, sign-off and approval is kept.
• What the models are told changes as the team works: the agreed glossary, the style notes, the team's own earlier translations and the borrowed examples all go into the next request.
• Which models may offer options changes too. Models are tested again on the team's calibration answers and signed-off strings, and every night the hub counts which sources' options the team chooses.
• The models themselves are not trained on the team's work. That may come later, once a language has enough agreed translations.

When a new language gets its first member, the hub prepares that language on its own. It gathers the borrowed translations, tests the models where test sentences exist, and drafts the main screen first.

9. What is recorded

For every option the hub keeps the text, every source that produced it, the checks it passed or failed, and any reason it is hidden. For every decision it keeps who made it, which options were shown and in what order, what was chosen or written, and when. Sign-offs and approvals keep who and when. The credit line in Voyant names the team members who asked to be credited, and the open projects whose translations were chosen.

10. Earlier work this builds on

• Translate.org.za localised free software into South African languages. The Translate Toolkit came out of that work.
• The African Network for Localisation (ANLoc), begun with IDRC support in 2006, added African locales to the Unicode CLDR, gathered a glossary of 2,500 computing terms in ten African languages, and built the Pootle and Virtaal translation tools.
• Microsoft's Local Language Program brought Windows into Yorùbá with the African Languages Technology Initiative (Adegbola, Owolabi and Odejobi 2011), and Microsoft once kept style guides for Hausa, Igbo and Yorùbá.
• Mozilla volunteers brought Firefox into Fulah in 2012, creating its computing terminology almost from scratch.

Those projects began with human translators writing every string. Voyant Compass begins with options from models and from other projects, and gives every decision to speakers of the language.

11. Limits

• The published test sentences are news text, not interface labels, so a model's score is a guide and not a guarantee.
• A borrowed translation can carry the sense it had in another product, such as 'Terms' in the sense of terms of service.
• For some languages no model reaches the bar yet, and the team writes most translations itself.
• This work translates Voyant's interface. Analysing texts in these languages well, for example counting Yorùbá words that carry tone marks, is a separate problem.
• Some of Voyant's interface text may be written into its code rather than its translation sheet, and cannot be translated here yet.

The states of a string

• Draft: options are ready and nobody has chosen yet.
• In review: a team member has chosen or written a translation.
• Signed off: the lead has confirmed it, and it shows on the experimental server.
• Approved: the project lead has approved it for the main Voyant site.
• English changed: Voyant changed the English after it was translated, so it needs another look.

Speak a language that Voyant does not yet speak?