SynthyraResearch
5 min read
Translator: Broad Protein Annotation, Fast
Translator predicts structured protein annotations directly from sequence using the Annotation Vocabulary framework. It is built for high recall on unknown proteins, and its precision varies enormously by annotation type.
Logan Hallee
March 18, 2025
Protein sequences arrive faster than anyone can curate them. Metagenomic surveys, newly sequenced organisms, designed proteins, and variant libraries all produce candidates with sparse or absent functional annotation.
Sequence similarity search is the standard first move and it is genuinely good. It is also weakest exactly where the interest is highest: a sequence distant from everything characterized, or one carrying several partial functional hints that no single homolog explains.
Translator is built for that first pass. Given a sequence, it returns a structured set of annotation hypotheses across eight aspects: enzyme commission number, the three gene ontology branches, InterPro domain, 3D fold, keywords, and cofactor.
How it works
Translator inherits its output space from the Annotation Vocabulary framework, where structured biological ontologies are mapped onto a transformer token vocabulary. Instead of emitting free text, the model emits identifiers that already have meaning in existing databases.
That choice does a lot of work. The output is directly checkable against a database rather than requiring interpretation. It is unambiguous, since two curators phrasing the same function differently map to the same token. And it makes the annotation task a retrieval problem over a fixed vocabulary rather than open-ended generation.
The retrieval knob
Predictions come from a logit-retrieval step with a parameter k controlling how many candidate terms are pulled per aspect. Sweeping k from 1 to 100 traces the precision-recall tradeoff directly; the deployed default is k = 3, which maximizes F1. Raising k widens the net and lowers precision. There is also a minimum-confidence threshold that trims low-scoring suggestions.
What the numbers actually say
Translator was evaluated two ways. A held-out test set of 1,000 high-quality UniProt entries the model never saw, and a case study of 656 UniProt entries deposited after the model was trained, all with experimentally verified annotations. The second is the more meaningful test, since it is a genuine prospective check rather than a random split.
The pattern is consistent and it defines the tool. Recall is uniformly high; precision is not.
| Aspect | Precision | Recall | Test-set F1 | Case-study F1 |
|---|---|---|---|---|
| 3D fold | 0.82 | 0.81 | 0.81 | 0.93 |
| InterPro domain | 0.78 | 0.76 | 0.77 | 0.88 |
| Keywords | 0.41 | 0.86 | 0.56 | 0.67 |
| Cofactor | 0.38 | 0.81 | 0.52 | 0.69 |
| Molecular function | 0.37 | 0.73 | 0.49 | 0.50 |
| Cellular component | 0.25 | 0.77 | 0.38 | 0.39 |
| Enzyme commission | 0.21 | 0.85 | 0.33 | 0.55 |
| Biological process | 0.16 | 0.75 | 0.27 | 0.23 |
Structural aspects are reliable. Fold and domain predictions run above 0.75 on both precision and recall, and improve further on the case study, reaching 0.93 F1 for fold and 0.88 for domain. These can be used close to directly.
Functional aspects are suggestions. Enzyme commission recall is 0.85 with precision at 0.21, which in plain terms means: if this protein has an EC number, Translator's shortlist very probably contains it, alongside roughly four wrong ones. Biological process is the weakest on both axes and the only aspect that gets worse on the case study.
In plain terms
Translator is a metal detector, not a shovel. Tuned this way it beeps at nearly everything worth digging up, and it also beeps at bottle caps. That is the correct tradeoff when the alternative is walking the field by hand, but it means the output is a list of places to dig.
Why tune it this way
The high-recall setting is deliberate and follows from what the tool is for. In a triage workflow, a miss and a false positive have very different costs. A false positive gets discarded in a few seconds of expert attention. A miss means a protein's real function never enters the analysis at all, and nothing downstream will recover it.
That calculus flips if you use the output as ground truth. Feed unfiltered Translator annotations into a downstream training set and you inherit its precision, which for biological process means most of what you added is wrong.
Better case-study numbers are not straightforward good news. Newly deposited, experimentally verified entries are a different distribution from a random test split. They skew toward proteins that were interesting enough to characterize, which tend to be better covered by existing domain families. The improvement is real; it is not evidence the model will do this well on genuinely novel metagenomic sequence.
Terms used here
Where it fits
Translator is a triage layer. It sits between "we have a sequence" and "we should spend real effort on this," and its job is to make the second decision faster and better informed.
The natural pairing is with search rather than instead of it. Similarity search is authoritative when a close homolog exists. Translator adds a structured hypothesis when one does not, and its structural predictions are strong enough to be useful even when the functional ones need heavy filtering.
A mid-range F1 on a sparse multi-label ontology task is a useful tool, and the aspect-by-aspect precision spread tells you exactly how much narrowing you are getting.
Source
Annotation Vocabulary (Might Be) All You Need
Logan Hallee, Niko Rafailidis, Colin Horger, David Hong, Jason P. Gleghorn
Preprint, 2024. Translator performance figures are from the supplementary material of the Diffusion Sequence Models manuscript.