Synteract-1: Predicting Protein Interactions from Sequence

Proteins bind, recruit, block, modify, stabilize, and regulate one another. Those interactions are central to signaling, disease biology, immune response, and drug discovery.

The obstacle is scale. Wet-lab assays remain the source of truth and testing every possible pair is impossible. A single organism holds thousands of proteins, which is millions of pairs. Synteract-1 asked whether a protein language model could narrow that search: given two amino acid sequences, estimate whether the proteins are likely to interact.

The negatives problem

Any binary classifier needs both classes. Interaction databases supply one of them abundantly and the other barely at all.

The preprocessed BioGRID multi-validated set used here yielded 179,018 positive pairs. Negatome, the main curated source of experimentally supported non-interactions, yielded 3,958 after preprocessing. A 45-to-1 imbalance, where a model that always predicts "interacts" scores above 97 percent and has learned nothing.

Fine-tuning ProtBERT-BFD on this required feeding both sequences as one input separated by a [SEP] token, pooling the [CLS] representation, and projecting to two classes. The paper is candid that training was unstable: many runs plateaued at or near 50 percent accuracy, apparently stuck in local minima, before one converged.

What it scored

Four of the five evaluation sets look strong. The fifth does not: on interaction data derived from subcellular compartment sampling, Synteract-1 manages 50 percent, exactly chance. Broken down, it calls 85 percent of that set's positives correctly and only 38 percent of its negatives.

A model can fail for two reasons. It can be wrong, or the test can be asking a different question. The paper went looking for which.

The dataset was the model

The test is simple and the result is stark. Take the compartment-derived datasets. Train Synteract-1 on them from scratch. Evaluate on their own held-out split, and then on the multi-validated interaction test set.

Trained on human compartment data, Synteract-1 reaches 96 percent on that data's own test set and 54 percent on validated interaction data. Trained on all five compartment datasets, 87 percent on its own and 64 percent on ours. The model is learning something real and highly generalizable within the dataset it was built from, and that something is not interaction.

The confirming experiment removes the language model entirely. Take two crude features per protein, predicted subcellular compartment (one of nine) and predicted solubility (membrane-bound or not), giving four features per pair, and fit a support vector machine.

On the multi-validated test set that classifier scores 50 percent, which is the correct answer for four features that contain no interaction information. On human compartment data it scores 65 percent, and correctly classifies 95 percent of the negatives. There are only 324 unique combinations of those feature values across the entire dataset, so duplicates between train and test are unavoidable.

This is the same failure mode we later found in a different guise. Compartment sampling creates a localization shortcut; random negative sampling across a multi-species dataset creates a taxonomy shortcut. The mechanism is identical: some property correlates with the label because of how the negatives were generated, and the model finds it.

Do the synthetic negatives hold up?

Fair question, given that the same paper is criticizing other people's negatives.

The test: fit a support vector machine on embeddings of 25,000 real positives and 25,000 synthetic negatives, then check whether it can classify real Negatome negatives it never saw. With untrained ProtBERT embeddings it scores 50 percent, no better than chance. With embeddings from the trained Synteract-1 model it reaches 73 percent, and correctly classifies 238 real Negatome pairs without ever having seen a real negative.

So the synthetic negatives do occupy a region of representation space that overlaps with real non-interactors. But there is a limit, stated plainly: training Synteract-1 with no Negatome or Negatome-derived samples at all never produced better than 50 percent on the test data. Synthetic negatives can amplify a small set of real ones. They cannot replace them.

What it established

Two things, of unequal importance.

The first is the headline: a protein language model can learn the interaction objective from sequence alone, given a balanced dataset. That was not obvious in 2023, and 92 percent on held-out data across 50 organisms was a real signal.

The second is the one that shaped everything after. Several widely used interaction datasets are much easier than they look, because the procedure used to make their negatives leaves a fingerprint that a model can find. High accuracy on such a dataset is not evidence of interaction understanding, and the paper's own suspicion of its own strong numbers is the right instinct to carry forward.

Its self-assessment is worth quoting: because of the compartment-data performance, "SYNTERACT has only learned some notion of physical contact that mediates chemical or conformational changes with or without a biologically relevant function."

That skepticism became the through-line of the research program. Synteract-2 pushed toward affinity and interface prediction. The accidental taxonomist work generalized the shortcut finding from compartments to phylogeny. And Synteract-4 was built with cluster-aware training and leakage-controlled evaluation from the start, because by then the lesson had been learned twice.