The short version
Genomic foundation models post strong benchmark numbers on tasks like “which organism is this coding sequence from.” The obvious worry is that they are reading codon composition, which a lookup table could do, and that everything else is decoration.
So I built the experiment that separates those. Synonymous codon swaps let you rewrite a sequence so that the protein is unchanged and the codon composition is identical to the original, exactly, asserted per sequence — while the order of the codons, and therefore the dinucleotides sitting at codon boundaries, changes. Anything the model still reads after that is arrangement, not composition.
The short answer: order carries real signal, but it is small, and it sits well below what plain composition gives you. That has a consequence I care about more than the benchmark number — a rearrangement that leaves composition untouched is invisible to any screen built on composition alone.
Isolating the arrangement signal
Side project, in progress — presented at the IOB student seminar, September 2026
A team project, with three collaborators and two mentors.
The design is the part I would want to be judged on:
- One corpus, several taxa. A few thousand coding sequences spanning bacterial, fungal, mammalian and viral classes, all read from a single pre-training corpus file so no class enters through a different door. Length and GC content at the wobble position are reported per class, because those are the confounders that would otherwise explain the result.
- Eight rewrite conditions, exactness asserted per item. A natural baseline, a no-op control, a random-budget series, whole-sequence rewriting, a matched pair targeting two immunologically relevant dinucleotides, and a dinucleotide-exact series. Rewrites are applied to held-out sequences only, and every sequence is checked to confirm its composition really is unchanged rather than assumed.
- Two independent read-outs. A task-coupled one — per-layer embeddings, mean-pooled, through one fixed linear head, scored as balanced-accuracy drop with a stratified paired bootstrap. And a label-free one — the model’s own masked-codon distribution, with no classifier head, no labels and no split at all, scored as KL divergence by edit-distance band. If both move together, the effect is not an artifact of the head.
- A reference level with no model in it. A classifier over a handful of summary statistics, plus gradient boosting on codon frequency. If a billion-parameter model cannot beat counting, that is the finding.
Two results from this that I think generalize:
Scale does not buy robustness. Across codon- and nucleotide-tokenized models spanning more than an order of magnitude in parameter count, sensitivity to composition-preserving rewrites varied widely with no monotone trend in model size, and the plain frequency baseline sat inside the same band as the largest models.
Decodability is not use. A linear probe and a direct causal intervention disagreed about which dinucleotide feature drives the decision — the probe ranked one, erasing features found the other. Probing tells you what is recoverable from a representation, not what the network computes with. I now treat probe rankings as hypotheses to be checked by intervention, never as findings.
I also read the effect through the encoder with a tuned lens, per layer, which showed the separation opening early and staying open rather than appearing at the end — consistent with an effect riding a direction the model uses throughout, rather than corrupting one late computation.
Robustness, stated carefully
The application driving this is DNA synthesis biosecurity screening: today’s screening is largely homology based, and foundation models are the proposed upgrade because they could in principle catch a sequence of concern that has been rewritten past homology search. I built such a screener and tried to defeat it.
The finding worth stating publicly is a defensive one. The most effective evasion in my hands was not an adversarial attack at all — it was an ordinary, already-published sequence-engineering practice, applied by someone with no knowledge that the model exists. Threat models built around a sophisticated adversary with gradient access are aimed at the wrong risk. Any screening proposal needs to be evaluated against routine practice first.
On defenses, I compared head-only training, LoRA and full fine-tuning, and the pattern was consistent: hardening rotates the read-out without removing the underlying signal, which stays just as recoverable after fine-tuning as before. Transfer between models was low, so stacking several models is less of a defense than it looks.
On detail: this work is unpublished and a related manuscript is under review, so I am deliberately not publishing the specific rewrite recipes, effect sizes, or which models were defeated. That is partly the usual caution about unreviewed numbers, and partly because a concrete, quantified evasion recipe for biosecurity screening is not something to put on a public web page ahead of the people who would have to fix it. Happy to discuss the details directly — email me.
Why this connects to the dissertation
My PhD is about genes with no homologs — lineage-specific genes in Populus, sequences that look like noise to every homology-based method we have. The codon work is the same problem pointed the other way: a sequence a homology search cannot recognize, where you have to decide from the sequence itself whether what you are seeing is real.
That is also where the foundation-model interest started. I evaluated seven publicly available DNA foundation models, spanning several architectures and tokenization schemes, on Populus promoter sequences — diagnosing embedding quality and establishing architecture-specific pooling. The short lesson was that how you pool a model’s embeddings is not a detail you can leave at the default — and, more usefully, that genes with no relatives are a natural probe for when these models hallucinate structure that is not there.
I also model ohnolog expression divergence with XGBoost and SHAP under leakage-aware validation, which is where I came to rely on chromosome-held-out splits and permuted-label controls. Related sequences leaking across folds manufacture performance that does not exist.
What I work with
Models & training: PyTorch, Hugging Face, LoRA and parameter-efficient fine-tuning, full fine-tuning, embedding extraction and pooling, frozen probing, masked-language-model read-outs
Interpretability & robustness: per-layer linear probing, tuned lens, activation patching and direct feature erasure, in-silico mutagenesis, representation geometry, adversarial evaluation under domain-validity constraints, adversarial training
Experimental design: exactness-asserted counterfactual construction, matched random-edit floors, model-free reference baselines, label-free evaluation that needs no classifier head, stratified paired bootstrap, Wilson intervals, permuted-label controls, chromosome-held-out splits
The dissertation side of my work is on the research page.