Biological Sequence Tokenisation | How DNA, RNA and Proteins Become Model Units

Biological sequences look deceptively simple: strings of letters. But the letters are representations of molecular units, and the way those strings are tokenised changes what a model can see efficiently.

DNA can be represented as individual nucleotides, overlapping k-mers, non-overlapping k-mers, byte-like characters or learned subwords. Protein models often use amino-acid residues directly, but higher-level motifs and domains can also become useful units. RNA adds sequence, structure and modification layers that one text-like string cannot fully capture.

This article continues the eduKateSingapore Representation and Tokenisation series. It is an educational representation article, not a laboratory protocol or clinical guide.

The Biological Sequence Route

BIOLOGICAL MOLECULE
→ MEASUREMENT / SEQUENCING
→ LETTER SEQUENCE
→ CLEANING + ORIENTATION + AMBIGUITY HANDLING
→ BASE / k-MER / RESIDUE / SUBWORD TOKENS
→ EMBEDDINGS + POSITION
→ CONTEXTUAL MODEL
→ PREDICTION / REPRESENTATION
→ BIOLOGICAL INTERPRETATION
→ EXPERIMENTAL OR DATABASE RETURN

1. The Sequence Is Already a Representation

A DNA string such as ATGC is not the molecule itself. It is a symbolic representation of nucleotide identities measured from a biological sample or reference assembly.

Sequence tokenisation begins after measurement, base calling, assembly and reference choices have already shaped the data.

2. DNA Has a Small Primitive Alphabet

The canonical DNA alphabet is usually written with A, C, G and T, with additional ambiguity symbols available when identity is uncertain or mixed.

This tiny alphabet makes single-nucleotide tokenisation straightforward but creates very long sequences.

3. One Nucleotide per Token Maximises Local Resolution

Single-base tokens preserve the finest ordinary sequence granularity. A point mutation changes one token directly.

The cost is sequence length: a 12-kilobase region requires roughly 12,000 nucleotide positions before special tokens or compression.

4. k-Mers Compress Local Context

A k-mer is a sequence of k consecutive nucleotides. A 6-mer packages six bases into one model unit.

k-mer tokenisation reduces sequence length and gives the model direct access to recurring local patterns.

5. k Controls the Vocabulary–Length Trade-Off

With four canonical DNA bases, there are 4^k possible unambiguous k-mers. Larger k produces a rapidly growing vocabulary while shortening the sequence.

The same representation economics seen in natural-language tokenisation appears in genomics.

6. DNABERT Demonstrated Overlapping k-Mer Tokenisation

DNABERT explored DNA language models using overlapping k-mer tokenisation, including 3-, 4-, 5- and 6-mers.

Overlapping k-mers preserve local redundancy but greatly increase sequence length compared with non-overlapping groups.

7. Overlapping k-Mers Duplicate Local Evidence

In an overlapping 6-mer representation, one nucleotide appears in several neighbouring tokens.

This gives strong local context but means masking or evaluation must account for information leakage from overlapping neighbours.

8. Non-Overlapping k-Mers Compress More Aggressively

A non-overlapping 6-mer representation reduces the number of model positions by about a factor of six for ordinary unambiguous sequence.

The trade-off is that one token boundary now covers six nucleotide positions together.

9. Nucleotide Transformer Uses 6-Mer Tokens

The Nucleotide Transformer family uses mainly non-overlapping 6-mers, with fallback handling for individual nucleotides and ambiguity. Its published work illustrates how token length, context window and model architecture jointly determine the amount of genomic sequence visible to the model.

The representation choice is therefore inseparable from model context design.

10. Token Boundaries Can Hide Single-Nucleotide Resolution

If six nucleotides share one token, the model’s basic output positions are coarser than one base.

Downstream decoders can recover nucleotide-level predictions, but that requires an explicit representation bridge.

11. Reading Frame Is Not the Same as Token Boundary

Protein-coding DNA is read biologically in codons of three nucleotides within a specific reading frame. A tokenizer using 6-mers, BPE pieces or single bases need not align with codon boundaries.

Computational tokenisation and biological translation grammar are different layers.

12. Codons Are Biologically Meaningful Triplets

A codon represents three nucleotides interpreted by the translation machinery within an open reading frame.

Codon tokenisation can be natural for some protein-coding tasks but is poorly matched to non-coding DNA and frame-uncertain sequence.

13. The Same DNA Has Several Reading Frames

A double-stranded DNA region can be read in multiple frames depending on strand and starting position.

A fixed codon segmentation therefore assumes biological orientation that may not be appropriate for every task.

14. Strand Orientation Matters

DNA is double-stranded, and the reverse complement represents the opposite strand orientation.

A sequence model should state whether orientation is canonicalised, augmented or treated as distinct input.

15. Reverse Complements Expose Representation Symmetry

The same physical genomic locus can be represented by one sequence or its reverse complement depending on strand convention.

Models that should be strand-invariant need evaluation for that symmetry rather than assuming it.

16. Ambiguous Bases Are Real Information

Symbols such as N can indicate an unknown or unresolved nucleotide. Replacing all ambiguity with an ordinary base invents certainty.

Unknown state should remain explicitly represented.

17. Sequencing Quality Is Upstream Evidence

A high-confidence base call and a low-confidence base call can produce the same letter in a simplified FASTA representation.

If confidence matters, quality information must travel alongside the sequence rather than being discarded before tokenisation.

18. Reference Genome Coordinates Add Provenance

A sequence fragment becomes easier to verify when the assembly version, chromosome and coordinate interval are known.

A biological token should be able to return to its source locus when claims depend on exact sequence identity.

19. Genome Assembly Version Matters

Coordinates can refer to different sequences across assembly versions.

Sequence identity without reference version can create silent mismatches.

20. Variant Representation Needs Reference Context

A variant is defined relative to a reference sequence and coordinate system.

The changed token alone does not tell the full biological claim.

21. Insertions and Deletions Shift Naive Token Alignment

Adding or removing one nucleotide can move every later fixed-width token boundary.

Comparing token sequences directly can therefore exaggerate local biological changes.

22. Learned Subwords Can Adapt to Recurrent Sequence Motifs

BPE-like tokenizers can learn frequently recurring DNA fragments rather than fixing one k for every location.

This creates variable-length biological tokens but makes nucleotide-level alignment less uniform.

23. Variable-Length Tokens Move Boundaries With the Sequence

A mutation can change which surrounding fragment is chosen by the tokenizer.

Downstream interpretation should therefore distinguish biological change from token-boundary change.

24. RNA Adds Alphabet and Structure Differences

RNA typically uses U rather than T in its canonical sequence alphabet, but biological RNA also carries secondary structure, processing and modification information not contained in the base string alone.

Sequence-only tokenisation is therefore necessarily partial.

25. RNA Secondary Structure Is Relational

Base pairing connects distant sequence positions. A simple left-to-right token stream hides those long-range structural relationships.

Graph or paired-position representations can preserve structure more directly.

26. Protein Sequences Use Amino-Acid Residues

Proteins are commonly represented as one-letter amino-acid sequences. One residue per token is therefore a natural primitive representation for many protein language models.

The sequence still omits conformation, modifications and environmental state.

27. Amino-Acid Tokens Are Biologically More Direct Than Natural-Language Words

Each standard amino-acid letter corresponds to a residue identity in the protein sequence. But a residue’s functional role emerges from context, structure and interaction.

Primitive identity is not functional meaning.

28. Protein Language Models Learn Context Across Residues

Protein language models such as ESM-family systems learn contextual residue representations from large sequence corpora. Recent literature continues to emphasise that tokenisation, embeddings and biological interpretation are separate design layers.

See the Nature Machine Intelligence perspective on biologically reliable protein language models.

29. Protein Motifs Are Higher-Level Sequence Tokens

A short motif can recur across proteins and participate in binding or regulation.

Motif-level representation compresses several residues into one biologically interpretable pattern class.

30. Domains Are Coarser Functional Units

Protein domains can span dozens or hundreds of residues and often correspond to reusable structural or functional units.

Domain tokens sit much higher on the granularity ladder than residue tokens.

31. Sequence Does Not Equal Structure

A protein’s three-dimensional structure emerges from sequence plus physical and environmental constraints.

Sequence models can predict aspects of structure, but the sequence token stream itself is not the folded molecule.

32. Structure Adds Spatial and Graph Representations

Atoms or residues can become nodes and spatial contacts edges.

This connects biological sequences to Graph Tokenisation and spatial representation.

33. Long Genomes Exceed Ordinary Model Context

A human chromosome is far longer than the context of typical sequence models.

Windowing, hierarchical processing and compressed tokens decide which long-range relationships remain visible.

34. Window Boundaries Can Split Regulatory Context

A promoter, enhancer and target gene can be separated by long genomic distances.

Fixed windows are computationally convenient and biologically imperfect.

35. Overlap Protects Sequence Boundaries

Overlapping genomic windows allow a motif near one edge to appear centrally in another window.

The cost is duplicated sequence and computation.

36. Context Length Defines Biological Horizon

A model cannot directly integrate relationships beyond the sequence span available to its architecture without an additional memory or hierarchical mechanism.

Biological interpretation should respect that horizon.

37. Species Context Matters

Identical short motifs can have different prevalence or function across species and genomic contexts.

Training-corpus composition shapes what a biological language model treats as ordinary.

38. Sequence Frequency Is Not Functional Importance

A rare regulatory sequence can be biologically crucial. A repetitive element can be extremely frequent.

Tokenizer frequency and biological significance are distinct.

39. Model Tokens Are Not Genes

A gene can span many thousands of nucleotide tokens and contain multiple regulatory and transcribed regions.

Calling k-mers “biological words” is a useful analogy only if its limits remain visible.

40. Model Tokens Are Not Functional Elements

A six-base token can overlap part of a splice site, motif or neutral region. The token boundary does not declare biological function.

Function is a higher-level inference.

41. Biological Predictions Need Independent Validation

A model can generate a high-confidence prediction from sequence and still be wrong biologically.

Prediction should be checked against appropriate databases, experiments or domain evidence depending on the use case.

42. Tokenisation Benchmarks Should Match the Biological Task

Evaluate nucleotide resolution, mutation sensitivity, long-range context, species transfer, protein residue tasks or structural prediction separately.

No single tokenisation wins every biological problem.

43. The Biological Sequence Tokenisation Audit

What biological molecule is represented?
What sequencing or reference source produced the string?
What assembly or database version applies?
What alphabet and ambiguity symbols are allowed?
Is orientation or reverse complement handled explicitly?
Are tokens single bases, fixed k-mers, residues or learned subwords?
Are k-mers overlapping or non-overlapping?
Does token granularity align with the prediction resolution?
How are insertions, deletions and unknown bases represented?
What context length is biologically visible?
Do windows split known long-range relationships?
Are species and training-corpus biases measured?
Is sequence identity separated from structure and function?
Can predictions return to exact source coordinates?
What independent evidence validates the biological claim?

44. What Students Should Remember

Biological sequences are symbolic representations of molecules.
Single bases preserve detail but create long sequences.
k-mers compress local context while increasing vocabulary size.
Codons are biological units only under a valid reading frame.
RNA and proteins require additional structural representations beyond sequence.
Token boundaries do not prove biological function.
Genome version, orientation and source provenance matter.
Model predictions require biological validation.

45. The Deep Principle

Biological tokenisation converts molecular sequence into a finite computational alphabet. Its success depends on keeping one distinction clear: the token is a representation of sequence, while biological function emerges from relationships that can extend far beyond the token boundary.

A nucleotide token tells the model which letters are present. Biology begins when those letters are interpreted in strand, sequence, structural, evolutionary and experimental context.

Biological Sequence Tokenisation | How DNA, RNA and Proteins Become Model-Ready Units