Advanced Technical Expansion I | Probe Mathematics, Capacity, Selectivity and Minimum Description Length

The readable spine establishes that probes are measurement instruments whose conclusions depend on the receiver. This expansion turns that principle into mathematics. We derive the ordinary linear probe, show how sample size and margin affect interpretation, connect selectivity to control-task risk, and unpack why information-theoretic and MDL approaches measure a different property from endpoint accuracy. The aim is not to crown one probing method. It is to make every probe result explicit about the work performed by the representation and the work performed by the probe.

A. Binary linear probing begins with one geometric question

Let representation h_i∈R^d and binary label y_i∈{0,1}. Logistic probing chooses weights w and bias b to minimise cross-entropy:

L(w,b) = -Σ_i [ y_i log σ(wᵀh_i+b)
              + (1-y_i) log(1-σ(wᵀh_i+b)) ].

If the classes are well separated by one hyperplane, a small-norm w can achieve strong accuracy and margin. If the representation entangles the classes nonlinearly, linear performance can remain poor even when a nonlinear classifier succeeds.

The probe therefore measures one property of the representation geometry: whether the target aligns with an affine partition.

B. Multiclass probing creates a set of separating directions

For K classes, the probe uses weight matrix W∈R^{K×d}. Each row defines one class score. Predictions depend on score differences, so adding the same vector to every row does not change decisions. The physically meaningful object is the subspace of class contrasts rather than every raw coefficient.

This matters when interpreting probe directions. One class’s weight vector is not an isolated concept axis; decisions arise from relative logits.

C. Margin distinguishes robust accessibility from knife-edge separability

Suppose two representations both achieve 95 percent test accuracy. In representation A, examples sit far from the learned hyperplane. In B, most sit very close. Small activation noise can flip B’s predictions much more easily.

Margin is therefore an accessibility-strength diagnostic. Report signed margin distributions after appropriate feature normalisation. A large margin under a low-capacity probe can indicate information arranged in a stable, easy-to-read geometry.

Margins are scale-dependent. Standardise or otherwise document the representation scale before comparing them across layers.

D. L2 regularisation defines how much probe complexity costs

Add λ||w||² to the loss. Larger λ discourages large coefficients and can improve generalisation when labels are limited. The regularisation strength changes the receiver. A heavily regularised linear probe asks whether the property is accessible with a smooth, low-norm decision; a weakly regularised probe can exploit smaller irregularities.

Hyperparameter selection therefore needs a validation policy that does not quietly give one representation more tuning effort than another.

E. Feature standardisation can alter which directions look simple

If one activation coordinate has variance one thousand times another, an L2-regularised probe can favour the high-variance coordinate because it needs a smaller coefficient. Standardising dimensions changes that bias.

Raw neural scale may itself be meaningful. There is no universally correct preprocessing. Report whether activations were centred, standardised, normalised or whitened.

Probe simplicity is defined relative to the coordinate gauge supplied to the probe.

F. Whitening makes probe complexity closer to subspace complexity

Whiten h so covariance is approximately identity. Linear directions then have comparable variance scale. This can help compare probe norms across representations and can amplify low-variance directions that the native model itself treats weakly.

Whitening therefore changes the receiver. A raw linear head and a whitened linear head do not answer the same deployment question.

G. Probe sample complexity is a property of geometry plus regularisation

Train the same probe with increasing labelled examples. If a property is cleanly linearly separated, useful performance can appear with very little supervision. If the property is present only as a weak statistical signal, the probe may need many examples to estimate the direction reliably.

The learning curve is therefore a representation diagnostic. Endpoint accuracy after enormous supervision can hide large differences in accessibility.

H. Cross-validation must be nested when probe hyperparameters are tuned

If λ, architecture, early stopping and layer are all selected on the same test set later reported as evidence, the probe result is optimistically biased. Use train/validation/test splits, and when comparing many layers or probe capacities, treat model selection as part of the experiment.

Layer-wise probing can involve dozens or hundreds of comparisons. The best layer’s score is an extreme selected from many noisy estimates.

I. A control task estimates what the probe can learn without target structure

Hewitt and Liang’s selectivity idea can be written schematically as real-task performance minus control-task performance, though exact implementations can use task-appropriate metrics. The central logic is comparative: a probe should exploit representation structure more easily than it memorises arbitrary labels.

As probe capacity rises, both numbers can rise. Selectivity asks whether meaningful structure retains an advantage.

J. Random-label memorisation has a sample-size dependence

A high-capacity probe can memorise random training labels and still fail on held-out control labels because there is no generalisable pattern. Control design must distinguish train memorisation from test performance.

Word-type control tasks are more subtle because repeated word identities create generalisable arbitrary mappings across contexts. The probe can learn a random lexicon and succeed on held-out tokens whose word types were seen during training.

This is exactly why token-identity shortcuts matter in linguistic probing.

K. Control tasks should match nuisance information available to the real task

If the real POS task can be solved partly from word identity, a useful control retains word identity while randomising the label association. If the control removes identity entirely, it may become artificially difficult and exaggerate selectivity.

Designing the control is an attempt to isolate the representation structure being claimed while preserving other learning opportunities.

L. Selectivity is not one universal subtraction

Accuracy, F1, log loss and code length have different scales. Selectivity should be defined in a metric compatible with the task and control. A ten-point accuracy gap under heavy class imbalance can mean something different from a ten-point balanced-accuracy gap.

Report the two underlying scores in addition to their difference.

M. Information-theoretic probing asks about uncertainty reduction

Mutual information can be written I(Y;H)=H(Y)-H(Y|H). A representation is informative about Y when knowing H reduces uncertainty about Y. A predictive model q(y|h) can provide an upper bound on conditional entropy through cross-entropy, and a better probe can tighten that estimate.

This is why Pimentel and colleagues argue that more expressive probes can be desirable for estimating information: underfitting the probe can make an informative representation look empty.

The information question is coordinate-independent in ways linear accessibility is not. It is also harder to estimate faithfully.

N. Mutual information does not measure convenience for a downstream head

An invertible scrambling of a representation preserves mutual information with Y. It can turn a linearly separable label into a complicated nonlinear pattern. From an information perspective nothing was lost; from a linear receiver’s perspective accessibility collapsed.

This is the cleanest demonstration that information content and representational usability are distinct.

O. Minimum Description Length measures how expensive the probe is to teach

In MDL probing, labels are imagined as a message to be transmitted to a receiver who already has representations. A representation is better when fewer bits are needed to communicate the labels.

The description length includes predictive uncertainty and, depending on the coding scheme, the cost of learning the decoder. A representation that needs fewer labelled examples to establish a reliable mapping yields shorter code.

P. Online coding makes model learning cost observable

Split labelled data into sequential blocks. Before seeing each block, train the probe on previous blocks and encode the next labels using predicted probabilities. Early blocks are expensive because little supervision has been seen; later blocks become cheaper if the representation exposes the task.

The total code length integrates the learning curve instead of looking only at its final point.

Q. A uniform code supplies the initial baseline

Before the receiver has any examples, K balanced classes can be encoded at roughly log₂K bits per label using a uniform model. A good representation reduces code length quickly as labelled examples arrive.

Class imbalance changes the appropriate baseline. Use empirical or explicitly specified priors rather than assuming balance.

R. Compression score can make MDL comparisons easier to read

Some probing work reports compression relative to a uniform or label-only code. Larger compression means the representation allows labels to be encoded more efficiently. Always state the baseline and code scheme.

Compression is receiver-specific: a linear online coder and nonlinear online coder can produce different results.

S. Variational coding introduces a model-complexity term explicitly

Variational MDL approaches encode model parameters relative to a prior plus data given the sampled model. This makes the cost of a complex probe explicit rather than relying only on data-efficiency.

Prior choice and approximation quality become part of the metric. MDL is principled and not assumption-free.

T. Information gain and selectivity can be two views of the same tension

Zhu and Rudzicz show a formal relationship between the selectivity and information-gain criteria under their setup. The conceptual tension is shared: distinguish structure genuinely made available by the representation from work performed by a flexible decoder.

This is useful because it moves the debate away from slogans such as “simple probes are always better” or “powerful probes are always better”. The correct probe follows the estimand.

U. The estimand should be written before the probe architecture

• Linear accessibility: can one affine readout recover Y?
• Low-complexity accessibility: can a constrained receiver recover Y?
• Information presence: can a sufficiently expressive receiver reduce uncertainty about Y?
• Sample efficiency: how much supervision is needed?
• Coordinate sparsity: can a small set of native axes recover Y?
• Subspace complexity: how many directions are needed?
• Causal use: does changing the relevant representation change the base model’s behaviour?

Each bullet can lead to a different best probe.

V. Low-rank multiclass probes estimate target subspace dimension

Factor the probe matrix as W=AB with rank r. Sweep r and measure performance. If rank two captures almost all performance for a ten-class property, the target can be represented through a compact contrast subspace. If performance rises steadily with rank, access is more distributed.

This does not reveal the true feature count; it measures the readout dimension needed by the chosen linear classifier.

W. Coordinate-sparse probes reveal basis alignment rather than intrinsic simplicity

Add an L1 penalty on w. If high performance survives with a few nonzero coefficients, the property aligns with a small set of native coordinates. Rotate the representation orthogonally and that sparsity can disappear while linear separability remains unchanged.

Sparse-probe interpretability is therefore basis-dependent. This is useful when native neurons are the operational interface and should not be confused with coordinate-free feature simplicity.

X. Group sparsity can test structured sets of coordinates

If representation coordinates come in heads, channels, spatial regions or feature groups, group-lasso style probes can test whether a small number of groups suffice. This respects architecture structure better than selecting individual scalar dimensions.

The group definition itself encodes a hypothesis about where coherent information should live.

Y. Calibration reveals whether probe probabilities are meaningful

A probe can classify accurately and produce badly calibrated probabilities. If it will be used as a monitor or measurement device, reliability diagrams, expected calibration error and proper scoring rules matter.

Calibration depends on class prevalence and shift. A probe calibrated on one domain can become overconfident on another.

Z. Probe uncertainty has several sources

Uncertainty in a probing conclusion comes from finite labelled data, stochastic probe training, finite representation samples, uncertain labels and model stochasticity. Repeat training seeds and bootstrap examples.

Layer differences of a fraction of a percent should not receive mechanistic stories if confidence intervals overlap broadly.

AA. A capacity–selectivity phase diagram is more informative than one probe

Train probes across hidden width, depth, regularisation and training-data size. For each, plot real-task performance against control performance. The ideal region combines high target accessibility with low arbitrary-task learning.

Different representation layers can occupy different parts of this diagram. A later layer can have slightly lower raw accuracy and much stronger selectivity.

AB. Worked laboratory: linearly accessible versus nonlinearly recoverable

Generate 2D points arranged in concentric circles with label determined by radius. The label is perfectly recoverable and not linearly separable. A linear probe fails; a small nonlinear MLP succeeds.

Now map radius into one coordinate explicitly. The linear probe succeeds. Both representations contain the target information; only the second makes it linearly accessible.

This synthetic example is the cleanest introduction to the difference between information and accessibility.

AC. Worked laboratory: random-label memorisation under capacity sweeps

Create a real target with simple linear structure and a random-label control tied to repeated input identities. Train linear, shallow and larger nonlinear probes. Track train and test performance on both tasks.

As capacity grows, control memorisation can rise. The meaningful target can remain easy even for simple probes. The gap illustrates selectivity operationally.

AD. Worked laboratory: same endpoint accuracy, different MDL

Construct two representations. In A, the target is separated by a large linear margin. In B, the target has a much smaller margin but remains linearly separable. With full data, both probes reach similar accuracy. Under small labelled subsets, A learns faster.

Online coding assigns A shorter description length, capturing accessibility missed by endpoint accuracy.

AE. Worked laboratory: basis rotation and sparse-probe instability

Create a representation where the target equals one coordinate. An L1 probe selects that coordinate cleanly. Apply a random orthogonal rotation. Linear accuracy remains unchanged, but the sparse probe now uses many coordinates.

The experiment demonstrates that coordinate sparsity is an alignment property, not intrinsic information complexity.

AF. The probing-mathematics evidence card

• Estimand: accessibility, information, sample efficiency, sparsity or causal use.
• Probe family: linear, low-rank, sparse, MLP or other.
• Preprocessing: centering, standardisation, whitening and pooling.
• Regularisation: strength and selection policy.
• Data: train/validation/test sizes and group independence.
• Controls: random labels, identity controls and trivial baselines.
• Learning curve: performance over supervision size.
• Margin/calibration: strength and reliability of access.
• MDL: code scheme and baseline if used.
• Uncertainty: bootstrap and seed variation.
• Coordinate dependence: which transformations change the conclusion.

The technical conclusion is that probing does not have one “correct complexity”. The probe is part of the question. A linear receiver measures linear geometry. A powerful receiver estimates broader recoverability. A sparse receiver tests basis alignment. MDL measures how efficiently a receiver can learn. Control tasks test whether the receiver can manufacture apparent knowledge. The next layer asks how these measurements should be designed, validated and escalated into causal claims.

Advanced Technical Expansion II | Experimental Design, Causal Escalation and Probe Reliability

A probe becomes trustworthy when its conclusion survives alternative explanations. This second expansion focuses on research design: how to choose labels, split data, prevent lexical or identity leakage, compare across layers and groups, and decide when a decodability result should be escalated to an intervention such as activation patching. The goal is a complete evidence ladder from “a classifier can read this” to “this representation participates in the model’s computation in the way the label suggests.”

AG. Define the target variable operationally before collecting activations

Labels such as “syntax”, “truth”, “uncertainty”, “planning” or “toxicity” are too broad until converted into reproducible annotation rules. A probe can only measure the target actually supplied.

Write inclusion, exclusion and ambiguous cases. If human annotators are involved, measure agreement. If the target is generated by another model, document that model and its known failure modes.

Ambiguous ontology creates ambiguous probing conclusions no matter how precise the classifier.

AH. Split data by the unit that could leak information

If the same word type appears in train and test, a probe can memorise lexical associations. If the same image subject appears in different crops across splits, an instance-specific probe can memorise identity. If adjacent frames cross splits, near duplicates leak.

Group splits by document, speaker, subject, image source, patient, code repository or whatever unit could carry a shortcut. Random row splitting is often insufficient.

AI. Lexical memorisation should be measured explicitly

For linguistic properties, create a held-out-word evaluation where test word types never appear during probe training. Compare with ordinary random-token evaluation. A large drop reveals reliance on lexical identity.

This does not make the representation useless. Lexical identity may be a legitimate source of POS or morphology. It narrows the claim from contextual abstraction to memorised or encoded type information.

AJ. Ambiguous-token subsets isolate contextual structure

Select words or entities whose label varies by context. A probe cannot solve these examples from type identity alone. Performance on ambiguous subsets therefore provides stronger evidence for contextual representation.

Stratify by ambiguity degree. One label changing twice in a corpus is a weaker test than a balanced set of genuinely ambiguous uses.

AK. Counterbalanced datasets break spurious correlations by design

If sentiment correlates with topic, create matched examples crossing positive/negative sentiment with the same topics. If object identity correlates with background, vary background independently. If uncertainty language correlates with question difficulty, construct easy and hard examples with both certainty states.

Counterbalancing is stronger than statistically controlling for a confound after training because it gives the probe direct evidence that the variables can vary independently.

AL. Synthetic controls can establish whether the pipeline recovers known planted structure

Before probing a mysterious model feature, inject or construct a known signal with controlled accessibility. Verify that the probe family detects a linear feature, misses a deliberately nonlinear feature, and reacts predictably to noise.

This unit-tests preprocessing, labels, regularisation and evaluation code. If the pipeline fails on planted ground truth, conclusions from real activations are premature.

AM. Negative controls test whether the pipeline invents structure

Shuffle labels, randomise representation rows, probe an unrelated layer or use random network activations. A robust pipeline should fall toward the appropriate baseline.

If strong accuracy survives label shuffling, there is leakage or evaluation error. If random representations perform almost as well as trained ones, the target may be easy from raw input structure or probe capacity.

AN. Positive controls prevent overly sceptical probe designs

A probe can be made so restricted or regularised that it fails to detect structure known to exist. Include a representation where the target is explicitly available, such as one-hot labels or a planted feature direction, to verify the measurement has enough sensitivity.

Scientific controls should test both false positives and false negatives.

AO. Layer sweeps create multiple-comparison problems

Probe 96 layers and report the maximum accuracy: even noisy differences can create an apparent “best layer”. Use bootstrap intervals and validate the selected layer on a second dataset or held-out run.

Better still, preregister hypotheses about early/middle/late stages when theory supports them and treat full heatmaps as exploratory.

AP. Compare layer curves across probe capacities

A property can appear early under a nonlinear probe and later under a linear probe. This suggests the information exists earlier but becomes progressively easier to read.

Plot layer on one axis and probe capacity on another. The resulting accessibility surface is more informative than one line.

This can reveal representational “untangling” through depth without claiming the property first comes into existence at the linear-probe peak.

AQ. Probe the same property at several token positions

For autoregressive models, compare the source token, subsequent tokens and final answer token. Information can be copied, transformed or summarised through the sequence.

A property decodable at the final position may have arrived through attention from earlier locations. Position-wise probe maps provide descriptive routing evidence that can later guide causal tracing.

AR. Temporal order matters in reasoning tasks

When a chain-of-thought or multi-step computation unfolds over generated tokens, a static probe on one final state can miss when information became available. Probe trajectories across steps.

Do not interpret late decodability as late computation without checking whether the signal existed earlier in another form.

AS. Cross-layer probe transfer tests whether the readout coordinate system stays stable

Train a linear probe on layer ℓ and apply it directly to layer ℓ+1 after matching dimensionality where necessary. If performance remains high, the target direction may persist across layers. If retraining is needed, the information can remain present while its coordinate organisation changes.

This test complements CKA and representation similarity. It is receiver-specific and directional.

AT. Cross-model probe transfer tests shared interfaces

After aligning representation spaces or using models with compatible activations, train a probe on model A and apply it to model B. Successful transfer suggests the property occupies a comparable interface.

Failure can mean the models lack the same information, encode it differently, or the alignment is poor. Use within-model probe accuracy and representation-similarity baselines to separate these possibilities.

AU. Probe performance across random seeds measures representation reproducibility

Train the base model multiple times when possible. If the property is equally accessible but through different probe directions, the model family learns stable information with unstable coordinates. If accessibility itself varies, the property is not a robust consequence of training.

Report between-model variance separately from probe-training variance.

AV. Domain shift tests whether accessibility is robust or dataset-specific

Train the probe on one corpus and test on another without retraining. A property can be linearly accessible within each domain through different shortcut directions yet fail cross-domain transfer.

Cross-domain generalisation is stronger evidence that the representation exposes a reusable variable rather than a dataset-specific correlation.

AW. Calibration under shift matters if probes become monitors

A diagnostic probe may later be used to flag unsafe or uncertain internal states. A monitoring receiver needs calibrated probabilities, low false-positive rates and explicit abstention under unfamiliar distributions.

Probe accuracy from a research benchmark is not enough to deploy a monitor. Evaluate base rates, threshold stability and drift.

AX. Rare-property probing needs precision–recall, not only accuracy

If the property occurs in one percent of examples, a classifier can achieve 99 percent accuracy by always predicting absence. Report precision, recall, PR-AUC and calibrated operating points.

For rare safety features, false positives can dominate operator workload. The relevant probe metric follows the deployment consequence.

AY. Probe labels can encode protected or sensitive attributes unintentionally

Representations can make demographic or private information decodable even when the original task did not need it. Probing can reveal such leakage. The analysis should use appropriate privacy, consent and data-governance controls.

Decodability of a sensitive attribute is evidence of potential exposure, not proof that the model uses or can reveal the attribute in every context.

AZ. Probe failure does not prove absence of information

A linear probe can fail because the information is nonlinear. A nonlinear probe can fail because data are insufficient, optimisation is poor or labels are noisy. Negative probing results need sensitivity controls.

The strongest absence claim requires a broad receiver class, sufficient data and a demonstrated ability to recover comparable planted signals.

BA. Probe success does not prove a compact concept exists

A high-capacity probe can aggregate thousands of weak distributed signals. The target can be decodable without one stable concept direction, neuron or sparse feature.

To claim a compact internal concept, add dimensionality, sparsity, stability and intervention evidence.

BB. Probe directions should be compared with native model readout directions

If the model has an explicit classifier or output head for a related task, compare probe directions with native readout weights. Alignment suggests the model’s own computation may use similar geometry.

Misalignment can indicate the information is available but the base model chooses another readout. Causal tests are still needed.

BC. Activation patching is the natural next escalation

Once a probe locates where a property is accessible, create clean and counterfactual examples differing in that property. Replace internal activations across runs and observe whether the target behaviour changes.

This tests whether the representation site lies on a causal pathway. It does not automatically prove the probe direction itself is the mediator.

The next article develops activation patching as its own causal measurement system.

BD. Directional intervention tests the probe’s own feature hypothesis

For a linear probe with direction w, move the representation a small amount along ±w and measure base-model output. This is a stronger test of the probe direction than patching the whole activation.

But w can point off the natural activation manifold and can combine correlated features. Use natural-range magnitudes, random-direction controls and counterbalanced inputs.

BE. Conditional probes can distinguish context-specific accessibility

Train separate probes by domain, task state or context and compare directions. If the same property requires different readouts in different contexts, one global probe can average incompatible geometries.

This observation connects directly to 2026 work on context-aware representation steering: effective directions can vary locally through representation space.

BF. Probe agreement across methods strengthens conclusions

Use linear accuracy, MDL, control selectivity and perhaps a nonlinear upper-bound probe on the same target. If all indicate the same layer ordering, the conclusion is more robust to measurement choice.

Method disagreement should be explained structurally rather than averaged away. For example, high nonlinear information with weak linear accessibility is a meaningful result.

BG. A probe evidence ladder prevents overclaiming

• Level 1 — Above baseline: target can be predicted from the representation.
• Level 2 — Selective: prediction exceeds matched control-task learning.
• Level 3 — Efficient: property is learned with low probe complexity or short MDL.
• Level 4 — Robust: result survives data, seed, subgroup and domain changes.
• Level 5 — Compact: target lives in a stable low-dimensional or sparse interface.
• Level 6 — Causal: interventions on the relevant state affect base-model behaviour as predicted.
• Level 7 — Mechanistic: the feature is linked into a validated computational circuit.

Not every study needs all seven levels. The public language should match the level actually reached.

BH. Worked experiment: lexical shortcut versus contextual syntax

Build three splits for POS probing: random tokens, held-out word types, and ambiguous word types whose POS varies by context. Train the same linear probe on each layer.

High random-split accuracy with large held-out-type drop indicates lexical memorisation. Strong ambiguous-word performance provides stronger evidence for contextual syntax. Compare selectivity with a random word-type control task.

BI. Worked experiment: probe capacity × layer heatmap

Across every layer, train a linear probe, rank-4 probe, shallow MLP and larger MLP. Plot test accuracy and control selectivity.

One possible pattern: information becomes nonlinearly recoverable early, linearly accessible in middle layers, then remains available but less selective late. The heatmap prevents one probe family from defining the whole representation story.

BJ. Worked experiment: probe success without causal use

Train a network on inputs containing target-relevant feature A and correlated feature B, while architecturally forcing the final classifier to read only A. An intermediate representation can encode both. Probes predict both strongly.

Intervene on B while preserving A. The model output stays stable. Probe success revealed information, not causal use. This synthetic control should be part of any course on mechanistic probing.

BK. Worked experiment: nonlinear information that becomes linearly untangled

Construct a network that maps concentric-ring inputs through layers until classes become linearly separable. Probe each layer with linear and nonlinear classifiers.

Nonlinear probes succeed throughout; linear accuracy rises with depth. The network did not necessarily “learn the label” only when linear probing rose—it reorganised existing information into an easier geometry.

BL. Worked experiment: MDL under domain shift

Compute online-coding MDL for a target on domain A, then train the same probe curriculum on domain B. A representation can have short code within A and long cross-domain code because the readout depends on domain-specific directions.

This distinguishes easy local accessibility from reusable interface structure.

BM. The probe research-design evidence card

• Target ontology: exact annotation rule and ambiguity.
• Leakage unit: document, word type, person, image source or other group split.
• Counterbalancing: confounds deliberately broken.
• Positive/negative controls: known structure and known absence.
• Layer/position family: multiplicity and selection policy.
• Capacity ladder: probe families and complexity.
• Cross-domain transfer: receiver stability.
• Subgroups: heterogeneous accessibility and calibration.
• Rare states: precision–recall and targeted evaluation.
• Seed uncertainty: base model and probe model.
• Causal escalation: patching or directional interventions.
• Claim level: decodable, accessible, compact, causal or mechanistic.

BN. Final longform release gate for probing

1. The article distinguishes information presence from linear accessibility.
2. It explains why probe capacity is part of the estimand.
3. It includes Hewitt–Liang controls, Pimentel information probing and MDL rather than presenting one side of the debate.
4. It treats selectivity, sample efficiency and endpoint accuracy separately.
5. It covers lexical identity and label confounds.
6. It specifies layer, token position and hook point.
7. It discusses finite-sample uncertainty and multiple comparisons.
8. It includes random and planted controls.
9. It distinguishes native concept features from probe-created readout directions.
10. It connects probing to representation similarity and sparse feature analysis.
11. It shows when probe results should escalate to causal intervention.
12. It does not call decodability proof of model use.
13. It includes receiver-specific operational evaluation for probes used as monitors.
14. It provides worked synthetic counterexamples.
15. It keeps claims scoped to the target, dataset, receiver and model site tested.

The longform conclusion is deliberately conservative and useful. Probing is one of the most powerful low-cost tools for understanding learned representations because it turns an internal state into a measurable interface. It is also one of the easiest tools to overinterpret. The same high score can mean a clean linear feature, a weak distributed trace exploited with large supervision, a lexical shortcut, or a task learned substantially by the diagnostic model. Good probing research therefore designs the receiver and the controls together.

A probe tells us what a representation permits a chosen reader to recover. The moment we claim what the original model computes, believes or uses, we have moved beyond probing and owe the reader causal evidence.

Advanced Casebook I | What Probing Looks Like Across Language, Vision, Audio, Multimodal Models, World Models and Safety Analysis

The abstract debate over probe capacity becomes much clearer when the receiver job changes. A part-of-speech probe, an object-pose probe, an audio-speaker probe, a world-state probe and a safety monitor all attach classifiers to internal representations, but they do not ask the same question. This casebook therefore treats probing as a family of scientific instruments whose design must follow the structure, geometry, timing and risk of the property being measured.

BO. Regression probes test continuous accessibility

Not every target is categorical. Depth, age, distance, angle, reward, uncertainty or physical state can be continuous. A linear regression probe asks whether the target is approximately an affine function of the representation. Report mean absolute error or task-appropriate loss, and compare with a baseline using input statistics or label mean.

A low regression error does not imply the model stores one scalar “depth neuron”. The target can be distributed across many coordinates. For periodic variables such as angle, direct scalar regression can create a seam; predicting sine and cosine can better match the target geometry.

BP. Ordinal probes preserve ordering without pretending intervals are equal

Some properties have ordered categories: low, medium, high; beginner, intermediate, advanced; stages of a process. Treating them as independent classes ignores order, while numeric regression assumes equal spacing. Ordinal logistic or ranking probes can test whether the representation preserves order while avoiding an unsupported interval metric.

The probe family therefore encodes target semantics. The receiver should not inherit a numerical geometry merely because ordinary regression software is convenient.

BQ. Multi-label probes test simultaneous properties rather than one winner

An image can contain several objects, a text can express several topics, and a hidden state can contain several active semantic features. Multi-label probing uses one or more binary outputs rather than forcing mutually exclusive classes. Measure per-label precision and recall as well as macro and micro averages.

A shared linear head can reveal whether the properties occupy a common low-dimensional subspace. Separate one-vs-rest probes can reveal each property’s accessibility but make comparisons across labels sensitive to prevalence and regularisation.

BR. Structured-output probes test relations among labels

A dependency tree, segmentation mask, graph relation or sequence tag pattern has structure beyond independent labels. A structured probe can test whether internal representations expose the variables needed to reconstruct that structure through a constrained decoder.

The more structure supplied by the decoder, the more the probe itself contributes. A probe with a strong parser can recover a tree from weak token features. Compare against simpler local readouts and control representations so the division of labour remains visible.

BS. Sequence-label probes need a decision about context available to the probe

A token label can be predicted from that token’s representation alone or from representations of neighbouring tokens. A contextual probe can learn its own sequence model and obscure whether the base representation already integrated context.

If the scientific question is “does token state h_t already expose the property?”, use a local readout. If the receiver legitimately gets the whole sequence, a structured contextual probe is appropriate. State the receiver’s field of view explicitly.

BT. Span probes test whether boundaries are represented

Named entities, phrases and events occupy spans rather than single positions. A span probe can combine start and end states, mean-pool inside the span, or learn a span representation. Each construction adds assumptions about how information is distributed across tokens.

Boundary detection and span classification should be separated. A representation can identify which words belong together yet encode the entity type only at one position or in a pooled sequence state.

BU. Pairwise probes test relations rather than properties of one item

Coreference, semantic similarity, syntactic dependency and spatial relation depend on two representations. A pair probe can use concatenation, difference, dot product, bilinear form or distance. The combination rule determines which relation geometry the probe can exploit.

A bilinear probe can detect interactions invisible to independent linear probes. It also adds many parameters. Compare simple symmetric functions when the relation itself is symmetric and ordered constructions when direction matters.

BV. Graph probes test whether relational state can be reconstructed

For a graph-like internal representation, probe node attributes, edge existence and global graph properties separately. A model can encode node category well and relations poorly, or vice versa.

Edge prediction from node embeddings can be deceptively easy when graph degree or position leaks labels. Use degree-matched negative edges and held-out graph structures. The control task should preserve trivial graph statistics while removing the target relation.

BW. Set representations require permutation-aware probes

If a model represents a set of objects or slots, the probe should not depend on arbitrary slot order unless identity is aligned. Global properties can use invariant pooling; object-level properties can use assignment-based matching before evaluation.

A probe that sorts slots by an oracle property before classification can accidentally leak the answer. The matching procedure belongs in the evidence chain.

BX. Temporal probes ask when state becomes accessible

For recurrent models, world models and autoregressive reasoning, evaluate the same target at successive time steps. A target can appear gradually, peak and then be compressed away after it has served its purpose.

Temporal accessibility is more informative than one final-state probe. Pair it with interventions to determine whether the moment of maximum decodability is also the moment of causal use.

BY. Future-state probes can reveal predictive information

Given current hidden state, train a probe to predict a future observation or latent variable. Strong performance means the state contains information predictive of the future under the data distribution.

Prediction does not establish causal sufficiency. The probe can exploit correlations and may learn dynamics itself. Restrict probe capacity and compare prediction horizons to distinguish locally accessible dynamics from heavy decoder computation.

BZ. Action probes test whether a world model exposes control-relevant state

In embodied or reinforcement-learning systems, probe the latent state for action, value, reward, object position and hidden environment variables. The goal can be to determine whether a controller could act from that representation.

An action probe can perform well by imitating the policy rather than reading a world-state variable. Probe state variables separately when the scientific claim concerns representation rather than policy cloning.

CA. Value probes can be confounded by policy and visitation distribution

A latent state may encode expected return only for states visited under the training policy. Probe generalisation to off-policy or counterfactual states tests whether value information is robust or merely fitted to familiar trajectories.

For a representation intended for planning, the stronger test is whether state supports value estimation after action or environment changes, not only under the original policy.

CB. Vision probes should separate category, pose, texture and location

A vision representation can make object category linearly accessible while progressively discarding exact location. Or it can preserve pose and texture in early layers and produce invariant category features later. Probe these variables together to see the information trade-off through depth.

Use counterbalanced datasets so category does not predict background, viewpoint or texture automatically. A high “object category” probe can otherwise rely on context.

CC. Spatial probes need coordinate-frame discipline

Predicting x/y image coordinates is easy if positional encodings are directly present. Predicting world coordinates from a camera representation asks a harder geometry problem. State the coordinate frame and which transformations should preserve or change the target.

For equivariant models, a fixed linear probe can fail after rotation even though the representation transforms correctly. Probe evaluation should respect the expected transformation law.

CD. Segmentation probes measure spatially local accessibility

Attach a simple per-patch or per-pixel decoder to frozen features. Strong segmentation with a shallow head suggests local semantic information is already aligned spatially.

A large upsampling decoder can reconstruct masks from coarse global information and learned priors. Probe complexity again determines how much structure is attributed to the representation.

CE. Depth and surface-normal probes test geometric information

Self-supervised vision encoders can be probed for depth, normals and correspondence. A linear or shallow convolutional head can reveal whether geometric structure emerges without direct supervision.

Evaluate on new scenes and camera configurations. A probe can exploit dataset-specific object-size priors rather than true geometric representation.

CF. Image-patch probes can reveal object-centric structure before explicit slots

Cluster or classify patch embeddings by object membership. If patches belonging to one object become easier to group through depth, the representation may be organising the scene compositionally even without a slot architecture.

This is descriptive evidence. Object permanence, identity and action-conditioned dynamics require additional temporal and causal tests.

CG. Audio probes should separate content, speaker and environment

Speech representations can expose phoneme, word, speaker, accent, emotion, microphone and room properties. Probe several simultaneously to understand which invariances the encoder learned.

A model intended for speech recognition may deliberately reduce speaker accessibility. A speaker-verification model should do the opposite. Representation quality is receiver-relative.

CH. Phoneme probes need temporal alignment

Audio frames and phoneme labels have different durations. Forced alignment or annotated boundaries map labels to frames. Alignment errors can cap probe accuracy and create apparent representation failures.

Report label timing uncertainty and compare pooled segment probes with frame-level probes.

CI. Speaker probes reveal retained identity and potential privacy leakage

If a general-purpose audio representation supports highly accurate speaker identification, it retains identity information. That can aid diarisation and create privacy concerns.

Probe privacy-sensitive properties under strict data governance. Decodability can indicate exposure risk even when the base task never uses the attribute.

CJ. Multimodal probes test shared semantics and modality-specific residue

For aligned image-text or audio-text models, probe both modalities for the same semantic labels. Compare whether one linear classifier transfers across modalities after alignment.

Shared probe transfer is stronger evidence of a common interface than separate probes trained independently. It still does not imply identical internal geometry or equal information content.

CK. Cross-modal retrieval and probing answer complementary questions

Retrieval asks whether paired examples are near in the learned space. A probe asks whether a chosen semantic variable is accessible. High retrieval can coexist with poor access to an attribute absent from the pairing objective.

Use both to distinguish general alignment from attribute-specific representation.

CL. Biological-sequence probes need controls for simple composition statistics

A DNA or protein representation can predict family, function or localisation partly from simple sequence composition. Compare against k-mer, amino-acid-frequency and length baselines before attributing success to deep biological structure.

Held-out homology groups provide a stronger generalisation test than random sequence splits that leak close relatives.

CM. Homology-aware splits protect against memorised sequence neighbourhoods

Random train-test splits can place nearly identical proteins or genomic segments on both sides. A probe can then behave like nearest-neighbour recognition.

Cluster by sequence identity and split by cluster when the claim is about transferable biological representation.

CN. Chemical probes should respect molecular symmetries

Probe molecular representations for energy, charge, functional groups, geometry or activity. For scalar properties, rotation should not change the target; for vector properties it should transform.

A probe can appear weak because its readout violates the symmetry type of the property. Match receiver geometry to the scientific variable.

CO. Sensor representations need unit and calibration controls

Probe time-series states for temperature, speed, anomaly type or regime. Raw sensor units and calibration offsets can leak labels directly.

Use cross-device and cross-site evaluation. A representation that makes “machine failure” accessible only because one failing machine uses a different sensor is not learning the intended mechanism.

CP. Uncertainty probes should separate aleatoric and epistemic targets where possible

Probe hidden states for prediction entropy, ensemble disagreement, data noise or evidence sufficiency. These variables can correlate while representing different uncertainty sources.

A model can be confident under ambiguous data or uncertain because it lacks knowledge. Label construction determines which kind of uncertainty the probe is learning.

CQ. Correctness probes can become post-hoc confidence models

Train a probe to predict whether the model’s answer will be correct from hidden state. Strong performance can support selective prediction or abstention.

The probe can overfit benchmark-specific error patterns. Test on new domains and model versions before deploying it as a confidence estimator.

CR. Hallucination probes need a precise factuality label

“Hallucination” can mean unsupported claim, contradiction with source, factual error, fabricated citation or answer outside evidence. A probe trained on one definition should not be advertised as a universal hallucination detector.

Use source-grounded labels and distinguish evidence insufficiency from reasoning error where the application requires it.

CS. Safety probes can reveal signals and can create false confidence

Internal states may correlate with harmful intent, refusal, policy categories or jailbreak-related behaviour. A probe can help analyse the system and is not a standalone safety guarantee.

Adversarial distribution shift, strategic inputs and low base rates make deployment harder than benchmark classification. Safety-critical monitors need calibration, redundancy and external controls.

CT. Privacy probes measure recoverable sensitive information

A probe can test whether representations expose identity, location, demographic attributes or membership. Strong decodability can identify privacy risks even if the base model does not intentionally use those variables.

Privacy probing should be performed on authorised datasets with minimised retention of sensitive labels and careful interpretation of what decodability implies.

CU. Fairness probes should not treat demographic categories as simple intrinsic features

Demographic labels can be socially constructed, context-dependent and noisy. A probe predicting them may expose correlations with environment, language or dataset source rather than a stable biological attribute.

Use probing to diagnose representation leakage and disparities, not to reify sensitive categories as one “true” hidden feature.

CV. Invariance probes test whether nuisance information remains decodable

If a representation is intended to be invariant to camera ID, speaker, colour or viewpoint, probe those variables. High decodability means the information survived even if the downstream head ignores it.

Low decodability under a weak probe is not proof of erasure. Use a capacity ladder and sensitivity controls before claiming invariance.

CW. Equivariance probes should predict transformation state rather than invariance

For a rotation-equivariant representation, orientation should remain decodable and transform predictably. A probe can test angle or transformation class across rotated inputs.

Failure of one fixed linear probe after rotation can indicate the probe does not respect representation type rather than failure of the base model’s equivariance.

CX. Disentanglement probes test factor accessibility and leakage

For each ground-truth factor, train a simple probe and measure which latent coordinates or subspaces predict it. Then train cross-factor probes to measure leakage.

This provides an operational view of disentanglement: factors are easy to read independently. It remains basis- and receiver-dependent and should be combined with intervention tests.

CY. SAE-feature probes can validate semantic dictionaries

Use sparse SAE coefficients as inputs to a target probe. If one or a small group of features predicts the concept strongly, the dictionary aligns with that concept.

Compare with a probe on the original dense activation. If dense state performs much better, the SAE lost or fragmented relevant information.

CZ. Probe sparsity over SAE features tests explanatory compression

An L1 or low-rank probe over sparse features can identify a compact semantic interface. The resulting selected features are hypotheses for causal testing.

Feature selection can be unstable under correlated dictionary atoms. Bootstrap the selected set and report recurring features rather than one deterministic list.

DA. Probe directions can serve as candidates for activation patching

Once a target is linearly accessible at one site, project clean–corrupt activation differences onto the probe direction. Patch only that component and measure the behavioural effect.

This tests whether the decodable variable mediates the behaviour rather than merely coexisting in the representation.

DB. Probe disagreement across modalities can reveal missing shared semantics

If an attribute is linearly accessible in image embeddings and not text embeddings, the shared multimodal space may preserve modality-specific detail. This can be desirable or a limitation depending on the receiver.

Use common labels and cross-modal transfer to distinguish shared from modality-specific accessibility.

DC. Probe families can be organised as an accessibility ladder

• Native coordinate: one neuron or feature.
• Sparse linear: small coordinate set.
• Dense linear: one affine subspace.
• Low-rank structured: compact multi-output subspace.
• Kernel or shallow nonlinear: simple curved boundary.
• Deep nonlinear: broad recoverability.
• Generative/structured decoder: rich reconstruction of target structure.

Report where the property becomes recoverable along this ladder. That profile is more informative than choosing one point and calling it “encoded”.

DD. The modality casebook evidence card

• Target type: categorical, continuous, ordinal, relational or structured.
• Row unit: token, span, patch, frame, object, sequence or state.
• Geometry: scalar, periodic, vector, set or graph.
• Receiver field of view: local versus contextual.
• Baselines: identity, metadata and domain-specific shortcuts.
• Split policy: subject/domain/homology/device independence.
• Capacity ladder: accessibility profile.
• Transfer: cross-domain or cross-modal reuse.
• Causal escalation: patch/intervention plan.

The casebook demonstrates why “probe accuracy” is not one universal representation metric. Each domain defines a different target geometry, leakage risk and receiver. The same methodological discipline survives across them: specify the property, constrain the reader, break shortcuts, measure sample efficiency, test shifts and escalate important conclusions to causal intervention.