How to read this

Status labels

Every substantive page carries one of three labels. They are not decorative.

Method — a description of how something is done. Checkable by reading; does not assert anything about the world.

Observation — something measured, with the conditions under which it was measured. An observation is not yet a claim.

Withdrawn — was stated, no longer is. The page records what was originally said, what failed, and what is now unknown. Withdrawn pages are not deleted, because the deletion of an inconvenient record is itself a finding about the project that produced it.

Terms

Condition — the framing under which a task is presented. Two are used throughout: self-directed and externally framed.

Pair — one item presented under both conditions. The unit of analysis.

Construction — the generated body of material for a run. Varying it answers “would this hold with different items?”

Policy sampling — the model’s stochastic choice among continuations. Varying it answers “would this hold on a rerun?”

Execution — the serving stack and how it happened to run the model on a given occasion. Varying it answers “would this hold tomorrow, on the same machine, after a restart?”

Resolution floor — the smallest effect an estimator can distinguish from zero given the size of the design. An effect below the floor is reported as unresolved, never as absent. This distinction is the single most common failure in behavioural evaluation of language models: a design too small to see something reports a clean zero and reads like evidence of no effect.

Standard of evidence

A claim is publishable here when all four hold:

1. The decision rule was written before the data was seen.
2. The effect exceeds the resolution floor of its own estimate.
3. An independent run, on material not used to design the analysis, reproduces it.
4. Someone tried to refute it and failed.

The fourth condition is a real step, not a figure of speech. Findings are handed to a reviewer whose task is to break them, and who is expected to do so by rerunning the analysis rather than by reasoning about it.

What this documentation is not

It is not a tutorial, and it is not a claim that the methods here are correct. It is a record precise enough that a disagreeing reader can find the specific decision they disagree with.