Observe
Use the J-lens to find intermediate concepts, such as successive answers in a multi-step arithmetic problem.
Result · a hypothesis about what is represented, where
Interpretability · J-space and beyond
How researchers look inside an LLM.
J-space offers a window into concepts a model is silently holding. It is one part of a broader effort to find representations, trace computations, and test what actually causes behavior.
Current through July 2026 · primary research linked below
One token’s residual stream, layer by layer
EARLYparse input
syntaxtoken
MIDDLEworkspace
ItalyEuropelocation
MIDDLEreason
southernanswer
LATEprepare output
EuropeRome
J-space is not another layer. It is a sparse, readable part of the activation patterns already moving through the model.
01The internal workspace
J-space is a readable part of the residual stream
Every transformer layer reads and updates a running vector called the residual stream. Most of that activity handles automatic work: grammar, token identity, routine recall, and computational bookkeeping. The J-space is the small portion that can be expressed as a sparse combination of directions associated with words the model could later verbalize.
When a J-lens readout shows ERROR, it does not mean the model is currently saying—or even about to say—“error.” It means a representation associated with being able to talk or reason about an error is active.
prompt → transformer layers → next-token scores · green marks the middle “workspace” band
tokenswhere · is · italy
→
L1
L8
L24
L48
L72
L92
L96
→
logitsEurope · Rome · …
input-likepersistent abstract conceptsoutput-like
SMALLIt carries only a few dozen strongly active directions at once and less than 10% of total activation variance in the reported experiments.
FLEXIBLEInformation placed here can be reported, held in mind, or handed to different downstream computations.
SELECTIVEComplex reasoning depends on it more than fluent grammar, simple classification, or routine retrieval.
Technical precision: why it is not quite an ordinary linear subspace
There is one J-lens direction per vocabulary token, so the set is overcomplete and does not form a unique basis. The paper defines J-space using sparse, non-negative combinations of a limited number of these directions. Geometrically, that is closer to a union of small cones—or a sparse subframe—than one flat subspace.
02The reading tool
The Jacobian lens asks what an activation could make the model say
A normal logit lens treats an early-layer activation as though it already used the final layer’s coordinates. The J-lens corrects for how representations change through the remaining layers. Walk through the four moves.
Move 1 · choose a state
Take an intermediate activation
Pick the residual-stream vector at one token position and one layer. It contains many overlapping features—not a neat list of concepts.
hℓ · a dense activation vector
Move 2 · measure influence
Differentiate through future layers
The Jacobian measures how a tiny change here would propagate through the rest of the network and alter later residual states.
∂ future state / ∂ current state
Move 3 · find the stable map
Average across positions and prompts
A single prompt mixes general representational structure with context-specific effects. Averaging isolates the model’s broader disposition to verbalize a concept.
many local Jacobians → one map for the layer
prompt 001
prompt 002
prompt 003
⋮ many contexts
average Jℓ
Move 4 · translate to words
Map the effect into vocabulary scores
The normal unembedding matrix converts the transformed activation into ranked tokens—a human-readable approximation of what is “on the model’s mind.”
example J-lens readout
Europe#1
location#2
Italy#3
Rome#9
1 of 4
J-lens(hℓ) = softmax( Wᵤ · norm( Jℓ · hℓ ) )03Correlation → causation
Reading a concept is not enough—change it and test the result
A probe can find information that the model never actually uses. Strong interpretability research therefore combines a readable hypothesis with an intervention.
Replace
Swap one concept direction for another. If “soccer” becomes “rugby” and the report changes, the direction has causal leverage.
Result · evidence the representation can drive behavior
Ablate
Remove active J-space components while leaving most other activity intact, then measure which capabilities fail.
Result · evidence the workspace is necessary for that task
J-space is
- a sparse set of verbalizable internal directions;
- most workspace-like in a middle band of layers;
- causally important for some flexible reasoning;
- different at every layer.
J-space is not
- a new architectural module or memory bank;
- the model’s visible chain-of-thought;
- all of the model’s knowledge or computation;
- evidence by itself for consciousness.
04Current interpretability map
Seven ways researchers are opening the black box
These approaches answer different questions. A feature detector tells you what is present; a circuit tells you how it was used; an intervention tells you whether it mattered.
01 · Feature discovery
Sparse autoencoders
Expand a dense activation into a much larger dictionary where only a few learned features are active at once.
AsksWhich concepts are represented here?
ExampleA feature activates on scams, code errors, places, or refusal language.
LimitA readable feature may correlate with behavior without causing it.
02 · Computation tracing
Attribution graphs
Connect interpretable features into a graph of the information flow that produced one particular output.
AsksWhich intermediate steps led to this answer?
ExampleTrace a country feature into a capital feature and finally the token “Paris.”
LimitCurrent graphs are partial and often valid only for one prompt.
03 · Causal validation
Patching, steering and ablation
Change an internal state, rerun the model, and test whether the predicted behavior changes with it.
AsksDoes this representation actually do anything?
ExampleSuppress a refusal feature and see whether refusals decrease.
LimitLarge interventions can push the model into unnatural states.
04 · Internal cognition
Workspaces, personas and emotions
Study higher-level state variables that organize reasoning, character, self-monitoring, or decision-making.
AsksWhat global state is shaping the model right now?
ExampleJ-space, an Assistant Axis, and functional emotion representations.
LimitHuman labels are useful metaphors, not proof of human-like experience.
05 · Change detection
Model diffing
Align two models’ internal feature dictionaries and isolate features that exist only in one model or version.
AsksWhat new behavior appeared after this update?
ExampleSurface model-specific censorship, persona, or copyright-refusal mechanisms.
LimitA high-recall diff can still produce thousands of candidates to review.
06 · Interpretability by design
Weight-sparse models
Train networks with far fewer active connections so the learned algorithms form smaller, cleaner circuits.
AsksCan we build models that arrive untangled?
ExampleA tiny circuit remembers which quote character must close a Python string.
LimitResults are early and have not demonstrated frontier-scale completeness.
07 · Safety auditing
Hidden objectives and evaluation awareness
Combine internal monitors with behavioral evaluations to look for cognition that a model’s final answer may conceal.
AsksDoes the model recognize a test, notice a prompt injection, or pursue a hidden goal without saying so?
ExampleJ-lens readouts have surfaced internal markers related to “fake,” “injection,” “trick,” and planted misaligned objectives.
LimitNo current monitor is complete. Automatic or deliberately obfuscated cognition may bypass the feature being watched.
Alignment applications and limitations ↗Level 1 · representation
What is present?
J-lenses, probes and sparse features turn activations into candidate concepts.
Level 2 · mechanism
How is it computed?
Attribution graphs connect those concepts through attention, MLPs and layers.
Level 3 · causality
Does it control behavior?
Patching, steering and ablation test whether the proposed mechanism matters.
05What remains unsolved
Interpretability is a microscope, not source code
The newest methods reveal real structure, but none provides a complete, guaranteed-faithful explanation of a frontier model.
1
Vocabulary bottleneck. The current J-lens naturally names single-token concepts; multi-token or relational ideas can be split or missed.
2
Bag-of-concepts problem. Seeing
spider, eight, and legs does not explain how the model binds them into “a spider has eight legs.”3
Incomplete coverage. Early-layer J-lens readouts are noisy, automatic computations can bypass J-space, and attribution graphs omit some model activity.
4
Intervention artifacts. Removing or amplifying an activation can create a state the model never encountered during training.
5
Scale. A model performs billions of operations per generated token; turning local explanations into a global theory remains an open problem.