Interpretability

J-Lens Bias

Interpretability research using the anthropics/jacobian-lens repository on Qwen models. Covers BBQ bias analysis, activation-steering experiments, and negative random-direction controls that overturned an early result.

DomainInterpretability
RoleML Engineer
Timeline2026
Code AccessRepository
Why this project?

Validating whether activation-steering vectors reflect genuine mechanisms requires rigorous control baselines rather than selective confirmation.

System Objective

Demonstrated the necessity of random-direction negative controls in model steering evaluations.

End-to-end execution pipeline

01
Data Ingestion & Context

Document normalization, raw text extraction, and domain chunking

02
Embedding & Indexing

Dense vector representations via fastembed and FAISS vector indices

03
Retrieval & Grounding

Top-k semantic similarity matching with thresholded relevance verification

04
Inference & Response

Constrained LLM context injection with strict source citation formatting

Core technologies & frameworks

InterpretabilityJacobian LensQwenPyTorchPython
Impact & Validation

Open interpretability contribution shared with the Cohere Labs community emphasizing rigorous negative-control methodology.

Next System Iteration

Revealed that initial steering metrics were matched by random direction controls, establishing strict evaluation standards.

Key engineering learnings

01

Conducted layerwise Jacobian Lens analysis on open-weight Qwen models.

02

Ran systematic BBQ bias benchmark evaluations across demographic categories.

03

Designed null-hypothesis random direction controls that proved vital for valid interpretability claims.