alphaXiv 发布可解释性领域图解指南

摘要

alphaXiv 发布了一份可解释性图解指南，从早期计算机视觉研究讲到语言模型，梳理特征可视化、SHAP 与积分梯度、电路追踪，以及注意力头、残差流、激活引导、消融与探针实验、多义性与 SAE 等方法。指南还讨论了该领域的局限与开放问题，并回应了近期走红的文章 Neural Networks Are Already Clean 引发的争论：我们是否真在接近逆向工程 LLM。

摘要由模型根据原文撰写，以原文为准。

The ultimate guide to interpretability is here 🔥

Are we actually on the verge of reverse engineering LLMs or is the excitement outpacing what interpretability methods can actually deliver?

We put together an illustrated walkthrough of the field, from early vision research to the latest debates about understanding LLMs.

The recent viral essay Neural Networks Are Already Clean has ignited long essays and online debates. In this guide, we build the foundation from the ground up starting with computer vision:

Feature visualization
SHAP and integrated gradients
Circuit tracing

Then we move to language models where we go over:

Attention heads, the residual stream, induction heads
Activation steering: amplifying a Golden Gate Bridge feature to mitigating evaluation awareness
Ablation, activation patching, and probe experiments
Polysemanticity, superposition, and SAEs

We also discuss the shortcomings and open questions of the field that can be easy to miss if you’re only following the latest interpretability breakthroughs from Anthropic blogs.

Read the full guide at https://www.alphaxiv.org/trends/interpretability

alphaXiv 发布可解释性领域图解指南

摘要

正文 · 原文

引用原文