AI 与机器学习jalammar.github.io编辑精选
图解 Transformer
The Illustrated Transformer
2018 — 2025，它一次次回到首页
黑点是上首页（≥ 50 分）的那次投稿，空心点是没冲上去的投稿，点越大分数越高。
导读
这是 Jay Alammar 的经典图解教程，用直观的可视化逐层拆解 Transformer 架构，讲解自注意力机制、多头注意力和编码过程。它自 2018 年发布以来一直是学习 Transformer 最受欢迎的入门读物之一。
为什么经久不衰：它把抽象的注意力机制变成了看得见的图示，多年后仍是理解 LLM 底层原理最清晰的起点。
HN 评论精选
图解对入门很有帮助，但仅凭架构知识无法预测或解释现代 LLM 的涌现能力，应保持谦逊
我当年学 Transformer 基础时读过这篇文章，可视化非常有帮助。不过回头看，了解 Transformer 的工作原理对我在日常工作中应用 LLM 其实没什么直接用处，只是作为一种背景知识，让我对那个产出 token 的黑盒有个大致概念，并理解上下文长度限制等问题的数学基础。我要强烈提醒那些以为仔细研究架构就能更好理解或解释 LLM 行为的人：这是个陷阱。如今的大型前沿模型表现出大量非平凡涌现现象（部分归功于大规模强化学习），其能力远超架构刚问世时几乎所有人的预期。我们很多人在 2023 年还基于架构和训练算法自信地断言 LLM 永远做不好新颖的编程或数学任务，结果我们错了。这提醒我们对仅凭网络架构去推断 LLM 的能力和行为要保持谨慎与谦逊——你真得能去戳一个大模型内部的权重才能回答这类问题，而这基本只有大实验室的可解释性研究员才做得到。不管怎样，这是篇好文章。
libraryofbabel查看英文原文
I read this article back when I was learning the basics of transformers; the visualizations were really helpful. Although in retrospect knowing how a transformer works wasn't very useful at all in my day job applying LLMs, except as a sort of deep background for reassurance that I had some idea of how the big black box producing the tokens was put together, and to give me the mathematical basis for things like context size limitations etc. I would strongly caution anyone who thinks that they will be able to understand or explain LLM behavior better by studying the architecture closely. That is a trap. Big SotA models these days exhibit so much nontrivial emergent phenomena (in part due to the massive application of reinforcement learning techniques) that give them capabilities very few people expected to ever see when this architecture first arrived. Most of us confidently claimed even back in 2023 that, based on LLM architecture and training algorithms, LLMs would never be able to perform well on novel coding or mathematics tasks. We were wrong. That points towards some caution and humility about using network architecture alone to reason about how LLMs work and what they can do. You'd really need to be able to poke at the weights inside a big SotA model to even begin to answer those kinds of questions, but unfortunately that's only really possible if you're a "mechanistic interpretability" researcher at one of the major labs. Regardless, this is a nice article, and this stuff is worth learning because it's interesting for its own sake! Right now I'm actually spending some vacation time implementing a transformer in PyTorch just to refresh my memory of it all. It's a lot of fun! If anyone else wants to get started with that I would highly recommend Sebastian Raschka's book and youtube videos as way into the subject: https://github.com/rasbt/LLMs-from-scratch . Has anyone read TFA author Jay Alammar's book (published Oct 2024) and would they recommend it for a more up-to-date picture?
推荐搭配 Transformer Explainer 和 3blue1brown 的视频一起学习效果更好
也要感谢 Transformer Explainer 团队做出的精彩可视化（poloclub.github.io/transformer-explainer）。读完这两篇再看 3blue1brown 的视频，我才真正豁然开朗。
boltzmann_查看英文原文
Kudos also to Transformer Explainer team for putting some amazing visualizations https://poloclub.github.io/transformer-explainer/ It really clicked to me after reading this two and watching 3blue1brown videos
吐槽 Transformer 教程泛滥，很多作者自己也是一知半解
（跑个题。）Transformer 的讲解和教程数量已经多到让人应接不暇了。这让我想起 monads（或者微积分）：有人在某个瞬间感到顿悟（而实际上往往仍然一头雾水），然后就忍不住要把他们新获得的（错误）理解分享给广大读者。
Koshkin查看英文原文
(Going on a tangent.) The number of transformer explanations/tutorials is becoming overwhelming. Reminds me of monads (or maybe calculus). Someone feels a spark of enlightenment at some point (while, often, in fact, remaining deeply confused), and an urge to share their newly acquired (mis)understanding with a wide audience.
作者本人推荐了其他学习 Transformer 的优质资源
这是作者本人（jayalammar）的评论，谈了其他学习 Transformer 的好资源：https://news.ycombinator.com/item?id=35990118
laser9查看英文原文
Here's the comment from the author himself (jayalammar) talking about other good resources on learning Transformers: https://news.ycombinator.com/item?id=35990118
历次讨论
同类还值得读
如何写一个拼写纠正器
How to Write a Spelling Corrector
Peter Norvig 用一个几十行的 Python 程序演示如何做拼写纠正：先用编辑距离（删除、插入、替换、换位）生成所有可能的候选词，再用从大量文本中统计出的词频概率，选出…
AI 与机器学习上首页 10 次 · 2007–2022norvig.com编辑精选面向黑客的概率编程与贝叶斯方法
Probabilistic Programming and Bayesian Methods for Hackers (2013)
《黑客的贝叶斯方法与概率编程》是一本开源的贝叶斯统计入门书，以 Python/PyMC 代码和交互式 Jupyter Notebook 为主线，强调“先实践、后数学”的学习路径，…
AI 与机器学习上首页 5 次 · 2013–2018camdavidsonpilon.github.io编辑精选神经网络黑客指南
Hacker's Guide to Neural Networks
Andrej Karpathy 写的神经网络入门教程，面向程序员，从最简单的表达式求导讲起，逐步推导出反向传播和梯度下降，最后引出神经网络。
AI 与机器学习上首页 4 次 · 2014–2019karpathy.github.io编辑精选神经网络能逼近任意函数的可视化证明
A visual proof that neural nets can approximate any function
这一章来自 Michael Nielsen 的在线书《Neural Networks and Deep Learning》，用 sigmoid 单元叠加成局部隆起/平滑阶跃的图示…
AI 与机器学习上首页 4 次 · 2014–2022neuralnetworksanddeeplearning.com编辑精选论乔姆斯基与统计学习的两种文化
On Chomsky and the Two Cultures of Statistical Learning
Norvig 回应 Chomsky 对统计语言模型批评的文章，围绕“统计学习能否解释语言”展开，比较生成与解释、少参数模型与 Shannon 式高参数统计模型。
AI 与机器学习上首页 4 次 · 2011–2025norvig.com编辑精选