AI 与机器学习explained.ai编辑精选
深度学习所需的矩阵微积分
The Matrix Calculus You Need for Deep Learning
2018 — 2025，它一次次回到首页
黑点是上首页（≥ 50 分）的那次投稿，空心点是没冲上去的投稿，点越大分数越高。
导读
这是 explained.ai 上的一篇矩阵微积分教程，由 Jeremy Howard 与 Terence Parr 编写，专门讲解深度学习所需的矩阵（标量/向量/矩阵对变量）求导知识：记号约定、标量对向量、向量对向量、链式法则以及逐元素运算的求导。目的是让读者能自己动手推导神经网络反向传播中的梯度，而不是把求导当作黑箱。
为什么经久不衰：它把通常被课程跳过的"矩阵级求导"讲得异常清晰，还给出逐元素运算的紧凑新记号，篇幅短、免费、可随时查，因此多年后仍是推导梯度时的首选参考。
HN 评论精选
清晰度超过经典参考手册，向量链式法则与逐元素运算两节尤其值得看。
这篇的清晰度让我印象深刻。以前需要引用矩阵求导定理时我通常链接《The Matrix Cookbook》，以后可能会改引这篇。我尤其喜欢向量链式法则那一节（讲得很清楚）和逐元素运算那一节（用新颖的记号把许多结果紧凑地呈现出来）。
olooney查看英文原文
I am very impressed with the clarity of presentation here. I usually link to The Matrix Cookbook[1] when I need to cite a reference for matrix calculus theorems but I might reference this instead in the future. I particularly like the section on the vector chain rule (which is very clear) and the section on element-wise operations (which uses novel notation to present many results in a compact form.) [1]: https://www.math.uwaterloo.ca/~hwolkowi/matrixcookbook.pdf
质疑受众定位：对理论岗位不够深入，对应用岗位又非必需。
我支持有更多更好的学习资源，但每次看到深度学习入门材料我都想问"这是给谁看的"。2016 年英伟达的演示中，一位研究生说"你十分钟三十行代码做的事，我干了一整个学期"。工具越来越容易用，不懂多少数学也能搭出能跑的模型。所以这资源：需要深入理解数学和理论的岗位不够用；只需要把东西跑起来的岗位又用不上。它是给那些已经在做深度学习、但还没需要或没去查数学概念的人看的。
madenine查看英文原文
I’m in favor of there being more and better resources to learn anything out there, but every time I see a deep learning 101 type material all I can think is “who is this for?”. In ~July 2016 I was at a presentation by NVidia at GW in DC. They showed off how easy it was to build out and train a model using some of their tooling (Digits maybe?). After the demo they opened it up for questions and a grad student ‘asked’ “You just did in 10 mins with 30 lines of code what I worked on for an entire semester”. That’s been the trajectory of the tools and increasing abstraction in this space. It’s just getting easier and easier to build models that work (which is great), and it gets easier and easier to do so without knowing more than an extremely high level overview of the math behind it all. So while this looks like a great resource - who’s it for? For jobs/problems that need you to have a thorough understanding of the math and theory behind the networks this isn’t going to cut it. For jobs/problems that need you to get something working math or not - this likely isn’t necessary to get started. So it’s for people that have been getting into DL but also haven’t bothered or needed to look up the math concepts?
推荐配合 3blue1brown 视频，并指出本文帮他理解了矩阵形式与 GPU 的关联。
如果喜欢讲座式讲解，我推荐 YouTube 上 3blue1brown 的内容，他讲得既清楚又好懂。我不是从那里学会人工神经网络的，那些概念我本来就知道，但我不了解应用其中的矩阵形式，所以这篇很好，让我明白为什么 GPU 适合做这件事。从数学角度讲，看得很舒服。
luk32查看英文原文
If someone likes more lecture style explanation I can recommend 3blue1brown's material on YouTube. He explained in a pretty good an accessible way imho. I didn't learn artificial neural network stuff from there. I knew those concepts but I didn't know the matrix formalism applied to it. So this was really nice to understand why GPUs are good for this. Math-wise it was really nice watch.
补充了另一份矩阵微积分讲义与经典教材作为延伸阅读。
对我来说这方面最好的指南绝对是 http://www.psi.toronto.edu/~andrew/papers/matrix_calculus_for_learning.pdf ，另外还有 Magnus 和 Abadir 的《Matrix algebra》。
Koncopd查看英文原文
Absolutely best guide (for me) on these things is http://www.psi.toronto.edu/~andrew/papers/matrix_calculus_for_learning.pdf Also "Matrix algebra" by Magnus and Abadir.
历次讨论
同类还值得读
如何写一个拼写纠正器
How to Write a Spelling Corrector
Peter Norvig 用一个几十行的 Python 程序演示如何做拼写纠正：先用编辑距离（删除、插入、替换、换位）生成所有可能的候选词，再用从大量文本中统计出的词频概率，选出…
AI 与机器学习上首页 10 次 · 2007–2022norvig.com编辑精选面向黑客的概率编程与贝叶斯方法
Probabilistic Programming and Bayesian Methods for Hackers (2013)
《黑客的贝叶斯方法与概率编程》是一本开源的贝叶斯统计入门书，以 Python/PyMC 代码和交互式 Jupyter Notebook 为主线，强调“先实践、后数学”的学习路径，…
AI 与机器学习上首页 5 次 · 2013–2018camdavidsonpilon.github.io编辑精选神经网络黑客指南
Hacker's Guide to Neural Networks
Andrej Karpathy 写的神经网络入门教程，面向程序员，从最简单的表达式求导讲起，逐步推导出反向传播和梯度下降，最后引出神经网络。
AI 与机器学习上首页 4 次 · 2014–2019karpathy.github.io编辑精选神经网络能逼近任意函数的可视化证明
A visual proof that neural nets can approximate any function
这一章来自 Michael Nielsen 的在线书《Neural Networks and Deep Learning》，用 sigmoid 单元叠加成局部隆起/平滑阶跃的图示…
AI 与机器学习上首页 4 次 · 2014–2022neuralnetworksanddeeplearning.com编辑精选论乔姆斯基与统计学习的两种文化
On Chomsky and the Two Cultures of Statistical Learning
Norvig 回应 Chomsky 对统计语言模型批评的文章，围绕“统计学习能否解释语言”展开，比较生成与解释、少参数模型与 Shannon 式高参数统计模型。
AI 与机器学习上首页 4 次 · 2011–2025norvig.com编辑精选