AI 与机器学习norvig.com编辑精选
论乔姆斯基与统计学习的两种文化
On Chomsky and the Two Cultures of Statistical Learning
2011 — 2025，它一次次回到首页
黑点是上首页（≥ 50 分）的那次投稿，空心点是没冲上去的投稿，点越大分数越高。
导读
这是 Norvig 回应 Chomsky 对统计语言模型批评的文章，围绕“统计学习能否解释语言”展开，比较生成与解释、少参数模型与 Shannon 式高参数统计模型。Norvig 认为 Chomsky 反对的是高维统计模型的可学习性，而非统计方法本身，并强调语音识别等解释任务本质上是概率问题。
为什么经久不衰：它清晰呈现统计学习与生成语法之争的核心分歧，在统计方法主导的今天仍值得回看。
HN 评论精选
统计学习不足以解释语言认知，能生成句子不等于理解语言系统。
文章开头乔姆斯基那段话比全文其余部分更清楚、更有思想；我觉得作者没抓住要点。就语言而言，观察并报告口语/书面输出中的统计概率，几乎不能解释语言习得和使用所涉及的认知系统。哪怕一个统计异常也能说明统计学习并非语言发展的全貌。乔姆斯基说过：骗人把潜艇误认成鲸鱼，不能说明潜艇真的会游，也不能否定这一事实。造出能生成数百万个合乎语法句子的计算机，几乎不能说明我们理解了语言系统；除非它能像人一样学会生成无限、新颖、有语境且有意义的合乎语法句子。
brockf查看英文原文
Chomsky's one paragraph quote at the beginning of this article is more clear and thoughtful than the rest of this. I feel the author's missing the point. In the case of language, observing and reporting statistical probabilities in written/spoken language output does very little to explain the cognitive systems used in acquiring and using language. Even one statistical anomaly serves to show that statistical learning is NOT the entire picture when it comes to language development. There was another article on HN a while back that had another great quote from Chomsky that does well to illustrate what I feel is his main point here: "Fooling people into mistaking a submarine for a whale doesn't show that submarines really swim; nor does it fail to establish the fact". Creating a computer that can produce millions of grammatical utterances does little to show that we understand language systems. Now, if a computer could - like humans - learn to produce infinite, novel, contextual, and meaningful grammatical utterances, that's a different story. But that story will take a lot more than statistical learning to write.
乔姆斯基的影响力可能阻碍统计 NLP，语言学训练也缺乏统计工具。
这不是新争论。语言学内部一直有反对统计 NLP 模型的声音，连 Manning 书的引言似乎都在为 NLP 辩护。乔姆斯基是巨人，但先驱有时会成为新观念的障碍；他对语义的强调扭曲了许多代研究者。我的导师 Raskin 是优秀的语义学研究者，却也认为统计方法不是方向。可悲的是，许多语言学系的人缺乏统计工具，无法基本理解 NLP 在做什么，所以 NLP 通常在 CS、EE 或 CompE 下教授。
Jun8查看英文原文
This is not a new debate. Within Linguistics there has been a continuous push against statistical NLP models. Read the introduction of Manning's book, even he seems to be defensive about NLP. Chomsky is a colossus, his achievements are well-known. However, at one point in many disciplines it comes to pass that the pioneers who pave the way in time become the very impediment to new ideas. His emphasis on Semantics have warped the minds of many generations of researchers (and some other ideas on universal grammar, too). I experienced this first hand, my advisor, Prof. Raskin, a great researcher on semantics, nevertheless thought that statistical approaches were not the way to go. Sadly, in many Linguistics departments people are just not equipped with the statistical tools necessary to have a basic understand of what's being done in the NLP field. So NLP is generally taught under CS, EE, or CompE.
语言交流依赖对歧义的相互预期，解释端本质上是概率问题。
Norvig 说：如果乔姆斯基关注另一端——解释，像 Shannon 那样，他也许会改变看法。在解释（如语音识别）中，听者收到嘈杂、有歧义的信号，需要判断许多可能意图信息中哪个最可能，因此这显然是概率问题，语音识别研究者早就认识到。这是关键，尤其因为说话者知道听者在做解释，并会不断利用结构歧义和语义歧义；听者又知道说话者知道这一点，形成无限层次的相互知晓。多数技术人士和商人直觉上明白，许多学术人却不明白。
christianpbrink查看英文原文
"If Chomsky had focused on the other side, interpretation, as Claude Shannon did, he may have changed his tune. In interpretation (such as speech recognition) the listener receives a noisy, ambiguous signal and needs to decide which of many possible intended messages is most likely. Thus, it is obvious that this is inherently a probabilistic problem, as was recognized early on by all researchers in speech recognition..." This is the money shot especially since speakers are aware of the interpretive activity of listeners, and effective speakers play constantly on the ambiguities in their statements - structural (i.e. grammatical) ambiguities as well as semantic ambiguities. Listeners in turn are aware of speakers' awareness of this.. There is, effectively, an infinity of mutual awarenesses of structural ambiguities. In any instance of communication. I think most technologists and (especially) businesspeople see this intuitively. I think many academics do not. Not sure how to articulate what I mean but I think I am saying something non-trivial about academics and their perspective on language.
数据模型可能漏掉生成规则，语言理解不能靠平滑弥补。
考虑最高法院握手问题，但假设有一条不成文规则：法官只能向资历比自己浅的法官主动握手；若两位法官资历完全相同，他们根本不握手。假设历史上从未有两位资历完全相同的法官，那么从握手数据学到的模型不会包含这条不成文规则的丝毫痕迹。乔姆斯基的意思大概是：如果不理解数据背后的生成原则，就不可能知道什么情况会完全推翻模型，也许没有平滑的办法。语言理解不同于语音识别，并不很适合平滑。
fauigerzigerk查看英文原文
Let's consider the supreme court handshake problem, but say there is an unwritten social law that forces judges to only ever initiate a handshake with judges less senior than themselves. If the seniority of two particular judges happened to be exactly equal, they would not shake hands at all. Let's assume (I don't know if it is actually true or not) that in the history of the supreme court, there have never been two judges of the exact same seniority. In that case, a model learned from handshake data would not include the slightest hint of this unwritten social law. I think what Chomsky is saying is that if we do not understand the generative principle behind any data, we cannot possibly know what circumstance might completely invalidate our model. There may not be a way to smooth this out. Language understanding, contrary to things like speech recognition, does not lend itself very well to smoothing.
历次讨论
同类还值得读
如何写一个拼写纠正器
How to Write a Spelling Corrector
Peter Norvig 用一个几十行的 Python 程序演示如何做拼写纠正：先用编辑距离（删除、插入、替换、换位）生成所有可能的候选词，再用从大量文本中统计出的词频概率，选出…
AI 与机器学习上首页 10 次 · 2007–2022norvig.com编辑精选面向黑客的概率编程与贝叶斯方法
Probabilistic Programming and Bayesian Methods for Hackers (2013)
《黑客的贝叶斯方法与概率编程》是一本开源的贝叶斯统计入门书，以 Python/PyMC 代码和交互式 Jupyter Notebook 为主线，强调“先实践、后数学”的学习路径，…
AI 与机器学习上首页 5 次 · 2013–2018camdavidsonpilon.github.io编辑精选神经网络黑客指南
Hacker's Guide to Neural Networks
Andrej Karpathy 写的神经网络入门教程，面向程序员，从最简单的表达式求导讲起，逐步推导出反向传播和梯度下降，最后引出神经网络。
AI 与机器学习上首页 4 次 · 2014–2019karpathy.github.io编辑精选神经网络能逼近任意函数的可视化证明
A visual proof that neural nets can approximate any function
这一章来自 Michael Nielsen 的在线书《Neural Networks and Deep Learning》，用 sigmoid 单元叠加成局部隆起/平滑阶跃的图示…
AI 与机器学习上首页 4 次 · 2014–2022neuralnetworksanddeeplearning.com编辑精选语音与语言处理（第三版草稿，2017）
Speech and Language Processing, 3rd ed. draft (2017)
Jurafsky 与 Martin 经典教材《Speech and Language Processing》第三版草稿的官方页面，按章提供 PDF，覆盖语音识别、自然语言处理与语…
AI 与机器学习上首页 4 次 · 2018–2025web.stanford.edu编辑精选