AI 与机器学习arxiv.org编辑精选
深度学习：一份批判性评估
Deep learning: a critical appraisal
2018 — 2019，它一次次回到首页
黑点是上首页（≥ 50 分）的那次投稿，空心点是没冲上去的投稿，点越大分数越高。
导读
这是 Gary Marcus 在 2018 年发表的一篇对深度学习的批判性评估：他指出深度学习存在数据饥渴、迁移能力有限、难以整合先验知识和推理等局限，认为仅靠深度学习不足以实现通用人工智能，需要与其他技术结合。文章在深度学习热潮中系统列出能力边界，因而多次引发讨论。
为什么经久不衰：它提供了一份清醒的深度学习局限清单，在 AI 热潮反复起落时仍可作为讨论能力边界与炒作风险的参照。
HN 评论精选
元学习等新方向已部分回应论文担忧，论文忽略了这些进展。
论文中的许多担忧，尤其是深度学习数据饥渴、迁移能力有限、难以整合先验知识，已被近期元学习和“学会学习”的论文在监督学习与强化学习场景中处理。元强化学习的近期工作似乎表明该机制与人脑学习方式很相似。我们组最近也发表了脉冲网络中的学会学习论文，显示网络能学习监督任务族的先验，并在元强化学习中自动学到有用的探索策略。我不声称这是通往 AGI 的正确道路，但这是深度学习中很有前景的新方向，论文似乎忽略了。
trextrex查看英文原文
Many concerns in this paper, especially about deep learning being data hungry, limited capacity for transfer, and integrating prior knowledge are addressed by recent papers in meta-learning and learning-to-learn [1,2,3 and many many others] both in the supervised and reinforcement learning contexts. In the case of meta-reinforcement learning there has been recent work [5] which seems to indicate that this mechanism is very similar to how learning works in the brain. In fact, my group recently published a paper [4] about learning-to-learn in spiking networks (making it biologically more realistic) and showing that the network learns priors on families of tasks for supervised learning, and learns useful exploration strategies automatically when doing meta-reinforcement learning. While I don't claim this is the right path to AGI, it's a very promising and new direction in deep learning research which this paper seems to ignore. [1] http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.5.323 [2] https://arxiv.org/abs/1611.05763 [3] https://arxiv.org/abs/1703.03400 [4] https://arxiv.org/abs/1803.09574 [5] https://www.nature.com/articles/s41593-018-0147-8 (Preprint: https://www.biorxiv.org/content/early/2018/04/06/295964)
担忧是合理的研究议程，深度学习社区已在用多种方法应对。
论文几乎所有担忧都是活跃研究主题，并且已有某些基于深度学习的方法在应对；你可以说这些是混合方案，但这只是解释角度问题。没人否认这些担忧有效，也没人认为当前深度学习的知识未来不会有用。例如数据饥渴：动物也接收大量输入，进化也已做了某种预训练；少样本、零样本和元学习是活跃领域。迁移能力有限：迁移学习、元学习、多任务学习正在处理。
albertzeyer查看英文原文
Almost all concerns in the paper are active research topics and do have certain solutions which do use some sort of deep learning approach. Depending on the viewpoint and interpretation, you could say that some of these approaches are hybrid solutions, but this is really just a matter of interpretation. No-one is really denying that the stated concerns are valid concerns. But also, no-one would say that the current knowledge gained from deep learning research will not be useful in the future. Of course, maybe for some aspects, you would need more radical new ideas, but I doubt that for future methods, nothing from the current methods will be used in some way. E.g.: 3.1. Deep learning thus far is data hungry. First, you could argue that on a low-level, an animal/human gets quite a lot of visual and audio input, so it's data hungry as well. Then, you could argue that the evolution did already do some sort of pretraining/pre-wiring which helps, using million of years of data. Then, related to this is the topic of unsupervised learning and reinforcement learning. Then, dealing with the aspect of learning with small amounts of data, there are the active research topics of one-shot-learning, zero-shot-learning of few-shot-learning. Related is also meta-learning. 3.2. Deep learning thus far is shallow and has limited capacity for transfer. Transfer-learning, meta-learning and multi-task-learning are active research areas which deal with this. 3.3. Deep learning thus far has no natural way to deal with hierarchical structure. There are various approaches also for this. This is also an active research area. 3.4. Deep learning thus far has struggled with open-ended inference. This is also an active research area. 3.5. Deep learning thus far is not sufficiently transparent. Also this is an active research area. And then, you could also argue that the biological brain also suffers at this. 3.6. Deep learning thus far has not been well integrated with prior knowledge. This is also an active research area. Etc.
质疑追求 AGI 的社会意义，担心“取代人类”的目标。
为什么目标是 AGI？从攀登珠峰的角度我理解，但从整个社会角度不理解。我们已经有 70 多亿个通用智能。目标是取代人类吗？作为人类，我不支持。
goatlover查看英文原文
Why is the goal AGI? I get it from a climb Mt Everest POV, but not for society as a whole. We already have 7.x billion general intelligences. Is the goal to replace humanity? Because being human, I’m not on board with that.
AGI 概念预设智能可脱离身体与情感，需要哲学反思。
我觉得 AGI 这个词很有误导性。它助长一种观念：人类心灵的奇妙、知识、感受、感觉等可以被量化并变成算法。我认为很多 AI 研究者需要补补科学哲学和不同知识形式（episteme、phronesis、techne）的基础，也许还有《科学革命的结构》。AI 的整个范式预设广义智能可以脱离生物学、感受、身体和感官而存在。这也是 AI 在 70 年代进入所谓“AI 寒冬”的原因之一，当时研究者把语言和人类知识简化为对符号的算法操作。
hestefisk查看英文原文
The term AGI I find quite misleading. It promotes the idea that the wonders of the human mind, knowledge, feelings, sensations etc can be quantified and turned into something algorithmic. I think a lot of AI researchers could do with a basic introduction to philosophy of science and the different forms of knowledge —- episteme, phronesis, techne — and perhaps also the structure of scientific revolutions (Thomas Kuhn, notably). The whole paradigm of AI presupposed that generalised intelligence can exist without biology, feelings, a body / senses. This is one of the reasons that AI reached the so-called “AI winter” in the 70ies where researchers boiled language and human knowledge down to algorithmic manipulation of symbols. EDIT: John Searle’s Chinese Room argument is a good (and fun) place to start —- https://en.wikipedia.org/wiki/Chinese_room
历次讨论
同类还值得读
如何写一个拼写纠正器
How to Write a Spelling Corrector
Peter Norvig 用一个几十行的 Python 程序演示如何做拼写纠正：先用编辑距离（删除、插入、替换、换位）生成所有可能的候选词，再用从大量文本中统计出的词频概率，选出…
AI 与机器学习上首页 10 次 · 2007–2022norvig.com编辑精选面向黑客的概率编程与贝叶斯方法
Probabilistic Programming and Bayesian Methods for Hackers (2013)
《黑客的贝叶斯方法与概率编程》是一本开源的贝叶斯统计入门书，以 Python/PyMC 代码和交互式 Jupyter Notebook 为主线，强调“先实践、后数学”的学习路径，…
AI 与机器学习上首页 5 次 · 2013–2018camdavidsonpilon.github.io编辑精选神经网络黑客指南
Hacker's Guide to Neural Networks
Andrej Karpathy 写的神经网络入门教程，面向程序员，从最简单的表达式求导讲起，逐步推导出反向传播和梯度下降，最后引出神经网络。
AI 与机器学习上首页 4 次 · 2014–2019karpathy.github.io编辑精选神经网络能逼近任意函数的可视化证明
A visual proof that neural nets can approximate any function
这一章来自 Michael Nielsen 的在线书《Neural Networks and Deep Learning》，用 sigmoid 单元叠加成局部隆起/平滑阶跃的图示…
AI 与机器学习上首页 4 次 · 2014–2022neuralnetworksanddeeplearning.com编辑精选论乔姆斯基与统计学习的两种文化
On Chomsky and the Two Cultures of Statistical Learning
Norvig 回应 Chomsky 对统计语言模型批评的文章，围绕“统计学习能否解释语言”展开，比较生成与解释、少参数模型与 Shannon 式高参数统计模型。
AI 与机器学习上首页 4 次 · 2011–2025norvig.com编辑精选