AI 与机器学习arxiv.org编辑精选
神经图灵机 [PDF]
Neural Turing Machines [pdf]
2014 — 2016，它一次次回到首页
黑点是上首页（≥ 50 分）的那次投稿，空心点是没冲上去的投稿，点越大分数越高。
导读
DeepMind 2014 年发表的论文《神经图灵机》(NTM)，提出将神经网络与可读写的外部记忆耦合，使网络能通过梯度下降学习复制、排序等简单算法任务，被视为可微分计算与记忆增强网络的开创性工作。
为什么经久不衰：它是记忆增强神经网络和后来可微分计算方向的奠基性论文，多次登上 HN 首页，至今仍是理解外部记忆机制如何与神经网络结合的经典入门材料。
HN 评论精选
NTM 预示了用训练代替编程的未来
很有意思的阅读。从网络学会复制和排序数据的方式来看，这预示着未来软件工程师可能不再编写程序，而是用正确的数据训练网络——那基本上会终结编程语言之争。
noiv查看英文原文
Interesting read. Extrapolating from how the network learned to copy and sort data it points a bit to a future with software engineers not coding programs to do the job but training networks with the right data instead. That would basically end the language war. :) I hope time to silicon is a single digit year.
论文公开意味着更先进的应用已在内部进行，可能冲击白领工作
如果他们公开了这篇论文，可以肯定他们已经在用这些技术解决更复杂的问题。这可能是未来白领工作被取代浪潮的又一个早期信号。
blackkettle查看英文原文
Wow, if they are publishing this, you can bet they are applying these techniques to much more sophisticated problems already. This might be another preliminary signal for a future wave of white-collar job destruction.
质疑论文的实验设置和训练效率：样本量太大，且未针对 LSTM 的强项做对比
我不太清楚这篇论文的意义所在。论文核心似乎是比较 LSTM 和 NTM，但他们没有让网络接触很长的序列或被任意长延迟打断的序列——而这正是 LSTM 擅长的。另外训练样本数量巨大：如果网络要 20 万个样本才能学会复制、排序这类规则，这真的算令人印象深刻吗？
iandanforth查看英文原文
I'm going to be stupid in public on the hope that someone will correct me. 1. I'm not clear on the point of this paper. There are a lot of buzzwords and an extremely diverse set of references. The heart of the paper seems to be a comparison between Long-Short-Term-Memory (LSTM) recurrent nets and their NTM nets. But they don't expose the network to very long sequences, or sequences broken by arbitrarily long delays which are what LSTM nets are particularly good at. They seem to make the jump from "LSTM nets are theoretically turing complete" to "LSTM nets are a good benchmark for any computational task." 2. The number of training examples seems huge For many of the tasks they trained over hundreds of thousands of sequences. This seems like very very slow learning. If I'm meant to interpret these results as a network learning a computational rule (copying, sorting etc) is it really that impressive if it takes 200k examples before it gets it right? (Not sarcasm, I really don't know.)
提问：神经网络的循环状态本身是否已经算一种存储
典型的神经网络难道没有任何存储吗？我见过的神经网络例子似乎都以某种循环方式工作，每个神经元的状态影响其他神经元，这难道不可能是存储吗？
macrael查看英文原文
Does a "typical" neural network not have any storage to speak of? When I've seen examples of neural networks working, it's seemed like they work in cycles in some way, with the states of each "neuron" affecting the state of others. Is that not potentially storage?
历次讨论
同类还值得读
如何写一个拼写纠正器
How to Write a Spelling Corrector
Peter Norvig 用一个几十行的 Python 程序演示如何做拼写纠正：先用编辑距离（删除、插入、替换、换位）生成所有可能的候选词，再用从大量文本中统计出的词频概率，选出…
AI 与机器学习上首页 10 次 · 2007–2022norvig.com编辑精选面向黑客的概率编程与贝叶斯方法
Probabilistic Programming and Bayesian Methods for Hackers (2013)
《黑客的贝叶斯方法与概率编程》是一本开源的贝叶斯统计入门书，以 Python/PyMC 代码和交互式 Jupyter Notebook 为主线，强调“先实践、后数学”的学习路径，…
AI 与机器学习上首页 5 次 · 2013–2018camdavidsonpilon.github.io编辑精选神经网络黑客指南
Hacker's Guide to Neural Networks
Andrej Karpathy 写的神经网络入门教程，面向程序员，从最简单的表达式求导讲起，逐步推导出反向传播和梯度下降，最后引出神经网络。
AI 与机器学习上首页 4 次 · 2014–2019karpathy.github.io编辑精选神经网络能逼近任意函数的可视化证明
A visual proof that neural nets can approximate any function
这一章来自 Michael Nielsen 的在线书《Neural Networks and Deep Learning》，用 sigmoid 单元叠加成局部隆起/平滑阶跃的图示…
AI 与机器学习上首页 4 次 · 2014–2022neuralnetworksanddeeplearning.com编辑精选论乔姆斯基与统计学习的两种文化
On Chomsky and the Two Cultures of Statistical Learning
Norvig 回应 Chomsky 对统计语言模型批评的文章，围绕“统计学习能否解释语言”展开，比较生成与解释、少参数模型与 Shannon 式高参数统计模型。
AI 与机器学习上首页 4 次 · 2011–2025norvig.com编辑精选