AI 与机器学习answer.ai
在家微调 70B 语言模型
Fine tune a 70B language model at home
2024 — 2025，它一次次回到首页
黑点是上首页（≥ 50 分）的那次投稿，空心点是没冲上去的投稿，点越大分数越高。
导读
Answer.AI 介绍 FSDP + QLoRA 技术，让个人在两张消费级 GPU 上微调 70B 大语言模型成为可能。文章讲解了该方法的实现细节与开源代码，大幅降低了大模型微调的硬件门槛。
为什么经久不衰：它把原本需要数据中心级硬件的 70B 模型微调带到家用设备上，是降低 LLM 开发门槛的经典实践，两年内两次登上 HN 首页。
HN 评论精选
希望文章补充成本、耗时与质量损失的具体数据，并肯定其赋能个人开发者的价值
我喜欢文中链接的每小时 0.6 美元租用双 24GB GPU 的方案，但用这套系统（4bit 权重）微调一个 70B 模型要花多久？作为消费者，我更关心最终成本，比如一张包含模型大小、训练数据量、训练成本和质量损失的表格。另外能否把这项技术应用到最近的 (-1,0,1) 编码模型上？我觉得这类工作非常有用，能让个人用户在 LLM 应用市场中有竞争力，这是在解决没人愿意解决的问题。
llmzero查看英文原文
I liked that you link to renting a dual 24GPU for 0.60cents/hour, but how long could it takes to fine tune a 70b model using your system (4 bits for weights)? If I were a consumer I would be interested in the final price of fine tuning, for example a table with model size, training size, cost of training, and expected loss of quality with this technology. One obvious question: Can you apply your technology with the recent (-1,0,1) encoding?, I think you will answers that the (-1,0,1) model is not available and you can't try it, but my question is whether once/if that model is available answer.ai will be able to use the same technology that this post to fine tune a big model in two very small GPUs, and then I should ask for a new table with cost/benefits analysis. Edited: I should add that I find this kind of work very useful for enhancing individual users like me to be able to compete in the applications of LLM market, this is great work and along the lines of the book "from zero to one" (not that I like or dislike the author) to solve the kind of problem that nobody is trying to solve. Edited: Now that I have a total of 23 points in HN, I will change my password to some random one, just to cure my desire to look for votes, and try to make some work, and again some tomorrow create a new presence in HN.
指出标题有误导性：这是微调而非从零训练
可能是我理解有误，但他们用的似乎是 LoRA，这是一种微调实现方式，需要一个已经训练好的 LLM。如果是这样，标题就不准确了——这并不能让你用两张消费级 GPU 从零训练一个模型。
doctoboggan查看英文原文
Maybe I misunderstand, but it seems like they are using LoRa, which is a fine tuning implementation. That requires an already existing trained LLM. If that's true, I think that the title of this submission is inaccurate, as this doesn't let you train a model from scratch with 2 consumer GPUs.
质疑实际效果，指出微调效果有限且小显存下上下文长度受限
希望能看到一些基准测试。根据我的经验，要得到显著效果需要更多算力。微调大多只在基模型已接近你的目标时才有效，否则结果不会太令人满意。另外在显存较小的 GPU 上，上下文长度也会成为问题。
darkbatman查看英文原文
Would be nice to see some benchmarks. Also from my experience you need more power to get some significant result. Mostly fine tuning would work if base model is very close to what you are trying to achieve and you won't be much happy with the results though. Also context length becomes an issue trying to fit in with gpu with lesser ram.
设想用分布式众包 GPU 训练大型开源模型的可行性
如果能做一个类似 folding@home 或 SETI@home 的“LLM@home”项目就太酷了，让大量用户捐出自己的 GPU 来训练一个巨大的开源模型。我不太了解这些模型的训练方式，能否把训练拆分分布到各处，再合并回来？
itsgrimetime查看英文原文
Would be cool to build an “LLM@home” project like folding@home or SETI@home (rip), where tons of folks could donate their GPUs and train something huge and FOSS. I don’t know enough about how these models are trained though. Could it be chunked up and distributed in that way, then stitched/merged back together?
历次讨论
同类还值得读
如何写一个拼写纠正器
How to Write a Spelling Corrector
Peter Norvig 用一个几十行的 Python 程序演示如何做拼写纠正：先用编辑距离（删除、插入、替换、换位）生成所有可能的候选词，再用从大量文本中统计出的词频概率，选出…
AI 与机器学习上首页 10 次 · 2007–2022norvig.com编辑精选面向黑客的概率编程与贝叶斯方法
Probabilistic Programming and Bayesian Methods for Hackers (2013)
《黑客的贝叶斯方法与概率编程》是一本开源的贝叶斯统计入门书，以 Python/PyMC 代码和交互式 Jupyter Notebook 为主线，强调“先实践、后数学”的学习路径，…
AI 与机器学习上首页 5 次 · 2013–2018camdavidsonpilon.github.io编辑精选神经网络黑客指南
Hacker's Guide to Neural Networks
Andrej Karpathy 写的神经网络入门教程，面向程序员，从最简单的表达式求导讲起，逐步推导出反向传播和梯度下降，最后引出神经网络。
AI 与机器学习上首页 4 次 · 2014–2019karpathy.github.io编辑精选神经网络能逼近任意函数的可视化证明
A visual proof that neural nets can approximate any function
这一章来自 Michael Nielsen 的在线书《Neural Networks and Deep Learning》，用 sigmoid 单元叠加成局部隆起/平滑阶跃的图示…
AI 与机器学习上首页 4 次 · 2014–2022neuralnetworksanddeeplearning.com编辑精选论乔姆斯基与统计学习的两种文化
On Chomsky and the Two Cultures of Statistical Learning
Norvig 回应 Chomsky 对统计语言模型批评的文章，围绕“统计学习能否解释语言”展开，比较生成与解释、少参数模型与 Shannon 式高参数统计模型。
AI 与机器学习上首页 4 次 · 2011–2025norvig.com编辑精选