AI 与机器学习tradientblog.com
构建机器学习交易系统的经验教训
Lessons learned building an ML trading system
2019 — 2022，它一次次回到首页
黑点是上首页（≥ 50 分）的那次投稿，空心点是没冲上去的投稿，点越大分数越高。
导读
作者复盘自己构建机器学习交易系统、把5000美元做到20万美元的经历，重点不是模型多复杂，而是数据、回测、滑点、执行延迟和风险控制这些容易被模型光环掩盖的细节。评论补充说，这套系统交易的是加密货币。
为什么经久不衰：它经久不衰，是因为用真实项目说明：交易系统里领域知识、数据工程和执行成本往往比模型本身更决定成败，今天读仍有警示价值。
HN 评论精选
高频交易实战中，最难的是价值锚、订单簿细节、API限制和低延迟工程，而非模型本身。
我在2017/2018年花了大量时间给币安做高频交易机器人，可以说这篇文章相当扎实。我们做BTC/ETH/USDT三角套利，买卖延迟压到3-7毫秒，一度占到币安日成交量的0.3%-0.7%。几点体会：所有币都在浮动时，找到客观的价值锚很难但至关重要，这也是我们最大的难题；订单簿看似简单，细节却很要命，纸面交易尤其如此；高效利用交易所API限额本身就是优化问题；我们的模型比较简单，重点放在速度和边界情况，比如轮询SSL连接选最快、直接读原始数据流而不解码。几个月后，一个关于“机构访问”的神秘电话之后，整个项目崩了：那项访问很贵，我们拒绝了，随后所有策略都变差。
nickreese查看英文原文
After having spent an insane amount of time in late 2017/2018 building an HFT bot for Binance I can say this is a pretty solid article. In our case we were doing triangle trading between BTC/ETH/USDT pairs and had our buys/sell delay down to 3-7ms. At one point moving 0.3-0.7% of Binance’s daily volume. Few notes: * Finding an objective point of truth for value when all of the currencies are floating is hard but vital to success. This was the hardest problem we encountered. We tried taking the realtime average of BTC and ETH across all exchanges, we tried tying it to the shortest route to USD, and several other routes... but ultimately this is where we ended up “losing” most of our alpha. * Order books are seemingly simple but the devil is in the details. This especially matters for paper trading. * Efficiently using API limits at exchanges is an optimization problem in and of itself. * Our model was relatively simple but we focused on speed and edge cases. For instance Binance would rotate IPs on their load balancers and we’d constantly check the latency between each open SSL connection and use the fastest. Further we wouldn’t decode the buy response to plaintext we’d just read the raw stream. After several epic months our entire project fell apart after a cryptic phone call about “institutional access” that didn’t follow the 1s websocket update. The access was quiet expensive and we said no to it and shortly after all of our strategies went to crap. Best we could tell someone was front running us due to an artificial delay for our account (delay between trades went to ~20ms up from our prior steady speed of 3-7ms) and/or a bunch of the trades in the orderbook were bogus. Frustrated we tried our strategy on another account and the delay dropped again to our normal range and was profitable again (the orderbooks were slightly different between bots!). It was in that moment we realized playing in unregulated markets is not fun or something we wanted to continue to do. Intermediary risk was something we didn’t account for. Further we realized that there will always been a better resourced or more dedicated team willing to fight you for your alpha. After months of effort and a ton of fun we decided it was best we went back and focused on a problem where we could build a long term competitive advantage. Edit: typos and formatting
实盘系统的大部分工作在处理数据和成本建模，实盘成交能否贴近回测比短期盈利更关键。
我过去一年全职做了一个类似系统，在币安交易前20名左右的加密货币，用深度学习模型（时间/因果卷积网络加RNN）并做大量数据增强。数据采集、训练、回测和实盘部署工具都是自己搭的。数据工程背景帮助很大：大部分时间都在处理数据，而不是调模型。最困难的部分之一是估计点差/滑点成本并把它放进损失函数。作者讲的大部分坑，我都是硬着头皮踩过来的。现在小资金实盘测试结果为正：能赚小钱，更重要的是实盘成交和回测很接近。我正在扩大模型并补监控/报表/CI。
pinouchon查看英文原文
I spent the last year working fulltime on a system similar to the one described here. I trade the top ~20 cryptos on binance. I use deep learning models (combination of temporal, causal convnets and RNNs) with heavy data augmentation. I built my own tooling for data collection, training, backtesting and live deployment. Having a data engineer background coming into this was hugely helpful: most of my time was spent manipulating data in some way (and not playing around with the models). One of the most demanding parts was estimating spread/slippage costs and including it into the loss function. Most of what the author talked about, I learned the hard way. I'm now at the point where I ran some tests (trading small amounts) live on binance and the results are positive: I do manage to make small profits, but more importantly, the recorded live trades reflect very closely the backtest trades (for a given period). I'm currently scaling up my model and adding better monitoring / reporting / CI. I'd be happy to chat with anyone having done similar projects or willing to exchange ideas.
ML项目成败很大程度取决于领域知识，而不只是模型技术。
这篇文章很好地展示了ML领域里领域知识和ML能力的关系。文章大部分篇幅在讲交易的门道，作者显然对市场有较广的了解。相比之下，一些ML爱好者闯进医学或假新闻领域却做出没用的结果，正说明缺乏领域能力。
thundergolfer查看英文原文
This post is an exemplar of the crucial relationship between domain-specific knowledge and ML competency in the ML space. The bulk of the post is detailing the tricky ins and outs of trading, and overall the author gives the impression that they're broadly knowledgeable about stock markets. Contrast this post with those you see with ML hobbyists who delve into medicine or fake-news and produce useless results testament to their lack of domain-specific competency.
对这类策略要保持怀疑：回测有效不等于能扛住黑天鹅和交易所风险。
这很像一篇会被后续文章打脸的文章，下一篇标题可能是《在闪崩中亏掉20万美元的教训》。这类系统和模型往往在失效之前都有效。通常存在没有被正确对冲的隐藏风险——模型没预测到的黑天鹅，或者交易所被黑后资金全丢。
once_inc查看英文原文
This feels very much like a post that will be followed by a future post titled: "Lessons learned losing 200k in a flash crash". These sorts of systems and models often work until they don't. Often, there is a hidden risk that isn't hedged correctly - a black swan event not predicted by the model, or even just an exchange losing funds after a hack.
历次讨论
同类还值得读
如何写一个拼写纠正器
How to Write a Spelling Corrector
Peter Norvig 用一个几十行的 Python 程序演示如何做拼写纠正：先用编辑距离（删除、插入、替换、换位）生成所有可能的候选词，再用从大量文本中统计出的词频概率，选出…
AI 与机器学习上首页 10 次 · 2007–2022norvig.com编辑精选面向黑客的概率编程与贝叶斯方法
Probabilistic Programming and Bayesian Methods for Hackers (2013)
《黑客的贝叶斯方法与概率编程》是一本开源的贝叶斯统计入门书，以 Python/PyMC 代码和交互式 Jupyter Notebook 为主线，强调“先实践、后数学”的学习路径，…
AI 与机器学习上首页 5 次 · 2013–2018camdavidsonpilon.github.io编辑精选神经网络黑客指南
Hacker's Guide to Neural Networks
Andrej Karpathy 写的神经网络入门教程，面向程序员，从最简单的表达式求导讲起，逐步推导出反向传播和梯度下降，最后引出神经网络。
AI 与机器学习上首页 4 次 · 2014–2019karpathy.github.io编辑精选神经网络能逼近任意函数的可视化证明
A visual proof that neural nets can approximate any function
这一章来自 Michael Nielsen 的在线书《Neural Networks and Deep Learning》，用 sigmoid 单元叠加成局部隆起/平滑阶跃的图示…
AI 与机器学习上首页 4 次 · 2014–2022neuralnetworksanddeeplearning.com编辑精选论乔姆斯基与统计学习的两种文化
On Chomsky and the Two Cultures of Statistical Learning
Norvig 回应 Chomsky 对统计语言模型批评的文章，围绕“统计学习能否解释语言”展开，比较生成与解释、少参数模型与 Shannon 式高参数统计模型。
AI 与机器学习上首页 4 次 · 2011–2025norvig.com编辑精选