工具与资源github.com
Memray:Python 内存分析器
Memray: a memory profiler for Python
2022 — 2024,它一次次回到首页
黑点是上首页(≥ 50 分)的那次投稿,空心点是没冲上去的投稿,点越大分数越高。
导读
Memray 是彭博开源的 Python 内存分析器,可追踪内存分配、定位峰值内存并生成火焰图等报告。项目在 GitHub 上提供原生调用栈、子进程支持等特性,是 Python 性能调优的常用工具。
为什么经久不衰:它填补了 Python 峰值内存分析的工具空白,且由大厂持续维护,对排查批处理任务内存瓶颈至今非常实用。
HN 评论精选
峰值内存分析对批处理任务至关重要,Memray 在多方面优于同类工具 Fil
很高兴看到更多 Python 分析工具!这个工具似乎能分析峰值内存,这对批处理任务至关重要,因为内存是瓶颈。内存与性能有本质区别:性能是累积成本,优化任何部分几乎都有帮助;而非峰值内存的优化毫无影响,必须先找到峰值才能降低内存占用。Memray 比 Fil 强在原生调用栈、更多报告类型、更低开销和子进程支持,但 Fil 的火焰图可能更好。
itamarst查看英文原文
I'm excited to see more profiling tools for Python! This sounds like it does peak memory, which is critical for batch jobs, since that's the bottleneck. Memory is fundamentally different than performance in that it's a limited resource, instead of cumulative cost; making any part of the program faster almost always helps speed up the program (at least a little, or at least reduces CPU load), but optimizing non-peak memory has no impact. You have to be able to identify the peak in order to reduce memory usage. If you want peak memory profiling for Python that also runs on macOS, check out https://pythonspeed.com/fil/ (ARM support has some issues, but once I unpack my new Mac Mini I plan to fix it.) Ways memray is better than Fil: - Native callstacks. - More kinds of reports, and ability to do custom post-processing of data. - Much lower overhead (but not always, see reply). - Subprocess support. Fil I suspect has better flamegraphs: https://pythonspeed.com/articles/a-better-flamegraph/ And if you're running Python batch jobs, and want both peak memory and performance profiling in production, check out Sciagraph: https://pythonspeed.com/sciagraph/ (You can probably cobble together something like Sciagraph with py-spy + memray, but you won't e.g. get timeline reports designed with batch jobs in mind.)
赞赏彭博作为金融机构愿意开源高质量工具,体现了优秀的工程文化
彭博把这个工具开源出来真是太棒了。很多金融机构根本不会考虑开源任何东西,即使像这样通用的工具也不行,我所在的(高频交易/做市)公司就不会这么做。这也体现了良好的工程实践,他们应该是这个领域里工程做得比较好的公司之一。
noitpmeder查看英文原文
Want to chime in to say it's awesome that Bloomberg is releasing this product to the world. I don't think many financial organizations would even consider releasing anything as open source, even if it is as general purpose as this tool. My current one (HFT,MM) wouldn't go for it. It also shows good engineering practices. I'd venture a guess that they are probably one of the better ones in the space. Makes me want to work there! Maybe one day...
异步协程代码的内存分析仍是难题,现有工具难以应对
顺便问一句:有人能推荐一个适合大规模异步代码库的分析器吗?我的经验是协程的并发特性让人很难推断某个时间点上正在发生什么。如果你不知道某一时刻有多少东西在 await(以及它们可能在与哪些外部事物交互),就很难确定代码路径的内存占用。
uniqueuid查看英文原文
Tangential question: Does anybody have a good recommendation for a profiler that works well with massively async codebases? My experience has been that the concurrent nature of coroutines can make it hard to reason about what's going on at a particular point in time. If you don't know how many things you're awaiting on at a specific moment (and what potential external stuff they may be interacting with), it's not exactly easy to identify memory usage of codepaths.
历次讨论
同类还值得读
直接在浏览器中启动 Linux 内核
Boot a linux kernel right inside your browser.
Fabrice Bellard 的 JSLinux 页面,用 JavaScript 在浏览器里启动一个 Linux 内核,并提供一个可交互的终端环境。
工具与资源上首页 5 次 · 2011–2025bellard.orgHacker News 未公开功能与行为清单(2018)
A List of Hacker News's Undocumented Features and Behaviors (2018)
数据科学家 Max Woolf 整理的 Hacker News 未公开功能与行为清单,涵盖投票机制、排名算法、账号惩罚等社区运作细节。
工具与资源上首页 5 次 · 2019–2025github.comMapSCII:在终端中用盲文和 ASCII 渲染地图
MapSCII – Braille and ASCII map renderer for the console
MapSCII 是一个在命令行终端里渲染地图的开源工具,用盲文字符和 ASCII 字符把地理数据画成可交互浏览的字符地图。
工具与资源上首页 5 次 · 2017–2025github.com顶尖大学的免费在线课程
Free Online Courses from Top Universities
Open Culture 整理的一份免费在线课程合集,汇集了全球顶尖大学(如 MIT、斯坦福、耶鲁等)向公众开放的网络课程资源,涵盖音频、视频等多种形式。
工具与资源上首页 5 次 · 2010–2022openculture.com淘气字符串:容易引发用户输入问题的字符串清单
Naughty Strings: A list of strings likely to cause issues as user-input data
一个 GitHub 仓库,收录了大量容易让程序出问题的用户输入字符串,如特殊字符、emoji、SQL 注入片段、系统命令等,供开发者用于测试输入处理逻辑。
工具与资源上首页 4 次 · 2015–2020github.com