系统与基础设施github.com编辑精选
我们如何让 GitHub 变快:GitHub 新架构详解
How We Made GitHub Fast: A detailed look at GitHub's new architecture
2009 — 2015,它一次次回到首页
黑点是上首页(≥ 50 分)的那次投稿,空心点是没冲上去的投稿,点越大分数越高。
导读
GitHub 官方博客详解其 2009 年为应对流量增长而重构的架构:用 Nginx、MySQL、Redis、RabbitMQ、Memcached 等组件搭建了一套高吞吐、可水平扩展的系统,并自研了 BERT/BERT-RPC 作为序列化与 RPC 协议。文章坦诚分享了各组件选型原因和踩过的坑。
为什么经久不衰:它罕见地公开了高速增长期网站后端的真实架构与权衡,是分布式系统设计的经典案例,至今仍有参考价值。
HN 评论精选
RabbitMQ 作者解释 AMQP 的学习难点:队列、Pubsub、消息传递是三件不同的事
大家好,我是 RabbitMQ 的 Alexis。是的,我们实现了 AMQP,同时也支持 STOMP 和 HTTP Pubsubhubbub 等协议,因为有时人们并不需要 AMQP 模型的全部能力。AMQP 初学时确实难懂,主要原因是它混合了三件事:队列、发布订阅和消息传递——这三者并不相同:队列以状态管理在途数据,Pubsub 负责路由到消费者,消息传递则负责封装。另外,人们常困惑何时该用消息中间件、何时该用数据库做枢纽——用数据库来排队和管理订阅通常不是好主意。对初学者来说,从客户端库入手会更容易,我们的客户端库多到数不清。
rabbitmq查看英文原文
Hi, alexis here from RabbitMQ. Yes, we implement AMQP. We also provide support for other useful things like STOMP and HTTP Pubsubhubbub. We implement these other protocols as well as AMQP because some times people don't need to use the full and awesome power of the AMQP model. AMQP is initially hard to grok. I think the main reason for this is that AMQP combines three things: Queues, Pubsub, and Messaging. These are Not The Same. Queues manage data in flight as state, Pubsub routes data to consumers, and Messaging frames it. So yes, as someone pointed out above, it would be nice to use some but not all of this from time to time. We are working on ways to make that super easy - please get in touch if you can help. Another thing that people find hard is figuring out when to use message hub technology, and when to use a database as a hub. Using a database to queue and manage subscriptions to data streams is generally Not A Good Idea. Here’s a presentation I did which attempts to articulate some of the issues with this: http://www.rabbitmq.com/resources/RabbitMQ_Oxford_Geek_Night.pdf So, for someone using AMQP or any other Pubsub tech for the first time, there can be a 'huh, where do I start' element. But as some commenters point out, if you look at the client libraries it may be easier to get started. We've actually lost count of how many clients there are, so take your pick. List of clients: http://delicious.com/alexisrichardson/rabbitmq+client Getting started: http://blogs.digitar.com/jjww/2009/01/rabbits-and-warrens/ (Python centric) and http://www.infoq.com/articles/AMQP-RabbitMQ (Ruby centric) To the commenter who said the AMQP spec is 300 pages long. You may have a better time if you look at AMQP 0-91 which is much shorter than that at 40 pages mostly covering edge cases that you can ignore. The nub of AMQP can be communicated in under a page. BERT and BERT-RPC look cool. But - re the comments above - I would not see BERT-RPC as an ‘alternative’ to AMQP though. The GitHub blog post talks about PB and Thrift and JSON-RPC, all of which have been integrated with RabbitMQ. If you want to do RPC, there is no ‘one true system’ yet. Typically we have found that different people get value from different RPC metaphors in different languages. Maybe BERT-RPC will get more traction than the others - it definitely looks interesting. I hope this is all useful or at least of passing interest. Here are some more links that may be worth a glance: General background: http://www.rabbitmq.com/how.html AMQP and XMPP: http://www.igvita.com/2009/10/08/advanced-messaging-routing-with-amqp/ Feel free to contact us directly at info at rabbitmq dot com. Cheers, alexis
质疑直接给 SSHd 打补丁的做法,建议用 PAM 模块替代
“我们修改了 SSH 守护进程,使其从 MySQL 数据库查询公钥”——这看起来很奇怪,至少可以用 PAM 模块实现。如果直接改 SSHd,就要承担跟进上游变更的负担。甚至已有直接对接 MySQL 的现成模块。
timf查看英文原文
> We have patched our SSH daemon to perform public key lookups from our MySQL database That seems strange, that could be a PAM module at the least. If you patch SSHd then you are burdened with keeping up with changes, etc. There's even a module for direct mysql: http://sourceforge.net/projects/pam-mysql/
惊讶于 GitHub 后端服务链路的复杂度
这篇文章让我大开眼界。我一直以为 GitHub 是做前端的,但也许他们其实是做后端的?这种复杂度常见吗?我不是说这不好,相信每个环节都有其理由,但一个网页被服务之前要经过这么多服务,实在令人惊讶。
Lewisham查看英文原文
This blog post blew my mind. I generally thought of GitHub as being in the front-end business, but maybe they're really in the back-end business? Is this sort of complexity common? I'm not saying I think it's bad, as I am sure it's all there for a good reason, but it just seems like so many services to bounce through before a web page is even served.
赞赏文章坦诚分享真实痛点与非主流方案
好文章,Tom。小规模部署很直接,复杂部署也相当模板化。这篇文章像一股清流,坦露了你们遇到的痛点和那些不常见的解决方案,非常感谢分享!
dylanz查看英文原文
Awesome post Tom. Smaller deployments are pretty straight-forward, and complex deployments are still pretty cookie-cutter. This is a breath of fresh air, as it uncovers a lot of the pain points you were faced with, and the not-so-common solutions. Thank you very much for sharing!
历次讨论
同类还值得读
TTY 揭秘(2008)
The TTY demystified (2008)
经典文章深入剖析了 Unix/Linux 中 TTY 子系统的工作原理,包括线路规程、终端驱动、信号与控制字符等机制。
系统与基础设施上首页 11 次 · 2009–2025linusakesson.net编辑精选为什么 GNU grep 这么快
Why GNU grep is Fast
FreeBSD 邮件列表上一篇广为流传的帖子,解释 GNU grep 为什么快:核心在于 Boyer-Moore 字符串搜索算法配合精心优化的内部循环和 I/O,让程序在搜索时“…
系统与基础设施上首页 9 次 · 2010–2022lists.freebsd.orgXplain:为普通人讲解 X11
Xplain – Explaining X11 for the rest of us
《Xplain》是作者(HN 用户 Jasper_)写的一系列 X11 长文,用他在浏览器中重建的 X11 绘图模型做交互演示,讲解 X11 的绘图模型与窗口系统设计。
系统与基础设施上首页 6 次 · 2013–2020magcius.github.io编辑精选旋风教程:为 Linux 创建极小 ELF 可执行文件(1999)
A Whirlwind Tutorial on Creating Really Teensy ELF Executables for Linux (1999)
breadbox 写于 1999 年的经典教程,主题是如何为 Linux 手工构造并裁剪出体积极小的 ELF 可执行文件。
系统与基础设施上首页 6 次 · 2007–2025muppetlabs.comPostgreSQL 内部原理
The internals of PostgreSQL
《The Internals of PostgreSQL》是一本免费在线书籍,系统讲解 PostgreSQL 的内部实现,包括查询处理、并发控制、WAL、缓冲区管理等核心机制。
系统与基础设施上首页 5 次 · 2017–2025interdb.jp编辑精选