← All posts

Twelve Innocent Commits

I have approved a lot of pull requests I did not fully understand.

Not carelessly. The diff was small, the description was plausible, the tests were green, and there were nine more waiting behind it. That is the job. Nobody has ever thanked me for spending forty minutes on a twelve-line change.

You can build a weapon out of exactly that.

The trick is where you put the pieces. Put the whole thing in one pull request and it gets reviewed as a unit, and anyone competent scrolling that diff has a fair chance of seeing the shape. So it does not go in one request. Each piece is opened on its own and reviewed on its own, ideally by different people and weeks apart. Nothing is hidden and nothing is clever. One of them adds a signature check dressed as a timestamp test. Another teaches a validator to check one extra field, and a third puts a heartbeat into a maintenance routine, which is the sort of thing that keeps a network healthy. Every one of them is small and does roughly what its description says, which is why none of them fails review. No single review holds the whole shape, because the shape only exists once the pieces are assembled, and assembling them is nobody's job.

This is how most software gets built now, and the blind spot is designed in rather than accidental. Review is local to a pull request. It answers one question well, whether that request is correct and safe on its own, and it was never built to answer the larger one, which is what the last three months of merges now let the system do. I have never been asked that question in a review. I have never asked it either.

Security people have a name for a version of this, the slow attack, or the distributed backdoor, and the cases they cite usually involve one patient adversary working a codebase over years. The version that actually worries me needs no adversary. Honest engineers, each solving the ticket in front of them, can assemble something none of them would have approved if they had seen it whole. Intent is not required. Pressure and time will do it, given a review process that only ever sees one request at a time.

Being more careful does not fix it, which is the annoying part, because a careful reviewer still only sees the one request in front of them. What would fix it is somebody whose actual job is the accumulation, reading a few months of merged change as a single thing and asking what the system can do now that it could not in the spring. Almost nobody pays for that. It is slow and most weeks it turns up nothing, which is exactly why the week it turns something up, the thing it turns up tends to be serious.

I did not have to invent any of this for the book. I have watched real systems grow abilities nobody designed, one reasonable merge at a time, and The Chain only gives the drift a purpose.

In the book the reviewer is a woman named Klara. The twelve requests are waiting in her queue when she opens her laptop in the morning. Performance work. Comments. A timestamp check. Every one of them authored by a colleague she trusts, all of them tagged to her overnight, the timestamps running past midnight.

She reads the first diff twice and understands about half of what it is for, which for his code is the usual ratio. It is the other half that keeps her hand still. If he had sold them out, it would arrive looking exactly like this. Dull. Padded. Tagged to a friend.

Approve is one key. So is Reject.