The Merge Queue

rubyarchitecturebuilding-in-public

Pull requests look simple from the outside. Open one, get it reviewed, merge it. But the implementation has a surprising amount of state to manage, and the edge cases are where things get interesting.

Nine states

A Codex pull request can be in one of seven states:

draft → open → in_review → requires_changes → approved → merging → merged

With two additional terminal states: merge_failed and closed (closed without merging). That’s nine states in total.

The transitions are event-driven. Adding a reviewer moves draft to open. A reviewer commenting moves open to in_review. A reviewer requesting changes moves to requires_changes. All required reviewers approving moves to approved. Clicking Merge moves to merging. Success moves to merged. Failure moves to merge_failed.

Nine states feels like a lot for a pull request, but each state represents something meaningfully different. in_review is different from open because it means someone is actively looking at it. requires_changes is different from in_review because the reviewer has an opinion and it’s not “looks good.” These distinctions matter for dashboards, notifications, and knowing at a glance where things stand.

Three merge strategies

merge_commit: a true three-way merge. Two parents, a merge commit. The merge commit is GPG-signed by Codex’s server key. This preserves the full branch history.

squash: all commits on the source branch are collapsed into a single new commit on the target. The commit message lists the individual squashed commits. Also server-signed. This is the default, because most feature branches don’t have a commit history worth preserving in the target branch.

rebase: fast-forward only. If the target has diverged from the source, the merge fails. No new commit is created. The target ref just advances to the source’s commit. This is the cleanest option when it works, but it requires the source to be up-to-date with the target.

The concurrency problem

Here’s the thing that makes merge queues necessary: what happens when two people click Merge on different PRs targeting the same branch at the same time?

Without serialization, you get a race condition. Both merge jobs read the current target branch state, both compute their merges against it, and one of them writes a ref that’s immediately stale. The second merge either silently produces a broken commit (if it doesn’t check) or fails with a conflict (if it does).

Codex solves this with SolidQueue’s concurrency limits:

limits_concurrency to: 1, key: -> { "repo-#{pr.repository_id}" }

One merge per repository at a time. Period. If two PRs are merging, the second one waits in the queue until the first completes. This is the correct solution. Merges must be serialized because each merge changes the target branch state that the next merge depends on.

What happens when a merge fails

The merging state is intentionally transient. The PR moves to merging when the merge job is enqueued, and stays there until the job completes. If the job fails (conflict, permission error, unexpected git state) the PR moves to merge_failed with a reason.

From merge_failed, the developer can fix the issue (rebase their branch, resolve conflicts) and try again. The PR doesn’t need to go back through the review cycle. It’s still approved, the merge just didn’t succeed mechanically.

SHA safety

Between clicking Merge and the job actually running, someone might push new commits to the source branch. Maybe a last-minute fix, maybe an accidental push, maybe a colleague who didn’t know the PR was about to merge.

Codex captures the source branch’s commit SHA at merge initiation time (expected_sha). When the merge job runs, it compares the expected SHA with the current source commit. If they don’t match, the merge fails with a clear message: the source branch changed after merge was initiated. The developer can review the new commits and click Merge again.

This prevents merging code that hasn’t been reviewed. The approval was for the code at a specific commit. If the code changes, the approval is no longer valid.

Reviewer reset

Related to SHA safety: when new commits are pushed to a PR’s source branch, Codex resets all reviewer states back to pending. This means approvals are revoked, and reviewers need to re-approve after the new commits. It’s aggressive, but it prevents the scenario where a developer gets approval, pushes a sneaky commit, and merges before anyone notices.

The alternative, keeping approvals valid across new commits, is what GitHub does by default, and it’s the source of a real class of security issues.

Snapshot refs

After a PR is merged or closed, the source branch might be deleted (Codex supports auto-delete on merge). But you still want to see the PR’s diff: what changed, what was reviewed, what was merged. If the source branch is gone, how do you reconstruct that?

Codex creates snapshot refs: refs/pr-{id}/base and refs/pr-{id}/head. These are permanent git references that capture the merge base and source head at the time of merge or close. Even if both branches are subsequently deleted or rewritten, the PR’s diff is reconstructable from these snapshots.

Pre-merge checks

Before a merge is allowed:

  1. All required reviewers must have approved
  2. All PR tasks (checklist items) must be marked done
  3. No failing build results for the head commit
  4. The merge itself must be conflict-free (for merge_commit/squash) or fast-forwardable (for rebase)

Build results come from external CI systems (Wave CI in our case) that post status via Codex’s API. If no CI is attached to the repository, the build check is skipped. We don’t want to block merges on a system that isn’t configured.

Lessons learned

The merge queue is where all the other Codex features converge. RBAC determines who can merge. GPG signing ensures merge commits are verified. Push policies can require PRs on protected branches (no direct push to main). Build results from CI gate the merge. The activity feed records who merged what and when.

None of these features are particularly complex on their own. The complexity is in the interactions, and the merge queue is where those interactions become visible.