That wall has a name: the agent era.

The problem with storing git at scale

Git was designed as a local tool. Linus Torvalds built it in 2005 to manage the Linux kernel — a single repository, on a single machine, with a single set of developers who knew what they were doing. It was brilliant for that purpose.

Hosting git for 100 million developers is a different problem entirely.

The core challenge is this: git stores data as a directed acyclic graph of objects — commits, trees, blobs, refs. Reading that graph requires following pointers. Following pointers across a network is slow. Replicating that graph across multiple servers for durability requires coordination. Coordination has overhead. The more you scale, the more overhead compounds.

Every major git hosting company has had to solve this problem from scratch. Their solutions reveal a lot about where the industry is going.

GitHub: Spokes and the three-phase commit

GitHub's storage system is called Spokes, originally launched under the name DGit (Distributed Git) around 2013. Before Spokes, GitHub stored repositories on single file servers. Losing a server meant losing repositories. That was obviously not acceptable at scale.

Spokes solved this with a straightforward principle: store every repository on three independent servers, separated at the physical rack level. If one rack fails, two others remain. The key insight was that git itself is already a distributed system — every clone contains the complete history. So instead of building a complex distributed file system, GitHub just ran three independent git servers and kept them in sync.

The coordination mechanism is three-phase commit (3PC). When you push to GitHub:

Phase 1 — Prepare: The orchestrator asks all three replicas: can you accept this update? Each replica checks if it is ready.

Phase 2 — Lock: If all three vote yes, the orchestrator says: get ready to commit. Each replica locks the resource.

Phase 3 — Commit: Orchestrator says: commit now. All three apply the change simultaneously.

If any replica fails at any phase, the whole transaction rolls back. Your push is rejected. This guarantees that when a push succeeds, the data exists in at least two independent locations. Durability before acknowledgment.

This worked extremely well for a decade. Three replicas per repository was the sweet spot — enough redundancy, enough headroom.

The problem that emerged is what Cursor described precisely in their engineering blog: 3PC has constrained horizontal scalability. The more replicas you add, the slower every push gets. You are only as fast as your slowest replica. Adding a fourth or fifth replica for a particularly busy monorepo means four or five round trips before every push is acknowledged. Latency compounds.

For repositories with thousands of pushes per day — large monorepos, active open source projects — this ceiling became real. And with AI agents that push code continuously rather than humans who push a few times a day, the ceiling gets hit much faster.

GitLab: Gitaly and the separation of concerns

GitLab approached the same problem differently. Instead of building a replication system on top of git servers, they first abstracted the git layer itself.

The result is called Gitaly — a gRPC service written in Go that handles every git operation GitLab needs. Clone, push, fetch, log, blame, diff — instead of these operations happening inside the main Rails application with direct disk access, they all go through Gitaly's remote procedure call interface.

Replication is handled by a separate component called Praefect. Praefect sits in front of multiple Gitaly nodes as a router and transaction manager. It knows which Gitaly node is the primary for each repository, routes reads to any available replica, and broadcasts writes to all replicas.

The key architectural difference from GitHub's Spokes is how Praefect handles consistency. Where Spokes uses strong consistency — a push is only acknowledged when a majority of replicas have applied it — Praefect uses a primary-first model. The push succeeds when the primary Gitaly node acknowledges it. Replication to secondary nodes happens asynchronously afterwards.

This makes GitLab's write path faster at the cost of a small eventual consistency window. If the primary node fails immediately after acknowledging your push but before replicating to secondaries, you could theoretically lose that push. In practice this window is milliseconds and Praefect has recovery mechanisms. For most use cases it is an acceptable trade.

What GitLab got right: Gitaly's separation of the git layer into a standalone service is genuinely elegant. It moved git operations from being an internal implementation detail of the Rails monolith to an explicitly defined interface. This made it possible to scale git storage independently from the rest of GitLab — add more Gitaly nodes without touching the application tier. It also made the system observable. Every git operation becomes a gRPC call you can trace, rate limit, and monitor.

What GitLab gave up: Gitaly is still fundamentally git-on-disk. JGit underneath, packfiles on NVMe, familiar git data structures. The abstraction is at the service boundary, not the storage format. This means GitLab inherits the same fundamental constraints as GitHub when it comes to the kinds of workloads that break disk-based git storage.

Cursor: Continuity and the S3 rethink

Cursor's engineering team published a detailed analysis of both GitHub and GitLab's approaches and concluded that the fundamental issue is the same in both cases: storing git repositories as packfiles on local disks is the wrong primitive for the agent era.

Their solution, called Continuity, makes a different bet: S3 is the source of truth.

Instead of storing git repositories as files on NVMe drives and then replicating those files across servers, Continuity stores git objects directly in S3. The local NVMe disk becomes a cache, not the primary store. Any server that needs to serve a clone or accept a push reads from and writes to S3.

The implications are significant:

Stateless compute. Any server can accept any push. There is no primary, no replica assignment, no rack separation to manage. Servers are interchangeable. Adding capacity means spinning up more compute, not provisioning more specialized storage servers.

Infinite horizontal scale in both directions. A repository with millions of pushes per day gets more compute allocated. A repository that gets used once and never touched again costs almost nothing to store. There is no minimum replica floor.

No 3PC overhead. Instead of coordinating three servers for every push, Continuity does a compare-and-swap operation on S3 — a single atomic write with conflict detection. If two pushes race, one wins and one retries. Much simpler coordination, much lower latency under load.

The trade-off Cursor accepted is that S3 is not as fast as NVMe for random reads. A cache miss — reading a git object that is not on the local NVMe — incurs S3 latency. For hot repositories this rarely matters because the cache stays warm. For cold repositories accessed after long periods, the first clone is slower.

This trade-off is acceptable when your workload is mostly agent-driven pushes rather than developer-driven clones. Agents push far more than they pull.

What else is being researched

Beyond the production systems, academic and industry research is exploring several directions:

Blockchain-based version control — researchers have proposed combining decentralized file storage systems like IPFS with blockchain for tamper-proof, transparent tracking of software changes. The appeal is immutability and auditability without a trusted central server. The practical challenge is throughput — blockchain consensus is orders of magnitude slower than git push.

Content-addressed object stores — git's own content-addressing (every object identified by SHA hash of its content) is a powerful primitive that has not been fully exploited by hosting systems. Research into purpose-built content-addressed stores that can serve git objects natively — not through JGit or libgit2 wrappers — could eliminate significant overhead.

JGit DHT storage — Shawn Pearce, one of the core JGit contributors who worked at Google, experimented with a DHT (Distributed Hash Table) backend for JGit that would store git objects across a peer-to-peer network rather than on dedicated servers. The experiment showed promise for read performance but clone latency was too high for production use. The work influenced thinking at both Google and the broader JGit community.

Research paper worth reading: "Distributed Version Control Systems: Leveraging Git for Effective Software Development" (International Journal of Science and Research, March 2024) covers the landscape of distributed VCS approaches. The key finding: git demonstrates superior efficiency in CPU and memory usage for branching operations, while Mercurial shows better storage optimization for large-scale projects with constrained storage capacity. The implications for multi-agent codebases where branching is constant are interesting.

What this means for the agent era

Every one of these architectures was designed for a world where humans push code. The assumptions baked into Spokes, Gitaly, and even Continuity reflect human development patterns: pushes measured in tens per day per developer, repositories with stable histories that grow predictably, clones dominated by CI systems running builds.

AI agents break every one of these assumptions.

An agent does not push ten times a day. It pushes hundreds of times. It creates repositories, abandons them, creates branches, merges them, patches set after patch set. The ratio of pushes to meaningful changes is enormous. The commit graph grows not linearly but explosively. The storage systems designed for human-paced development are hitting their ceilings faster than their designers anticipated.

This is the infrastructure problem underneath the workflow problem. DiffLoop exists to solve the workflow problem — removing humans from the merge decision, making merge criteria programmatic rather than social. But the infrastructure layer is catching up to the same realization: the fundamental primitives of git hosting were built for a different world.

The race is not just to build better review systems. It is to rebuild the entire stack from the ground up for the pace at which agents actually work.

That's when I realized the problem wasn't the AI. The problem was us pretending the old workflow still made sense.

The PR was designed for a different era.

The pull request was invented for a world where humans write code slowly, one feature at a time, over days or weeks. A colleague reviews it, leaves comments, you iterate. The feedback loop is slow but it works because the input is slow too.

AI agents broke this assumption completely.

When your agent ships 50 changes a day, the human review queue becomes a lie you tell yourself. Nobody reads 50 PRs. Nobody can. You either review nothing and ship everything, or you become the bottleneck that defeats the entire purpose of having an AI agent in the first place.

There is no middle ground. The math doesn't work.

The three ways teams cope. None of them work.

They rubber stamp.

Someone on the team becomes the PR approver. They click approve 40 times a day without reading anything. They trust the AI. Sometimes they're right. Sometimes a SQL injection makes it to production at 2am and they spend the weekend rolling back.

They slow the agent down.

They configure the agent to open fewer PRs, batch changes, wait for approval before proceeding. Now they've taken the fastest tool in their stack and deliberately made it slow. They've re-introduced the bottleneck they were trying to eliminate.

They give up on review entirely.

The agent pushes directly to main. No review, no gate, no safety net. This works until it doesn't. And when it doesn't, it's catastrophic — not a small bug but a fundamental trust breakdown with customers, with the team, with themselves.

The real problem is a mental model mismatch.

The PR assumes a human is the judge of quality. A human reads the diff, understands the context, makes a call. That assumption made sense when humans wrote all the code.

But when an AI writes the code, why is a human still the judge? The human didn't write it. They don't have the context the agent had. They're reading a diff cold, often hours or days after the agent has moved on to three other tasks. They're worse at this job than the system that created the code in the first place.

We've kept the human in the loop not because it adds value — but because we never questioned whether it should be there at all.

What actually needs to happen.

The merge decision needs to become programmatic.

Not "AI helps humans review faster." That's CodeRabbit. That's GitHub Copilot for PRs. That's still a human making the final call, just with AI assistance. Useful, but not the answer.

The answer is removing the human from the merge decision entirely — and replacing it with a set of agents that evaluate what actually matters.

Security — does this diff introduce vulnerabilities? SQL injection, exposed secrets, insecure dependencies.

Quality — does this code meet the standards of the codebase? Is it idiomatic, readable, maintainable?

Tests — does this change have adequate coverage? Do existing tests still pass?

Correctness — does this do what the commit message says it does?

These are answerable questions. Not perfectly — nothing in software is perfect — but answerable well enough that a machine can make the call faster, more consistently, and more thoroughly than a human scanning a diff at the end of a long day.

When all four pass, the code merges. When any one fails, it's blocked with a full explanation. The agent iterates. No human required.

This is not about trusting AI blindly.

The argument against autonomous merging is always "but what if the AI gets it wrong?"

The AI gets it wrong sometimes. So do human reviewers. The difference is humans get it wrong slowly, inconsistently, and at a rate that doesn't scale. An AI gets it wrong at a known, measurable, improvable rate. You can tune it. You can audit it. You can see exactly why it approved or rejected every single diff.

When was the last time you could audit why your human reviewer approved something? When was the last time they could tell you, with specifics, why they clicked approve on diff 37 of 50 on a Tuesday afternoon?

The bar for autonomous merging is not "perfect." It's "better than what we're actually doing." And what we're actually doing is rubber stamping 47 PRs a week and hoping for the best.

The PR isn't going away tomorrow.

GitHub has 100 million users. The PR workflow is deeply embedded in how teams think about code. This transition will take years, not months.

But the direction is clear. The teams shipping the most code with the fewest incidents in 2026 are not the ones with the most disciplined human reviewers. They're the ones who figured out how to make the merge decision programmatic — how to encode their standards into a system that can evaluate code at the speed AI can produce it.

The PR was a great solution to a problem that no longer exists. The problem now is different. The solution has to be too.

We're building that solution at DiffLoop.

Every diff reviewed autonomously. Security, quality, tests, docs — scored automatically. Good code merges. Bad code is blocked with a full explanation. No human in the loop.

If this is a problem you're living with right now — get early access at diffloop.com.