Post

I worked on a few fairly prominent efforts in open source (Istio, OpenAPI), but I think of all of them, it’s gRPC that I think has the coolest story. Pull up a chair and set for a spell (any inaccuracies in what follows are 100% my fault for having a bad memory!).

It was 2014. Cloud was really starting to be a thing at Google. I was working with Louis Ryan on Google’s API infrastructure team, Apiary (pre-dating and unrelated to Apiary, the API company).

At the time, it was probably one of the most high performance, high scale API proxy in the world. Handling over 1M qps (as we called requests per second internally) for a couple of hundred APIs, 90th %ile latency of around 8ms. It was really, really high performance and a fun team to work on.

Everything from Maps to Drive to Translate ran APIs on that infrastructure, which handled all authn/authz, logging, API key and OAuth token infra, routing, protocol translation (JSON/REST to Stubby), etc. And we were pretty proud of what it could handle.

For those types of APIs, that 8 extra ms (and a hop through the proxy) were well within a tolerable latency budget. After all: when you’re loading a map on your mobile device, translating a phrase into Spanish, or fetching a Google Doc, 8ms won’t even be noticed.

But with Cloud, we were starting to look at a new type of API. We were looking at database APIs like Spanner. We were looking at message queues. And those APIs bring new kinds of volume, and new kinds of latency tolerance.

We knew that latency would have to improve by at least an order of magnitude (for things like DB calls, you want to measure your latency in 𝜇s, not ms).

And we knew that volume would REALLY need to go up (I remember only a few years later talking about an individual customer calling an API at nearly 1M qps!).

would love a thread about this. I assume envoy was involved but would love to hear your perspective.

Internally at Google, we used the proprietary Stubby protocol for service to service calls. Proto on the wire, hard contract-driven, extremely efficient. We knew that JSON/REST worked fine for the “typical” web APIs we had been serving, but again, think about the new types of API we’d be doing.

For a high performance database, JSON just wasn’t going to work. It adds latency and eats bandwidth to transcode values in memory into JSON, serialize, push that across the wire, then do the same thing on the other end. Louis knew from experience that proto was good for this.

But protobuf is a data format, not a protocol. We’d need to have a protocol that people could use outside of Google to call these APIs. And, of course: it wouldn’t make sense for this to be a proprietary protocol.