Open Source GraphQL CDN / Edge Cache with Cloudflare, Fastly, and Fly.io

WunderGraph Cosmo is fully open source, Apache 2.0 licensed. Today, I'd like to explain how you can add edge caching to your federated GraphQL APIs with the Cosmo Router, without locking yourself into a specific vendor.

Caching GraphQL on the Edge should be vendor-agnostic.

Services like Akamai and GraphCDN / Stellate offer proprietary solutions to solving this problem. We'll compare the different approaches and their tradeoffs.

A good question to start with is why we've created proprietary GraphQL CDN solutions in the first place?

Most GraphQL implementations ignore how the web works

The problem with most GraphQL implementations is that they don't really use the "platform" they are operating on. By platform, I mean the web, or more specifically HTTP and the REST constraints.

The web has a lot to offer, if you're using it in the right way. If read requests (Queries) were using the GET verb, combined with Cache-Control Headers and ETag, Browsers, CDNs, Cache Servers like Varnish, Nginx and many other tools could handle Caching out of the box. You wouldn't really need a service that understands GraphQL.

However, the reality is that most GraphQL APIs make you send Queries over HTTP POST. Cache-Control Headers and ETags don't make sense in this case, as all participants on the web think we're trying to "manipulate" something.

So, we've created Edge Caching solutions that rewrite HTTP POST to GET and are capable of invalidating in very smart ways. By analyzing Queries and Mutations they're able to build up a cache and invalidate Objects as mutations flow through the system.

As discussed in the Layered System constraint by Fielding, intermediaries can be used to implement a shared cache to improve the latency of requests.

The problem I see with "smart" GraphQL Edge Caches is that they look into the GraphQL Operations to figure out what to cache and what to invalidate. Some GraphQL CDN implementations even allow you to use their API to invalidate objects.

What sounds very smart creates a huge problem, you're building a second source of truth. Before using the GraphQL CDN, all you've had to do is implement your resolvers. Now, you've got to think about how to invalidate the CDN.

Even worse, you're now programming against a single vendor and their implementation, coupling your application to a single service provider. You can't just switch easily from one GraphQL CDN provider to another. Contrary to REST/HTTP APIs, there's no standard on how to Cache GraphQL APIs, hence every implementation will be different.

Another issue is that we're creating a secondary source of truth. Imagine we're putting a GraphQL CDN in front of the GraphQL API of GitHub to cache the Issues for a repository. If we're using "smart" Cache invalidation using Mutations, we're not able to update the cache if a user is bypassing our CDN and uses the GitHub API directly.

A GraphQL CDN really only works if 100% of the traffic flows through the system.

Testing takes up a huge part of software development, and we definitely want to have a great Developer Experience when building our APIs. Using a proprietary, cloud-only, Caching solution means that we're not able to test it locally or from within our Continuous Integration systems.

You'll be developing on you local machine without the CDN, only to find out that something behaves weirdly when you're using the CDN in production.

If we were using standardized caching directives, we'd be able to use any Cache Server, like Nginx or Varnish within our CI/CD pipeline to test our APIs.

As mentioned earlier, there's no specification for how GraphQL Caching should work. Every implementation is different, so you're always tied to a specific vendor. They might get broke, they might get acquired, they might cancel their service, they might change their pricing model.

In any case, it's a risk that needs to be managed.

Depending on the setup, adding a GraphQL CDN to your architecture could mean that your browser-based applications have to make cross-origin requests. That is, when your application is running on example.com and your GraphQL CDN runs on example.cdn.com, the browser will always make an extra Preflight request.

It's possible to cache the Preflight request, but this still means that we'd have to do at least one extra request on the initial page load.

Let's say your application requires your users to be logged in, but you still want to be able to apply caching.

Aside from that, you'd also like to implement Server-Side Rendering (SSR). In the ideal scenario, you'd have your users log into your authentication server, which sets a cookie on your apex domain, so that you're logged in on all subdomains. If your CDN is running on a different domain, it's going to be impossible to server-side render a page as the browser will not send the user's cookie to the CDN domain.

Luckily, some providers offer custom domains, so cookie-based authentication might still work for you.

Here's an example of a GraphQL Operation that invalidates easily.

If you've previously Queried the user with id 1, you can invalidate all records with that id and invalidate the following Query:

Let's make it a bit more complex and query for all your friends:

Now, you made some good friends at the last conference. Let's add them:

We've now added all users as friends that had visited a conference whose id equals 7. At this point, we don't get back all the data when we query again for all your friends because the cache can't know that the addFriends mutation has invalidated the cache for the friends query.

At this point, you've got to start adding response tagging, surrogate-keys, or analysing the GraphQL return types to make your invalidation strategy "smarter".

We'll pick up this topic again after the benefits.

When there's a cost, there are also benefits!

Using a GraphQL CDN means, you don't have to change much of your application. In an ideal scenario, you simply change the URL of the GraphQL Server and everything should work.

You also don't have to deploy and configure any tooling. You're buying a ready-to-use service that just works.

Despite the problems discussed earlier, a GraphQL CDN can probably improve the performance of many applications.

As discussed in the section on Cache Invalidation, building a smart CDN service that doesn't know anything about your GraphQL Backend is actually extremely hard. The problem lies in the fact that the Backend is the source of truth, but doesn't share all this information with the Cache.

In fact, if you're running into such issues where invalidation becomes hard, you might want to apply caching at a different level, inside the resolvers, or even one level below, at the entity level, using the DataLoader pattern.

Additionally, if your data is expected to change frequently, it might make not much sense for you to cache at the Edge.

Another big underrated flaw is that the invalidation of a distributed cache is eventually consistent. This means you serve stale content after a mutation for a short period of time. If you don't respect that in your architecture it has the potential to break the business logic of clients.

This is not GraphQL specific but with HTTP Caching the semantics are better understood. In summary, don't use a GraphQL CDN if you need write-after-read consistency.

If your requirement is to support write-after-read consistency, there are multiple solutions to the problem.

One is to not cache at the Edge, but rather at the application layer. In this case, you'd trade latency for consistency, which can be a good trade.

Another way of solving the problem is by distributing the state across the edge and sharding the data based on location. This model of sharding is not always possible, but if shared state is only used across groups of users in the same location, this solution could work very well.

One example of this is Cloudflare Workers + Durable Objects, which gives you a simple Key-Value store that is persisted in a specific location, meaning that all users close to that one location can have consistent state at low latency.

If you've got a single GraphQL API, and this API is the only API your frontend is talking to, you fully own this API and no traffic is bypassing your Cache, then such a CDN might actually make sense.

Otherwise, I doubt you get the results you're expecting, especially not with the extra costs discussed earlier.

We've discussed the pros and cons of GraphQL CDNs, now I'd like to propose another approach that keeps you vendor independent.

When it comes to solving problems, sometimes it's smart to be dumb. A cache can be a lot smarter when it's dumb and plays by the rules of the web, and that's exactly what the Cosmo Router does.

How does it work?

I've been working with GraphQL for many years, and when an application talks to a GraphQL API, I've never seen it change the operations at runtime. What changes at runtime are the variables. The operations stay static.

The Cosmo Router uses that fact with Automatic Persisted Queries (APQ). The client hashes the operation and sends the hash. The first time the Router sees it, it doesn't have the operation yet, so the client sends the full body once and the Router stores it against that hash. After that, the client executes the operation by sending just the hash and its variables, the same idea as a prepared statement in a database.

Because the operation is now addressed by a hash instead of a full query body, the Router can accept it over HTTP GET, with the hash and variables in the query string.

Served over GET, a persisted operation becomes a plain, cacheable URL.

And that's the whole caching story.

We don't build a "smart" cache. We don't cache objects, and we don't build complex invalidation logic. A CDN caches the response of a unique URL, the same way it caches any other GET request.

To control what gets cached and for how long, the Router's Cache-Control policy sets the Cache-Control header on responses. You can set a default max-age for the whole graph and override it per subgraph, and the Router always applies the strictest policy across the subgraphs a query touches. Mutations are set to no-cache automatically, and so is any response that carries errors.

It's a very dumb approach. When a cached response expires, the CDN asks the origin for a fresh one. We're not creating a second source of truth, and we don't have to manage cache invalidation tags for nested objects.

It's similar to using Change Data Capture (CDC) as a source for Subscriptions versus simple Polling. CDC can get complex to get right, while simple server-side Polling often works just fine. It's simple and predictable.

There's really not much invented here. This is standard caching behaviour, and any service like Cloudflare, Fastly, Varnish, or Nginx supports it out of the box. The web has a standard for how everyone treats the Cache-Control header, and the Router plays by it.

By addressing operations as stable URLs, we've made GraphQL caching compatible with the web.

If you build tools for the web, you should respect how the web is built, otherwise you're creating more problems than you're solving.

It's not just CDNs and cache servers that understand the Cache-Control header. Browsers do too, so they cache and revalidate your responses without you adding a single line of code.

And if you register your operations ahead of time instead of relying on APQ, you can restrict the Router to that safelist and reject anything else. If your frontend never changes its queries at runtime, there's no reason to accept arbitrary ones, and that shrinks your attack surface.

Caching over GET only helps read queries that are safe to cache. Anything user-specific or behind authentication, and anything that changes often, won't benefit much, and mutations are never cached. So this pays off most for shared, read-heavy responses.

You also need your clients to send persisted operations, either through Automatic Persisted Queries or by registering them ahead of time. If a client always sends full ad-hoc queries over POST, there's nothing stable for a CDN to cache.

None of this locks you in. The Cosmo Router is Apache 2.0 licensed, so you can run it yourself anywhere, or use WunderGraph's managed service if you'd rather not operate it.

Now that we've discussed the pros and cons of this approach to caching, let's see how we can deploy the Cosmo Router to achieve good Caching results.

First, you don't have to deploy the Cosmo Router globally. It's possible to run it close to your origin, e.g. the (micro-) services that you'd like to use. In this scenario, it's possible to run the Cosmo Router as an Ingress to your other services.

The architecture of this scenario looks like this:

If you're deploying the Cosmo Router close to the origin and you enable its Cache-Control policy, it sets the Cache-Control header on responses. This setup might already be enough for your scenario.

However, in some scenarios you'd like to add another layer of Caching Servers. This could be e.g. a cluster of Nginx or Varnish servers, placed in front of your Cosmo Router.

Architecture updated with this scenario:

Depending on where your users are, a centralised Caching Layer might not be sufficient for your use case.

In this scenario, you can use services like Cloudflare (Workers) or fastly, to add a globally distributed GraphQL CDN / Edge Cache.

What's important to note here is that you're not tied to a specific solution. As mentioned before, we're using standardized Cache-Control directives, supported by all Cache Server solutions, so you're not tying yourself to a specific vendor.

Updated Architecture:

Another option is to deploy the Cosmo Router directly on the Edge, removing the need for an additional Caching Layer. Services like fly.io allow you to deploy Containers as close to your users as possible.

The architecture of this scenario looks like the following:

There's one important point I'd like to make that applies to all solutions mentioned in this post.

If we're deploying a workload on the Edge, this service will be very close to the users, which is generally a good thing.

At the same time, depending on where on the "Edge" we're processing a request, there might be a random latency between ~0-300ms to your origin server(s) per roundtrip. If we have to do multiple roundtrips to fetch the data for a single client request, this latency can add up.

As you can see, it's not always beneficial to the overall performance to have logic running on the edge.

Cosmo isn't only about GraphQL backends. With Cosmo Connect , you can bring non-GraphQL services into your federated graph by compiling a subgraph schema into gRPC, and you can wrap existing REST or SOAP APIs without writing a full subgraph.

The caching layer described above isn't limited to GraphQL origins. It applies to every service in your federated graph.

So you get one place to apply Authentication, Authorization, and Caching across all of them.

You're rarely connecting a frontend to a single GraphQL server. You're connecting it to many services, and caching is one part of making that simple.

We've discussed various options to improve application performance with different kinds Caching systems. There are ready-to-use solutions that come with simplicity but also lock you into specific vendors.

With WunderGraph, we're trying to offer an Open Source alternative that's based on standards, implemented by many tools and vendors, so you're able to choose the best of breed solution for your situation.

If you're looking into adding Caching to your stack, we'd love to talk! Join us on Discord or connect on Twitter .

Frequently Asked Questions (FAQ)