Cosmo Router: High Performance Federation v1 & v2 Router / Gateway

Launch Day 3 is all about routing federated GraphQL APIs.

Welcome back to the third day of our launch week. Today we're launching Cosmo Router, a high performance GraphQL API Gateway / Router compatible with Apollo Federation v1 and v2.

We've been in the business of building GraphQL API Gateways for more than five years now. When it came to building a Federation compatible Gateway, we were facing a couple of challenges which ultimately led to a refactor of some parts of our GraphQL Query Planner and Execution Engine.

As a result, we're happy to announce the Cosmo Router. Like all our API Gateways, it's written in Go and we've just open sourced it under the Apache 2.0 license. You can find the source code on GitHub in the Cosmo Monorepo. You can self-host it with Docker, Kubernetes, or an AWS Lambda-optimized binary, or run it through WunderGraph's managed service.

The WunderGraph Cosmo stack aims to provide an open source drop-in replacement for Apollo Federation / Apollo GraphOS. For some companies, it's not possible to use non-OSI approved licenses, especially for critical infrastructure like API Gateways. Others are afraid of vendor lock-in or simply cannot use cloud services for compliance reasons.

For all these companies, we've built Cosmo and the Cosmo Router.

Cosmo Router is built on top of graphql-go-tools , a high performance GraphQL engine written in Go. We've been working on this engine for more than five years now and it's the foundation of all our GraphQL products. In order to fully support Federation v2, we heavily refactored parts of the Query Planner, deprecated our existing Dataloader implementation and completely rewrote it from scratch with a new algorithm.

As you can see in the following benchmark, the new Dataloader implementation gives Cosmo Router a 3.3x performance improvement over the previous implementation (WG Gateway). Please take these numbers with a grain of salt, as they are only a snapshot of a specific benchmark. Different benchmarks will yield different results and we're by no means claiming that Cosmo Router is faster than other Gateways.

However, we're confident that our new Dataloader implementation is more efficient and resource friendly than the previous implementation. The new algorithm is using a lot less concurrency and works especially well with nested lists of entities.

That said, performance was not the reason for the rewrite. Implementing directives like @shareable and @requires gave us a lot of headaches with the old implementation. The new Dataloader implementation is conceptually much more complex, but the implementation is much simpler and easier to understand.

As such, the new Cosmo Router is not only faster, but most importantly, it's easier to maintain and extend.

If you want to learn more about the details, we will give an in-depth presentation of how Cosmo Router works under the hood at the GraphQL Conf in San Francisco Sep 21. I'd love to see you there.

In addition to the new Dataloader implementation, we've also added a new feature called Ludicrous Mode. The benchmark below shows the performance of Cosmo Router with and without Ludicrous Mode. It's important to note that Ludicrous Mode is a theoretical optimum and will usually not be reached in production, which is why we've put it in a separate diagram.

Ludicrous Mode is a feature that allows Cosmo Router to send origin requests that are read-only using single-flight. This means that if an origin request is read-only, e.g. a Query, and two or more requests in-flight are exactly the same, Cosmo Router will only send one request to the origin and share the result with all in-flight requests.

Ludicrous Mode still requires the engine to process each and every client request, but depending on the outgoing traffic, it can dramatically reduce the load on the origin.

This feature is especially useful for requests that are executed very often and are read-only, like resolving nested lists of entities.

As we've announced previously, we've open sourced Open Federation which is a specification for building federated GraphQL APIs. Open Federation is compatible with Apollo Federation v1 and v2, hence Cosmo Router is compatible with Apollo Federation v1 and v2 as well as Open Federation.

This means, you can use Cosmo Router as a drop-in replacement for Apollo Gateway and Apollo Router.

Understanding your API traffic is a key concern for running APIs in production, especially when it comes to federated GraphQL APIs. By definition, federated GraphQL APIs are distributed systems, which means that you need to understand the performance of each individual service as well as the overall performance of the system. You want to get insights into the performance of the gateway, all subgraphs, and other systems the subgraphs depend on. This way, you can identify bottlenecks and optimize your system.

To satisfy this need, we've added a whole stack of Open Telemetry (OTEL) tools to the Cosmo stack. As the Router is the entry point to your API stack, we're instrumenting it with OTEL and send metrics to the Cosmo OTEL Collector to give you an end-to-end view of your API traffic on Cosmo Studio.

That said, because it's OTEL, you can also send the metrics generated by Cosmo Router to any other OTEL compatible backend like Jaeger, Datadog, or others. This way, you can easily integrate Cosmo Router into your existing monitoring stack.

More info on Distributed Tracing can be found in the Docs . If you want to instrument your own subgraphs and other services, please follow this guide .

In addition to OTEL, it's also important to have metrics. As our goal is to build on top of existing standards, we've added Prometheus metrics to Cosmo Router. We're using the RED Method to provide you with the most important metrics.

For a federation gateway, the metrics worth watching are the RED signals, request rate, error rate, and request duration, plus the cache metrics that tell you whether planning is healthy: the query plan cache hit ratio and the normalization and validation cache hit ratios. A falling query-plan cache hit ratio is the early warning for the cold-start latency we cover below. The full list is in the Prometheus metric reference .

The Dataloader benchmark above is an internal comparison against our own previous gateway. The more useful question is how Cosmo Router holds up against Apollo, and the honest answer depends on the workload.

Where the gap shows up most is query planning under load. In our public federation benchmark, Cosmo Router has shown significantly lower P99 latency than Apollo Router on cold query-planning workloads, up to roughly an order of magnitude in the published tests. That is a P99-on-cold-planning result, not a blanket "10x faster" claim, so read the benchmark for the exact setup before quoting it.

At production scale the story is about efficiency, not just raw speed. eBay migrated its federated GraphQL traffic to the open-source Cosmo Router and reported 80% fewer router nodes and around 50% lower memory usage while serving the same production traffic, sustaining hundreds of thousands of requests per second. Because the Router is written in Go and built on graphql-go-tools, it uses CPU efficiently for the parsing, normalization, validation, and planning work that dominates a federation gateway's cost.

Query planning is the expensive part of routing a federated request, and the Router caches query plans to avoid redoing that work. The catch is that the cache is cold after every start and after every config reload. A fresh Router, or one that just picked up a new supergraph, has to plan queries from scratch, which shows up as a latency spike exactly when you scale up or ship a schema change.

The Cache Warmer addresses this. It precomputes query plans for your most expensive and most frequent operations from the Cosmo CDN, with an in-memory fallback, so a new instance is warm before it takes traffic. NerdWallet used it to cut query planning on its slowest operations from more than 10 seconds down to under a second.

The Router picks up new execution configs by polling the Cosmo CDN and hot-reloading without downtime, on a default 10-second interval you can tune.

The Router sits in front of every subgraph, so it is natural to worry about it becoming a single point of failure. In practice you run it the way you would run any critical, horizontally scaled service: multiple instances behind a load balancer, ideally spread across zones or regions. The deployment docs recommend at least three instances for high availability, and the Router scales vertically well thanks to Go.

Because each instance keeps its own query plan cache, the Cache Warmer matters here too: a replacement instance that comes up during a failover is fast immediately instead of cold-starting under load. SoundCloud runs the Cosmo Router across 10 containers in production, which is what this pattern looks like at scale.

Cosmo Router is compatible with Apollo Federation v1 and v2 at the directive level, so it works as a drop-in replacement for Apollo Gateway and Apollo Router. Most teams do not rewrite subgraphs to switch.

You can move in stages. Keep Apollo Router or Gateway running and swap only the schema registry first, using Compatibility Mode to point them at the open-source Cosmo registry, then cut the runtime over to the Cosmo Router when you are ready. Soundtrack Your Brand made the decision to move off Apollo in about a week and had Cosmo running in roughly a day. If you want the fuller comparison, see Cosmo as the open-source alternative to Apollo GraphOS.

The Router is more than a request/response gateway. With Cosmo Streams (event-driven federated subscriptions, or EDFS) it connects directly to Kafka, NATS, or Redis to power real-time federated subscriptions, while your subgraphs stay HTTP-only and stateless and the Router holds subscriber connections and applies per-subscriber filtering and authorization centrally.

With Cosmo Connect it federates non-GraphQL backends over gRPC. You describe a subgraph in SDL, generate a gRPC contract, and implement the RPCs in any gRPC language, so REST, SOAP, or SDK-backed services join the graph without a GraphQL server.

Since this launch, Cosmo Router has matured into the runtime for the full Cosmo stack: full Apollo Federation v1 and v2 compatibility, the Cache Warmer, Cosmo Streams for event-driven subscriptions, and Cosmo Connect for gRPC. Development continues in the open. If you find a bug or have a feature request, please open an issue on GitHub .

Alright, that's it for today. Now have a look at the GitHub Repository and give it a ⭐️.

If you want to learn more about Cosmo, check out the documentation .

Frequently Asked Questions (FAQ)