Should you enable GraphQL query batching in a federation router?

Only if you have clients that already use it. Cosmo disables batching by default and recommends HTTP/2 multiplexing for new implementations — it gives better load balancing and distributes requests across router instances. Enable batching when you need to support existing batch-dependent clients without requiring them to change.

Does GraphQL query batching reduce HTTP overhead compared to separate requests?

Yes — multiple operations go over one HTTP connection instead of several. That said, HTTP/2 multiplexing is the preferred approach for new work. It achieves the same overhead reduction while still distributing load across multiple router instances; a single batch always runs on one instance.

Is GraphQL query batching compatible with Apollo Client's BatchHttpLink?

Cosmo's batching accepts a JSON array of operations in a single POST request, which is the standard batching format. Whether a specific client library works depends on whether it sends operations in that format. The docs don't confirm or deny compatibility with any particular client library.

How do you set a maximum batch size for GraphQL requests?

Set max_entries_per_batch in the router's batching config. For example, max_entries_per_batch: 50 caps each batch at 50 operations. Pair it with max_concurrency to control how many of those run in parallel.

How does GraphQL query batching interact with rate limiting?

Rate limits apply per operation, not per batch. A batch of operations counts as that many operations toward the rate limit. Batching does not let clients circumvent quotas.

What happens if one operation in a batched GraphQL request fails?

The error is returned at that operation's position in the response array. The other operations in the batch still execute and return normally.

Does each operation in a batched GraphQL request get its own query plan?

Yes — each is planned independently. Because plans are cached by operation hash, identical operations in the batch reuse the cached plan rather than re-planning.

Can you include GraphQL subscriptions in a batched HTTP request?

No. If a subscription is included in a batch, the batched request will fail. Subscriptions are streaming operations — use a WebSocket connection instead.

How do you trace individual operations inside a batched GraphQL request?

The router emits three dedicated tracing attributes: wg.operation.batching.is_batched on the root span, wg.operation.batching.operations_count for the total number of operations, and wg.operation.batching.operation_index on each child span to identify which position in the batch it belongs to. These flow through OpenTelemetry to your observability platform.

Can you suppress batch operation error codes from GraphQL response extensions?

Yes. Set omit_extensions: true in the batching config. This omits error codes specific to batch operation failures from the response extensions. It defaults to false, so those error codes are included by default.

GraphQL Query Batching | Cosmo by WunderGraph

The problem

Support legacy batch clients without overwhelming the router

Teams need compatibility without giving up observability or control over resource usage.

Migration hits the clients first

A gateway that doesn't speak batched requests forces every client team to ship a new release before you can move traffic. That's quarters, not weeks.

Batched requests can overwhelm a router

A client sending one request with 200 operations can consume significant router resources. Without controls, a single batch can saturate the router.

Batch executions are opaque

Standard tracing shows one HTTP request, but not how many operations it contained or which one was slow.

Our solution

Keep legacy batch clients working without overwhelming the router

Cosmo Router accepts multiple GraphQL operations in a single HTTP request, executes them concurrently with configurable concurrency limits, and returns responses in the original request order, regardless of completion time.

This lets teams support existing batch-based clients without requiring client-side rewrites while still controlling resource usage at the router level.

1

2

3

4

5

Turn it on when you need compatibility. Keep HTTP/2 for greenfield clients.

Before & after

Migration and operations

Before Cosmo	With Cosmo
Batch-dependent clients block the gateway migration	Existing batch clients work without code changes
Oversized batches can exhaust router resources	`max_entries_per_batch` and `max_concurrency` limit resource usage
Batched requests hide inside single-request traces	Per-operation tracing attributes expose batch contents
Batch execution lacks configurable concurrency limits	Configurable concurrency limits control parallel execution

Tracing attributes

Per-operation visibility

wg.operation.batching.is_batched
wg.operation.batching.operations_count
wg.operation.batching.operation_index

See how individual operations behave inside a batched request.

Feature flags apply uniformly across all operations in a batch.

Use cases

Patterns teams enable first

Support legacy migrations, control resource usage, and improve observability from a single configuration.

Legacy client migration

Compatibility

If your app already packs several GraphQL operations into one HTTP request, turn on batching in the router config and set limits for batch size and concurrency. Those requests keep working when you move to Cosmo Router—no client rewrite for the cutover.

Resource-controlled batch processing

Limits

Configure max_entries_per_batch: 50 and max_concurrency: 5 to cap batch size and parallel execution. This keeps resource usage predictable and protects the router from oversized requests.

Batch performance analysis

Observability

Use OpenTelemetry spans from the router to analyse batch behaviour in your observability platform alongside the rest of your traces.

How query batching works

01

Opt in via config. Off by default.

Accept the batched POST

Clients send a JSON array of operations. Router recognizes the array shape when batching is enabled.

02

Reject oversized batches early.

Validate and enforce limits

Router checks the array against `max_entries_per_batch`. Batches larger than the limit are rejected before any planning happens.

03

Concurrency limited by max_concurrency.

Plan and execute concurrently

Each operation is planned and executed independently, with concurrency controlled by `max_concurrency`.

04

Failures are isolated per operation.

Return in order

Responses are collected and returned as an array in the same order as the request. Errors are returned at the matching position in the response array. One bad operation does not fail the batch.

Turn on batching for existing batch-based clients

Set concurrency limits, cap batch size, and trace individual operations.

Start Free Read the Docs

Batched GraphQL Operations Through One HTTP Request