<article><h2>What we're building</h2><p><a href="https://github.com/wundergraph/cosmo">Cosmo</a>, our open-source federated GraphQL platform, already ships with an <a href="/blog/graphql-persisted-operations-llms-mcp">MCP server that lets you expose your graph and curated capabilities to LLMs and agents as MCP tools</a>. You define named GraphQL operations like <code>get_employees</code>, <code>update_employee_mood</code>, or <code>get_schema</code>, and the MCP server exposes each one as a tool that any MCP-compatible AI client can call.</p><p>Authorization was already part of the story. But not every tool should require the same level of access. A read operation like <code>get_employees</code> is low risk. <code>update_employee_mood</code> mutates state. <code>get_schema</code> exposes your API's internal structure. These operations have different risk profiles, and they should have different authorization requirements. An MCP client shouldn't need a god token with every scope upfront - it should only obtain the scopes it actually needs, when it needs them.</p><p>So we implemented <a href="https://github.com/wundergraph/cosmo/pull/2438">per-tool OAuth scope enforcement</a>. In our configuration, each MCP operation declares the scopes it requires:</p><pre data-language="yaml">mcp:
  oauth:
    enabled: true
    authorization_server_url: &quot;https://auth.example.com&quot;
    scopes:
      # MCP protocol layer
      initialize: [&quot;mcp:connect&quot;]
      tools_list: [&quot;mcp:tools:read&quot;]
      tools_call: [&quot;mcp:tools:call&quot;]
      # Cosmo native tools
      get_schema: [&quot;schema:read&quot;]
      execute_graphql: [&quot;graphql:execute&quot;]
      # Graph-specific tools (customer operations)
      get_employees: [&quot;employees:read&quot;]
      update_employee_mood: [&quot;employees:write&quot;]
</pre><p>Authorization is enforced in layers. An AI agent first obtains <code>mcp:connect</code> to establish a connection, then <code>mcp:tools:read</code> to discover available tools. When it calls a specific tool, it needs <code>mcp:tools:call</code> plus the scopes for that tool: <code>employees:read</code> for <code>get_employees</code>, <code>employees:write</code> for <code>update_employee_mood</code>. Each layer narrows access further, and the agent only obtains what it needs at each step.</p><p>Graph-specific tool scopes can be configured statically as shown above, or derived dynamically from <a href="https://cosmo-docs.wundergraph.com/federation/directives/requiresscopes#and-scopes"><code>@requiresScopes</code></a> directives in the schema.</p><p>The MCP spec has a name for this: <strong>scope step-up authorization</strong>. It's described in the <a href="https://modelcontextprotocol.io/specification/2025-11-25/basic/authorization#step-up-authorization-flow">authorization specification</a>. The TypeScript SDK has code paths for it. The <a href="https://modelcontextprotocol.io/specification/2025-11-25/basic/security_best_practices#scope-minimization">security best practices</a> section actively recommends it.</p><p>We assumed this was a solved problem, but it isn't yet. What we found while implementing it is worth sharing, because it affects anyone building per-tool authorization on MCP.</p><h2>The test</h2><p>We used the official <code>simpleOAuthClient.ts</code> example from the MCP TypeScript SDK. Our server on <code>localhost:5025/mcp</code>, an OAuth authorization server on <code>localhost:9099</code>, per-tool scopes configured.</p><p>The server returns a minimal scope in the initial <code>WWW-Authenticate</code> 401 challenge:</p><pre data-language="http">HTTP/1.1 401 Unauthorized
WWW-Authenticate: Bearer scope=&quot;employees:read&quot;,
                         resource_metadata=&quot;http://localhost:5025/.well-known/oauth-protected-resource/mcp&quot;
</pre><p>The idea: client connects with just <code>employees:read</code>, calls <code>get_employees</code> successfully, then when it tries to call <code>update_employee_mood</code>, the server returns a 403 with <code>scope=&quot;employees:write&quot;</code>, the client re-authorizes with the additional scope, and so on.</p><h2>What broke</h2><p>After connecting successfully with <code>employees:read</code>, the agent called <code>get_employees</code> without issue. Then it tried to call <code>update_employee_mood</code>. The browser opened for re-authorization. Then it opened again. And again. <strong>Infinite loop.</strong></p><p>The trace:</p><ol><li>Client calls <code>update_employee_mood</code> with a token carrying <code>employees:read</code></li><li>Server returns <code>403 Forbidden</code> with <code>scope=&quot;employees:write&quot;</code></li><li>SDK re-authorizes with <code>scope=employees:write</code>, <strong>overwriting</strong> the previous scope</li><li>Client gets a new token with <code>employees:write</code> but <strong>without</strong> <code>employees:read</code></li><li>Next request needing <code>employees:read</code> fails → <code>scope=&quot;employees:read&quot;</code> → overwrites <code>employees:write</code></li><li>Go to step 1</li></ol><p>The client SDK's <code>streamableHttp.ts</code>:</p><pre data-language="typescript">if (scope) {
    this._scope = scope; // overwrites, previous scopes are lost
}
</pre><p>An assignment instead of a union. Was the SDK wrong, or was it implementing the spec correctly?</p><h2>What the spec says vs. what the RFC says</h2><p>The MCP spec's <a href="https://modelcontextprotocol.io/specification/2025-11-25/basic/authorization#runtime-insufficient-scope-errors">Server Scope Management</a> defines three strategies for what servers should include in a 403 scope parameter:</p><ul><li><strong>Minimum approach</strong>: the scopes for the operation, plus &quot;any existing granted scopes as well, if they are required, to prevent clients from losing previously granted permissions&quot;</li><li><strong>Recommended approach</strong>: &quot;existing relevant scopes and newly required scopes&quot;</li><li><strong>Extended approach</strong>: existing, new, and related scopes</li></ul><p>All three expect the server to include the client's previously granted scopes.</p><p><a href="https://datatracker.ietf.org/doc/html/rfc6750#section-3.1">RFC 6750 §3.1</a> says:</p><blockquote><p>the resource server SHOULD include the 'scope' attribute with the value of the scope necessary to access the requested resource</p></blockquote><p>&quot;The requested resource.&quot; Not &quot;the requested resource plus everything else the client might need in the future.&quot; There's a gap between what the MCP spec asks servers to do and what the underlying RFC defines.</p><p>We may be misreading the spec here. The likely intention is to simplify clients: if the server includes previously granted scopes in the challenge, the client can treat it as a complete scope set without tracking state. That makes sense as a design choice, but it needs to be documented explicitly. Without that clarity, the guidance reads as contradicting RFC 6750, the reference SDK doesn't handle it consistently, and clients end up requesting more scopes than they need for a given operation.</p><p>Here's why we think this matters:</p><p><strong>It diverges from the RFC.</strong> Calling <code>update_employee_mood</code> needs <code>employees:write</code>. Including <code>employees:read</code> in the 403 challenge misrepresents the operation's requirements. The scope attribute stops describing what the resource needs and starts describing client session history.</p><p><strong>It requires servers to track client state.</strong> To include &quot;existing granted scopes,&quot; the server must inspect the client's current token. Stateless servers that validate tokens and report per-operation requirements can't easily do this.</p><p><strong>It places responsibility at the wrong layer.</strong> The server knows what each operation requires. The client knows what it has accumulated. Scope accumulation across a session is inherently a client-side concern.</p><h2>How we're thinking to solve it</h2><p>In our <a href="https://github.com/wundergraph/cosmo/pull/2438">PR</a>, the default behavior is what we believe is correct: the server returns only the scopes needed for the requested operation, consistent with RFC 6750. When a client calls <code>update_employee_mood</code> without <code>employees:write</code>, the 403 challenge says <code>scope=&quot;employees:write&quot;</code>. Nothing more. The server describes what the operation needs. The client is responsible for accumulating scopes across its session.</p><pre data-language="go">func (m *MCPAuthMiddleware) sendInsufficientScopeResponse(
    w http.ResponseWriter, operationScopes []string,
    claims authentication.Claims, err error,
) {
    challengeScopes := operationScopes

    if m.scopeChallengeIncludeTokenScopes {
        existing := extractScopes(claims)
        seen := make(map[string]struct{})
        for _, s := range existing {
            seen[s] = struct{}{}
            challengeScopes = append(challengeScopes, s)
        }
        for _, s := range operationScopes {
            if _, ok := seen[s]; !ok {
                challengeScopes = append(challengeScopes, s)
            }
        }
    }
    // ...
}
</pre><p>The reality is that current MCP reference SDK implementations, including the TypeScript SDK, replace scopes on each 403 challenge rather than accumulating them. So we added a single override:</p><pre data-language="yaml">mcp:
  oauth:
    scope_challenge_include_token_scopes: true
</pre><p>When enabled, the server unions the operation's required scopes with the scopes already present on the client's token. It's not an elegant configuration option, but it exists for one reason: to interoperate with clients that don't accumulate scopes yet.</p><p>The idea is that when the spec and SDKs catch up (and we hope they will), users can simply remove this option. The default behavior is already the more secure one.</p><h2>Two parts of the spec that pull in different directions</h2><p>The MCP spec's own <a href="https://modelcontextprotocol.io/specification/2025-11-25/basic/security_best_practices#scope-minimization">security best practices</a> aligns with our default behavior (<code>scope_challenge_include_token_scopes: false</code>):</p><ul><li>&quot;Emit precise scope challenges; avoid returning the full catalog&quot;</li><li>&quot;Incremental elevation via targeted <code>WWW-Authenticate scope=&quot;...&quot;</code> challenges&quot;</li><li>&quot;Minimal initial scope set containing only low-risk discovery/read operations&quot;</li></ul><p>This is precisely what our default does. But following this security guidance with a client that doesn't accumulate scopes produces the infinite loop we hit.</p><p>There's tension between two parts of the spec:</p><ul><li><strong>Security best practices</strong>: Be precise. Only return what this operation needs.</li><li><strong>Server Scope Management</strong>: Also include existing granted scopes. The &quot;Recommended approach&quot; includes them explicitly.</li></ul><p>And the spec doesn't yet give clients explicit guidance on scope accumulation:</p><ul><li>The <a href="https://modelcontextprotocol.io/specification/2025-11-25/basic/authorization#scope-selection-strategy">Scope Selection Strategy</a> only covers the initial 401.</li><li>The <a href="https://modelcontextprotocol.io/specification/2025-11-25/basic/authorization#step-up-authorization-flow">Step-Up Authorization Flow</a> doesn't say whether to merge or replace scopes.</li><li>The spec says clients &quot;MUST treat the scopes provided in the challenge as authoritative&quot;, but authoritative for what? For what the operation needs? Or as the complete set to request?</li></ul><p>These are the kinds of ambiguities that surface when you actually implement scope step-up end to end. The spec is still evolving, and we think clarifying this will help everyone building on it.</p><h2>What we're contributing to the protocol</h2><p>We've filed <a href="https://github.com/modelcontextprotocol/typescript-sdk/issues/1582">issue #1582</a> against the TypeScript SDK, and there's already a <a href="https://github.com/modelcontextprotocol/typescript-sdk/pull/1618">PR #1618</a> that adds client-side scope merging. The SDK fix helps, but without spec-level clarity, every new SDK implementation will need to figure this out independently.</p><p>We've also opened <a href="https://github.com/modelcontextprotocol/modelcontextprotocol/issues/2349">an issue against the spec itself</a>, suggesting three clarifications:</p><h3>1. Align Server Scope Management with RFC 6750 §3.1</h3><p>We think the scope attribute in 403 responses should describe the scopes needed for the requested operation, consistent with RFC 6750. An <code>update_employee_mood</code> 403 should say <code>scope=&quot;employees:write&quot;</code>, full stop. The server shouldn't need to know or include what the client already has.</p><h3>2. Add explicit client-side scope accumulation</h3><p>The <a href="https://modelcontextprotocol.io/specification/2025-11-25/basic/authorization#step-up-authorization-flow">Step-Up Authorization Flow</a> should specify that clients compute the union of their existing scope set and the challenge's scope set before re-authorizing. One sentence would prevent every SDK from independently rediscovering this problem.</p><h3>3. Clarify &quot;authoritative&quot;</h3><p>Clarify that &quot;authoritative for satisfying the current request&quot; means the scopes are required for this operation, not that they are the exclusive set the client should request.</p><h2>Where we're headed: surfacing missing scopes from the schema</h2><p>Cosmo already supports <a href="https://cosmo-docs.wundergraph.com/federation/directives/requiresscopes#and-scopes"><code>@requiresScopes</code></a> on fields and types in the schema. When a token lacks the scopes needed for a field, the router knows exactly which scopes are missing.</p><p>The next step is connecting that to the scope challenge. When an MCP client calls an operation and the token doesn't satisfy the <code>@requiresScopes</code> directives on the fields it touches, the router should be able to surface those missing scopes in the 403 <code>WWW-Authenticate</code> challenge. The client can then re-authorize with precisely the scopes it needs, no more.</p><p>This closes the loop between schema-level authorization and MCP scope step-up. The scopes are already declared in the schema. The router already evaluates them. All that's left is returning them in the challenge so the client can act on them. First, we need the protocol to get scope step-up right.</p><h2>Why this matters</h2><p>As AI agents become the primary consumers of APIs, per-operation authorization becomes critical. You don't want an agent that starts a session with <code>admin:*</code> because it might need to delete something later. You want progressive, least-privilege escalation. The same model that's been standard in OAuth for years.</p><p>The MCP spec is in a unique position to get this right. It's the emerging standard for how AI agents interact with tools and APIs. The authorization model it defines will be implemented by every MCP SDK, every MCP server, and eventually every AI platform that supports tool calling.</p><p>Getting scope accumulation right (clarifying that servers describe operations and clients accumulate sessions) is a small spec change with outsized impact. It aligns MCP with established OAuth semantics, enables stateless resource servers, and makes progressive authorization work out of the box.</p><p>We're building the infrastructure for AI agents to interact with enterprise APIs at WunderGraph. Per-tool authorization isn't optional. It's what enterprises expect. The MCP spec and ecosystem are moving fast, and we want to contribute what we've learned so that scope step-up authorization works reliably for everyone building on it.</p><hr><p><a href="https://wundergraph.com">WunderGraph</a> builds API infrastructure for AI. Our <a href="https://wundergraph.com/cosmo">Cosmo</a> platform turns federated GraphQL APIs into <a href="/blog/graphql-persisted-operations-llms-mcp">AI-ready MCP tools</a> with per-operation authorization.</p></article>

mcp-scope-step-up-banner.png

Cosmo's MCP server already exposes your graph as AI-ready tools. When we added per-tool OAuth scope step-up authorization so clients don't need a god token, we hit an infinite loop. The root cause: a gap between the MCP spec and RFC 6750 on scope challenges, plus SDK behavior that overwrites scopes instead of accumulating them. Here's what we found and how we're approaching it.

MCP Scope Step-Up Authorization: From Implementation to Spec Contribution

<article><p>When I joined WunderGraph in March, my first full feature was to improve access control in Cosmo, WunderGraph’s platform for managing federated GraphQL APIs. I didn’t expect the scope to cover so much: frontend, backend, CLI. But the range of changes made it a good opportunity to improve how access control works across the platform. It's one thing to build something, but another to help define how teams manage who can do what.</p><p>Groups introduce a new system for managing <a href="https://cosmo-docs.wundergraph.com/studio/rbac">role-based access control (RBAC)</a> in Cosmo. Built to give you more control, especially when your organization gets bigger.</p><h2><strong>Where the Idea Came From</strong></h2><p>This began because customers requested more control over their members. Some needed to restrict publishing access to specific environments, like limiting production deployments to only lead developers. Others wanted to prevent users from even seeing certain namespaces or subgraphs unless they were explicitly granted access to them.</p><p>At the same time, there was inconsistency between user roles and API key permissions. API keys were managed in a separate interface and always had admin access by default. That made it harder to limit what automation or external tools could do. It also meant there was no way to apply the same policies to both users and keys.</p><p>Fine-grained GraphQL access control was technically possible before, but it required workarounds and manual setups that didn't scale well. We needed a centralized and flexible model that could define access rules once and apply them consistently across all systems.</p><p>That’s what led us to Groups.</p><h2><strong>What Groups Let You Do</strong></h2><p>Groups make it possible to assign roles and resources in one place. Both members and API keys can be added to a group. And then, the group defines what they can access.</p><p>You can:</p><ul><li>Assign access to specific namespaces, federated graphs, or subgraphs</li><li>Use different roles for different resources</li><li>Apply the same permissions to both users and API keys</li></ul><p>If someone does not have access to a resource, they won’t even see it in the UI. Namespaces, graphs, and subgraphs that aren’t assigned to them won’t show up in the sidebar or any of their views. In the CLI, restricted actions, such as publishing or introspecting, will return errors if the group doesn’t allow them.</p><h2><strong>How Cosmo Groups Work</strong></h2><p>You create a Group. Then you add rules to it. Each rule links a role to one or more resources.</p><p>Important behaviors:</p><ul><li>If a group has no rules, its members and keys have no access.</li><li>If a rule does not specify any resources, it gives access to all.</li><li>If a resource is deleted, the rule falls back to all resources.</li><li>Groups are synchronized through SSO (via OIDC mappers).</li></ul><p>Cosmo currently provides the following roles, grouped by category:</p><h3><strong>Organization:</strong></h3><ul><li><strong>Admin</strong>: Provides read and write access to all the organization resources, billing, and settings.</li><li><strong>Developer</strong>: Provides read and write access to all organizational resources and read-only access to organizational settings.</li><li><strong>API Key Manager</strong>: Provides access to managing the organization's API keys.</li><li><strong>Viewer</strong>: Provides read-only access to all the organization resources and settings.</li></ul><hr><h3><strong>Namespace:</strong></h3><ul><li><strong>Admin</strong>: Provides write access to the namespace configuration.<ul><li>When resources are not assigned to this role, members and API keys will be able to create new namespaces</li></ul></li><li><strong>Viewer</strong>: Provides read-only access to the namespace configuration.<ul><li>When resources are not assigned to this role, members and API keys will be able to create new namespaces.</li></ul></li></ul><hr><h3><strong>Graph:</strong></h3><ul><li><strong>Admin</strong>: Provides write access to the graph, like configuring check overrides, creating cache warmer operations and so on.<ul><li>When resources are not assigned to this role, the members and API key will be able to create new graphs.</li><li>If the role is assigned to a namespace instead of a graph, the members and API keys will be able to create new graphs under that namespace.</li></ul></li><li><strong>Viewer</strong>: Provides read-only access to graph configurations.<ul><li>When resources are not assigned to this role, the members and API key will be able to create new graphs.</li><li>If the role is assigned to a namespace instead of a graph, the members and API keys will be able to create new graphs under that namespace.</li></ul></li></ul><hr><h3><strong>Subgraph:</strong></h3><ul><li><strong>Admin</strong>: Provides write access to subgraphs.<ul><li>When resources are not assigned to this role, the members and API key will be able to create and publish any subgraph.</li><li>If the role is assigned to a namespace instead of a subgraph, the members and API keys will be able to create and publish to any subgraph under that namespace.</li></ul></li><li><strong>Publisher</strong>: Provides write access to subgraphs, but, unlike Admin, members and API keys will not be able to create new subgraphs.<ul><li>When resources are not assigned to this role, the members and API key will be able to publish any subgraph.</li><li>If the role is assigned to a namespace instead of a subgraph, the members and API keys will be able to publish to any subgraph under that namespace.</li></ul></li><li><strong>Viewer</strong>: Provides read-only access to subgraphs.</li></ul><p>Each role type can only be added once per group. For example, you can assign the Admin role to multiple resources, but you can’t assign Admin more than once within the same group.</p><p>In <a href="https://cosmo-docs.wundergraph.com/studio/intro">Cosmo Studio</a>, admins use the resource selector UI to assign roles to specific scopes and resources, like namespaces or subgraphs. For example, you can select only the development namespace or a single subgraph within a federated graph. The selector adjusts based on what you pick. If you choose a namespace, it disables individual graph selection. This helps avoid mistakes.</p><p>In the CLI, API keys inherit the roles of their group. This means that if a key is added to a group that can publish to a subgraph, that key can be deployed from CI/CD. But if it doesn’t have access, the operation fails.</p><h3>Rather than just describing it, let me show you how it works:</h3><h2><strong>What We Had to Build</strong></h2><p>We initially went with a hierarchical approach, but it ended up being confusing. If someone had access to a namespace, they also had access to all its graphs. This was not what we wanted. So we removed the hierarchy and made everything explicit.</p><p>We iterated over the resource selector a few times. We wanted it to be simple yet also capable of handling complex cases. For example, we had to make sure the selector could handle selecting all graphs in a namespace or just one. We also had to update the sidebar and features to hide content that users don’t have permission to access.</p><p>Most of the work was in the backend. But the changes are visible across the entire system. The permission checks are enforced everywhere, including introspection APIs.</p><p>We deprecated the separate subgraph member and API key selectors in favor of managing everything through a unified permissions system.</p><h2><strong>Why Groups Make Permissions Easier</strong></h2><p>Now you can:</p><ul><li>Assign production access only to specific users or teams.</li><li>Create groups with scoped roles for CI/CD keys, internal devs, or external collaborators.</li><li>Ensure users only see and interact with resources they’ve been granted access to.</li><li><a href="https://cosmo-docs.wundergraph.com/studio/api-keys/api-key-resources">API Keys</a> with limited resources will continue to work, but we recommend migrating them to groups.</li></ul><p>If you were already using the old system, existing users and keys were automatically placed into groups that reflect their previous permissions. The legacy roles still work, but using Groups makes management and audits much simpler.</p><p>WunderGraph Hub is our new collaborative platform for designing, evolving, and shipping APIs together. It’s a design-first workspace that brings schema design, mocks, and workflows into one place.</p><h2><strong>My Experience</strong></h2><p>This was my first full feature at WunderGraph. Since it involved access control, there was extra attention on getting the security right. Once we had aligned on the requirements and structure, implementation moved smoothly.</p><p>Groups are now available in Cosmo. This new RBAC model gives you more flexibility and security across your federated GraphQL setup. You can manage everything in one place: who can see what, who can publish, and who can create.</p><p><a href="https://cosmo-docs.wundergraph.com/studio/groups">Read the full Cosmo Groups documentation</a></p><p>Let us know how it works for your team. I’m especially interested to hear from users with complex environments.</p><p>Feel free to open a <a href="https://github.com/wundergraph/cosmo/discussions">discussion</a> on GitHub or reach out if your team has unique access needs.</p><hr><hr></article>

managing-permissions-groups-banner.png

Manage access with precision in Cosmo. Groups let you control who can view, publish, or deploy, across users and API keys, in one centralized system.

Managing Permissions in Cosmo Just Got Easier with Groups

<article><p>When I started the interview process at WunderGraph, I didn't think much of the take-home assignment. I assumed it would just be a solid frontend task to showcase how I think. What I <em>did not</em> expect was that it would end up in production with actual users.</p><p>This is the story of how a single interview task became my first open-source contribution to Cosmo—and marked the start of my journey at WunderGraph</p><h2>Designing the Playground Share Feature for Cosmo</h2><p>I applied for a Senior Frontend Engineer role. The assignment was to build a small but thoughtful UI feature inside <a href="https://cosmo.wundergraph.com">Cosmo Studio</a>, WunderGraph's frontend for managing GraphQL federation.</p><p>The task was to allow users to share their current Playground session with a link. It sounded simple on paper, but it involved:</p><ul><li>Selecting which parts of the session to include (operation, variables, headers)</li><li>Compressing and encoding the selected state into a shareable URL</li><li>Restoring the session state cleanly on page load</li><li>Designing a smooth UI flow for it all, and handling edge cases gracefully</li></ul><p>I went all in. I treated it like a real feature, not just a task. I took time to understand the architecture, asked questions when I got stuck, wrote tests, and focused on the full experience. I had no idea it was based on an actual customer request.</p><h2>How It Became a Real Contribution to Cosmo</h2><p>After submitting the assignment, I had a review call with WunderGraph's CTO, <a href="https://de.linkedin.com/in/dustin-deus">Dustin</a>. He liked the approach and suggested that, if I was interested, I could clean it up and raise a PR to the main repo as an open-source contribution.</p><p>That's when it clicked: this thing might actually go live.</p><p>It didn’t ship right away. After joining the team, I spent some time refining the feature before finally pushing it into production.</p><p>I cleaned up the code, polished the user experience, and addressed feedback before submitting <a href="https://github.com/wundergraph/cosmo/pull/1833">this PR</a>. It introduced a modal-based UI for generating shareable session links that are encoded via <code>lz-string</code>. The goal was to make it easier for teams to collaborate, debug, and share reproducible setups.</p><p>Shortly after it shipped, I found out it was already being used in production by companies like <a href="https://soundcloud.com/">SoundCloud</a>. That was a surreal moment.</p><h2>Under the Hood: Decisions, Edge Cases &amp; Trade-offs</h2><p>While implementing the feature, I made a few deliberate technical and UX decisions to keep the experience clean, performant, and intuitive:</p><ul><li><p><strong>Compression Strategy</strong>: I used <a href="https://github.com/pieroxy/lz-string">lz-string</a> to compress the session state. It's widely supported, easy to use in the browser, and produces small payloads that are suitable for URLs without relying on newer APIs like <code>CompressionStream</code>, which don't work in older browsers.</p></li><li><p><strong>State Restoration</strong>: The logic restores the active Playground session on load using a custom query parameter (<code>playgroundUrlState</code>). It handles both optional and required fields carefully, only restoring headers and variables if they were selected and present.</p></li><li><p><strong>UI Edge Cases</strong>: The share modal disables options when no data is available (like headers or variables), shows toast messages when the URL is too long, and handles hydration errors with clear messaging. These patterns ensure that the feature feels stable, even in messy or unexpected scenarios.</p></li><li><p><strong>No Backend Dependence</strong>: The feature is completely stateless, requiring no backend logic or persistence—all state is stored directly in the URL.</p></li><li><p><strong>Navigating a Custom GraphiQL Setup</strong>: WunderGraph's Playground builds on top of <a href="https://github.com/graphql/graphiql">GraphiQL</a> with several extensions and internal overrides. Understanding the internal structure and identifying the right integration points took some effort — but once it clicked, it was a rewarding challenge.</p></li></ul><p>These kinds of product and architectural decisions are what made this fun: solving small but meaningful UX problems while thinking about scale, resilience, and clarity.</p><h2>Writing Developer Docs</h2><p>I also wrote the official developer-facing documentation for the feature: <a href="https://cosmo-docs.wundergraph.com/studio/playground/shared-playground-state">Sharing Playground Sessions</a>. The guide walks users through:</p><ul><li>What's included and not included in the link</li><li>How to generate and share it</li><li>Security considerations</li><li>Use cases for collaboration and debugging</li></ul><p>This was my first time contributing to public OSS docs, and I loved being able to treat documentation as a first-class part of the product.</p><h2>Lessons from My First Open Source Contribution</h2><p>Looking back, this experience taught me a lot:</p><ul><li><strong>Start with ownership</strong>: I treated the assignment like a real feature.</li><li><strong>Communication matters</strong>: Thoughtful PR descriptions and async reasoning helped build trust early on.</li><li><strong>Know what to prioritize</strong>: Done is better than perfect, especially when it solves a real user problem.</li></ul><h2>Joining WunderGraph Through Open Source</h2><p>Finalizing and shipping this feature was more than just a technical milestone—it showed me that WunderGraph genuinely values trust, autonomy, and thoughtful engineering. The team was supportive from the start. They were quick with feedback, open to ideas, and always willing to dive into details together. I got to dive into a complex codebase, shape the user experience, and contribute to something real from day one.</p><p>If that sounds like your kind of work, you might enjoy being part of the team. <a href="https://wundergraph.com/jobs">We're hiring</a>.</p><p>WunderGraph Hub is our new collaborative platform for designing, evolving, and shipping APIs together. It’s a design-first workspace that brings schema design, mocks, and workflows into one place.</p><p>Curious about the feature? Check it out in <a href="https://cosmo.wundergraph.com">Cosmo</a>.</p><p>Questions or feedback? I'd love to connect — you can find me on <a href="https://www.linkedin.com/in/grg-akshay">LinkedIn</a> or drop a note in the <a href="https://discord.gg/Jjmc8TC">WunderGraph Discord</a>.</p></article>

interview_task_to_prod_banner.png

The story of how an interview assignment turned into a real feature and became my first open-source contribution to Cosmo.

From Interview Task to Production Feature

scaling_graphql_observability

GraphQL Observability at Scale

See how Cosmo’s cloud architecture scales GraphQL observability with Kafka, ClickHouse & regional load balancing for enterprise reliability.

Scaling GraphQL Schema Usage to billions of requests per day

<article><p>We officially released the first version of <a href="https://github.com/wundergraph/cosmo/tree/main/aws-lambda-router">Cosmo Serverless GraphQL Federation Router</a> for the AWS Lambda Runtime. This marks a huge milestone for us and the GraphQL community. It is the first time that a vendor officially support AWS Lambda to run a GraphQL Federation Gateway. Let's break down what this means for you and your team. Before we dive into the details, let's first talk about what a GraphQL Federated Gateway is.</p><h2>What is a GraphQL Federation Router / Gateway?</h2><p>A GraphQL gateway, also known as a GraphQL router, acts as a single access point that aggregates data from multiple sources, orchestrating responses to GraphQL queries. It provides a unified GraphQL schema from different microservices, databases, or APIs, streamlining query processing and schema management. By routing requests to the appropriate services and merging the responses, a GraphQL gateway simplifies client-side querying, enabling seamless data retrieval from a variety of back-end services and systems. This architectural design enhances scalability, maintainability, and the ability to evolve API landscapes without impacting client applications.</p><p>In order to scale out a federated Graph, you need more than just a Gateway, though. You should have a Schema Registry, Analytics, Schema Usage Metrics, breaking change detection, and a lot more. For that, we've built Cosmo which is a complete platform to manage your federated Graph. Check out Cosmo on <a href="https://github.com/wundergraph/cosmo">GitHub</a> if you're interested, it's fully open-source.</p><h2>Why AWS Lambda is a good fit for a GraphQL Federation Router?</h2><p>AWS Lambda provides a serverless computing service, offering a hassle-free solution where concerns about server provisioning, scaling, and infrastructure management are handled by AWS. This allows you to concentrate solely on your business logic. Although it might sound like a jargon-filled proposition, the practical benefits for companies are substantial. Operating gateway services is inherently complex and challenging, but AWS Lambda simplifies this process. It stands out as the most popular and mature serverless computing service, available in all regions and featuring an exceptionally generous free tier. These attributes make AWS Lambda an ideal choice for our specific needs.</p><h3>AWS Lambda is Event-driven and requires a small footprint</h3><p>Due to the architecture of Lambda, we had to make our router event-driven. AWS Lambda functions are invoked by events. You can't simply run a http server. Instead, you have to register a handler function that is invoked by an event. This is a huge difference to traditional servers. Luckily, the community already built a lot of adapters to make this process as easy as possible. We decided to use the <a href="https://github.com/akrylysov/algnhsa">akrylysov/algnhsa</a> to handle the event and response handling for us. This allows us to make our router compatible with only a few lines of code <em>&quot;simplified&quot;</em>.</p><pre data-language="go">func main() {

    // Create a new router
    r := internal.NewRouter()

    // Create a new server
    svr, _ := r.NewServer(ctx)

    // Create an adapter for AWS Lambda
    lambdaHandler := algnhsa.New(svr.HttpServer().Handler, nil)

    // Start the Lambda handler
    lambda.Start(lambdaHandler)

}
</pre><p>As a side effect, our Router can now be integrated into the existing AWS ecosystem. For example, we can run a GraphQL mutation through an SQS or S3 event. This allows us to build a very flexible and scalable system.</p><p>That's it! Of course not, there is a lot more going on under the hood. We down-shrink the router binary to <code>19MB</code> to advance a small footprint and therefore fast cold starts. We also went a different route to report metrics and traces to our collectors. We will talk about this in the next section.</p><h2>OpenTelemetry support</h2><p>Our Router has built-in support for <a href="https://opentelemetry.io/">OpenTelemetry</a> for tracing and metrics. We also collect schema usage data of your GraphQL operations to give you insights into your schema usage. In that way, we can identify if a schema change is safe to deploy or not.</p><p>In the AWS Lambda handler, you can't do work after the response was sent. If you are thinking in servers, we looked for a way to do arbitrary work in the background and on <code>SIGTERM</code> <a href="https://github.com/aws/aws-lambda-go/issues/318">#Issue</a>. This does not fit into the serverless mindset. Maybe your function will never shutdown, so you have to find another way. We have to flush the metrics and traces synchronously after each request. This is tricky, because it adds latency to the response. It is much better than it sounds, because we optimized the flushing process to be as fast as possible. All metrics and traces are sent in parallel without write acknowledgement to the collectors and traces are sampled to reduce the overhead. This adds less than 75ms of latency to the response. This is a very small price to pay for the benefits you get.</p><h2>Costs / Performance</h2><p>AWS Lambda has a free tier of 1 million requests per month. If you exceed the free tier, you pay $2.28 per 1 million requests. We assume that each request takes 1 second to process. This plan uses the 128MB memory size model which is the cheapest and less powerful one. The CPU power scales proportionally with the memory size.</p><p>Our tests showed that the Router handles subsequent requests with this plan very well but at the cost of much higher cold start times. We recommend using the 1536MB memory size model. This is the sweet spot of good cold starts and low costs. We were able to wake up the Router and making a subgraph request in 641ms. The initialization time of the lambda function took just 300ms.</p><p>Once the Lambda function is initialized <em>warm</em>, it can handle subsequent requests in 80ms. In this scenario, we made a Subgraph request to the <a href="https://employees-api.fly.dev/">Employees API</a> which is hosted on fly.io in the US West (LAX) region. This is a very good result and shows that our Router performs very well on AWS Lambda.</p><h3>Benchmark</h3><p>Additionally, we tested the performance of the router with 100 concurrent users over a period of 60 seconds. We warmed the functions with 2 pre-runs. Due to simplicity, I didn't redeploy the router near my location. Instead, I used the same setup as above. The router was deployed in the US West (Oregon) region while I'm located in Germany. We will definitely see better results if we deploy the router and subgraphs near my location, but I think the results already speak for themselves.</p><p>The router was able to handle all requests with <code>p95</code> of 480ms. This is a very good result and shows that our router scales well on AWS Lambda.</p><pre data-language="bash">          /\      |‾‾| /‾‾/   /‾‾/
     /\  /  \     |  |/  /   /  /
    /  \/    \    |     (   /   ‾‾\
   /          \   |  |\  \ |  (‾)  |
  / __________ \  |__| \__\ \_____/ .io

  execution: local
     script: bench.js
     output: -

  scenarios: (100.00%) 1 scenario, 100 max VUs, 1m30s max duration (incl. graceful stop):
           * default: 100 looping VUs for 1m0s (gracefulStop: 30s)


     ✓ is status 200

     checks.........................: 100.00% ✓ 11716      ✗ 0
     data_received..................: 5.7 MB  95 kB/s
     data_sent......................: 2.0 MB  32 kB/s
     http_req_blocked...............: avg=895.92µs min=60ns     med=160ns    max=135.24ms p(90)=240ns    p(95)=270ns
     http_req_connecting............: avg=332.8µs  min=0s       med=0s       max=58.28ms  p(90)=0s       p(95)=0s
     http_req_duration..............: avg=212.63ms min=168.82ms med=179.9ms  max=653.23ms p(90)=190.66ms p(95)=480.48ms
       { expected_response:true }...: avg=212.63ms min=168.82ms med=179.9ms  max=653.23ms p(90)=190.66ms p(95)=480.48ms
     http_req_failed................: 0.00%   ✓ 0          ✗ 11716
     http_req_receiving.............: avg=66.29µs  min=5.46µs   med=18.83µs  max=1.15ms   p(90)=204.73µs p(95)=242.36µs
     http_req_sending...............: avg=23.64µs  min=10.05µs  med=22.88µs  max=195.85µs p(90)=31.2µs   p(95)=34.76µs
     http_req_tls_handshaking.......: avg=294.56µs min=0s       med=0s       max=48.97ms  p(90)=0s       p(95)=0s
     http_req_waiting...............: avg=212.54ms min=168.79ms med=179.81ms max=653.11ms p(90)=190.61ms p(95)=480.41ms
     http_reqs......................: 11716   193.397417/s
     iteration_duration.............: avg=514.09ms min=468.94ms med=480.47ms max=1.02s    p(90)=492.15ms p(95)=782.61ms
     iterations.....................: 11716   193.397417/s
     vus............................: 100     min=100      max=100
     vus_max........................: 100     min=100      max=100
</pre><p>I want to call out <a href="https://twitter.com/codetalkio">@codetalkio</a> who does a fantastic job with their <a href="https://github.com/codetalkio/serverless-federated-graphql">repository</a> to unmistify the landscape of supported GraphQL Federated Gateways on AWS Lambda. It gave us the inspiration to support AWS Lambda as a first-class citizen.</p><h2>Getting started</h2><p>As you might know, we love Open Source! You can find the source code of the Serverless Router on <a href="https://github.com/wundergraph/cosmo/tree/main/aws-lambda-router">GitHub</a>. We manage separate releases for the AWS Lambda router. You can find the releases <a href="https://github.com/wundergraph/cosmo/releases?q=aws-lambda-router&amp;expanded=true">here</a>.</p><p>You don't have to build the router yourself. You can download the latest release and follow the instructions below.</p><ol><li><p>Download the Lambda Router binary from the official <a href="https://github.com/wundergraph/cosmo/releases?q=aws-lambda-router&amp;expanded=true">Router Releases</a> page.</p></li><li><p>Create a .zip archive with the binary and the <code>router.json</code> file. You can download the latest <code>router.json</code> with <a href="https://cosmo-docs.wundergraph.com/cli/federated-graph/fetch"><code>wgc federated-graph fetch</code></a>. The .zip archive should look like this:</p><p><code>bash     .     └── myFunction.zip/         ├── bootstrap # Extracted from the Router release archive         └── router.json # Downloaded with `wgc federated-graph fetch`     </code></p></li><li><p>Deploy the .zip archive to AWS Lambda. You can use SAM CLI or the AWS console. Alternatively, you can use your IaC tool of choice.</p></li></ol><h2>Conclusion</h2><p>Using AWS Lambda, we made it ridiculously simple to deploy, operate and scale a GraphQL Federation Router / Gateway. We are very excited to see what you will build with it. We are looking forward to your feedback and contributions.</p><p>If you are a company and want to use our router in production, please <a href="https://wundergraph.com/contact/sales">reach out to us</a>. We are happy to help you with the setup process and provide you with support.</p></article>

graphql_federation_router_on_aws_lambda

Serverless GraphQL Federation Router for AWS Lambda

Deploy GraphQL Federation on AWS Lambda with Cosmo’s serverless router. Small, fast, OpenTelemetry-ready, and built for real-world performance.

<article><p>WunderGraph Hub is our new collaborative platform for designing, evolving, and shipping APIs together. It’s a design-first workspace that brings schema design, mocks, and workflows into one place.</p><p><a href="https://github.com/wundergraph/graphql-go-tools">graphql-go-tools</a> is the open source GraphQL engine powering WunderGraph. Some of our enterprise customers go even one step further and they use the engine directly to build their GraphQL-based applications: proxies, gateways, caches, routers, etc...</p><p>Recently, one of our customers came back to us with a surprising benchmark: running a mutation against their GraphQL servers directly would take around 1 minute, but if they routed it through their gateway (built on top <a href="https://github.com/wundergraph/graphql-go-tools">graphql-go-tools</a>), it would take around 70 minutes. That's a staggering 70x difference and something we instantly knew we needed to improve.</p><p><a href="https://github.com/wundergraph/graphql-go-tools">graphql-go-tools</a> is built with performance in mind and regularly benchmarked, so we're confident the main request pipeline and common use cases are very fast. This would suggest that our customer was hitting a pathological case which we hadn't yet accounted for.</p><h2>Narrowing the problem down</h2><p>The first thing we did was work closely with our customer to reproduce the problem on our end. It turns out they were running a huge bulk operation consisting of a mutation receiving an array with ~70,000 items as its input without using variables but using a variable definition instead. We created a slightly different test that took 26 minutes to use as the reference for this problem.</p><p>We immediately noticed a potential workaround: Data provided as part of the variables JSON instead of defining the 70k input objects in GraphQL syntax requires less processing by the gateway, allowing us to take faster code paths. We made a quick check in the test case, and we confirmed that moving the data to the variables got rid of the slow-down. We communicated this workaround to the customer, to unblock them as fast as possible, and we started working on the full solution.</p><h2>Our approach to balancing maintainability versus writing fast code</h2><p>Performance matters a lot to us, let's make that clear. Unfortunately, sometimes chasing the best absolute performance gets in the way of maintainability, which is also a very desirable feature because it allows us to move fast and deliver new features quickly.</p><p>So our philosophy at WunderGraph is to keep a fine balance between both of them. This means sometimes we don't initially write our code in the most performant way, instead we choose to make it reasonably fast while still being maintainable. This same maintainability also enables us to quickly debug problems and profile performance problems when they arise.</p><h2>Profile like a pro</h2><p>As you might have guessed, <a href="https://github.com/wundergraph/graphql-go-tools">graphql-go-tools</a> is written in <a href="https://go.dev">Go</a>. We chose Go not just because of its simplicity and the productivity the language enables, but also because of the excellent testing, debugging and profiling tools it comes with. While investigating this problem, the most valuable tool was the CPU profiler.</p><p>There are multiple ways to enable CPU profiling in Go applications, but for our use cases the one that we've found to be the most useful is exposing the profiles using an HTTP handler. Go makes this incredibly easy: all you have to do is import <code>net/http/pprof</code> and, if you're not using the default HTTP server, start it in your <code>main()</code> function via <code>go http.ListenAndServe(&quot;:6060&quot;, nil)</code>.</p><p>This exposes several profiling endpoints. For example, to start collecting a CPU profile, you can run:</p><pre data-language="sh">go tool pprof -http=: 'http://localhost:6060/debug/pprof/profile'
</pre><p>This starts collecting a CPU profile for 30 seconds and then opens a browser with all the information. This means you should start the profile and then run the operation that you want to measure while the profile is being collected. Additionally, you can collect shorter or longer profiles using a <code>seconds=number</code> parameter in the query string. For this case, we use a smaller test than the initial one that completed in ~7.5 seconds, to help us get faster results, so we collected profiles that lasted 10 seconds.</p><p>Profiles can be inspected in several ways, but we've found the Graph view (available from the <code>VIEW</code> menu) very valuable for getting a general overview. Following the red arrows allows us to quickly notice the code paths taking the most time in the profile and get an idea of what's going on.</p><p>If you'd like to follow along, we've made the image with the <a href="/images/blog/go_profiling/initial_profile_graph.png">full profile</a> available.</p><p>Following the red lines, we can see a lot of the time is spent in manipulating JSON-encoded data. As you might already know GraphQL uses JSON when transported over HTTP, which means a gateway needs to decode a JSON payload, perform some work on the data and then encode it back to JSON to send it upstream.</p><p>The two spots that got our attention were:</p><ul><li>Input value injection (<code>astnormalization (*inputFieldDefaultInjectionVisitor) processObjectOrListInput</code>) taking 61% of the time</li><li>Encoding data back to JSON (<code>ast (*Document) ValueToJSON</code>) taking 23% of the time</li></ul><h3>JSON encoding</h3><p>Our GraphQL engine performs several passes over the input data to normalize it. This is necessary to simplify further processing steps like validation, execution planning, etc... It's much easier to implement an execution planner that can assume that all the input data is following specific rules.</p><p>We started by working on JSON encoding because we were very confident we could gain some time there. The existing approach involved a recursive function creating intermediate JSON objects that were eventually put together into an object representing the whole document. For example, this is how we were handling objects:</p><pre data-language="go">func (d *Document) ValueToJSON(value Value) ([]byte, error) {
  switch value.Kind {
    // Other kinds omitted for clarity
    case ValueKindObject:
      out := []byte(&quot;{}&quot;)
      for i := len(d.ObjectValues[value.Ref].Refs) - 1; i &gt;= 0; i-- {
        ref := d.ObjectValues[value.Ref].Refs[i]
        fieldNameString := d.ObjectFieldNameString(ref)
        fieldValueBytes, err := d.ValueToJSON(d.ObjectFieldValue(ref))
        if err != nil {
          return nil, err
        }
        out, err = sjson.SetRawBytes(out, fieldNameString, fieldValueBytes)
        if err != nil {
          return nil, err
        }
      }
      return out, nil
  }
}
</pre><p>We start with an empty object and populate its fields using a library that manipulates JSON in place. This becomes problematic with big objects because it required a lot of reallocations and copying. Additionally, we were also converting field names to strings, unescaping their JSON representations and then encoding them back to bytes.</p><p>Initially, we thought of using Go maps and slices to build and in-memory representation and feeding that into the JSON serializer in a single pass to encode everything. This would have reduced the number of allocations and made things faster, but then we came up with an even better approach: we read these objects from a JSON payload sent by the client, and keep them in-memory in their escaped form, only decoding them as needed. This means that in order to build that intermediate representation in Go we would be decoding them and then feeding them back to the JSON serializer to escape them again. Instead, we instantiate a <code>bytes.Buffer</code>, we recursively walk the document, and we write the values directly to the buffer without ever decoding and re-encoding any JSON string, only writing necessary characters to reconstruct the objects and arrays.</p><p>We started by changing <code>ValueToJSON()</code> to call instantiate the <code>bytes.Buffer</code> and into a helper function which does the recursion:</p><pre data-language="go">
func (d *Document) ValueToJSON(value Value) ([]byte, error) {
	var buf bytes.Buffer
	if err := d.writeJSONValue(&amp;buf, value); err != nil {
		return nil, err
	}
	return buf.Bytes(), nil
}

</pre><p>And then we implemented a recursive function writing directly to the buffer:</p><pre data-language="go">func (d *Document) writeJSONValue(buf *bytes.Buffer, value Value) error {
  switch value.Kind {
    // Other kinds omitted for clarity
    case ValueKindObject:
    buf.WriteByte(literal.LBRACE_BYTE)
    for ii, ref := range d.ObjectValues[value.Ref].Refs {
      if ii &gt; 0 {
        buf.WriteByte(literal.COMMA_BYTE)
      }
      fieldNameBytes := d.ObjectFieldNameBytes(ref)
      buf.Write(quotes.WrapBytes(fieldNameBytes))
      buf.WriteByte(literal.COLON_BYTE)
      if err := d.writeJSONValue(buf, d.ObjectFieldValue(ref)); err != nil {
        return err
      }
    }
    buf.WriteByte(literal.RBRACE_BYTE)
  }
}
</pre><p>This avoids creating any temporary strings or any kind of intermediate JSON object, we write everything to the buffer in a single pass.</p><p>This made <code>ast (*Document) ValueToJSON</code> disappear entirely from the graph! We had to look at the text list with the top functions to find it. Its execution time had reduced from 1.48s to 9ms, which accounts for a 99.39% reduction.</p><h3>Input value injection</h3><p>After getting JSON encoding out of the way, we turned our sights to default input value injection. Injecting default values is the process of looking at the input data of an operation and injecting default values defined in the GraphQL Schema if no value was provided by the client. Here's an example of a GraphQL Schema that defines a default value:</p><pre data-language="graphql">type Query {
  hello(name: String = &quot;World&quot;): String!
}
</pre><p>If the client sends the following query:</p><pre data-language="graphql">query {
  hello
}
</pre><p>The GraphQL engine will inject the default value for the <code>name</code> argument, which is <code>&quot;World&quot;</code>, and the query will be executed as if the client had sent:</p><pre data-language="graphql">query {
  hello(name: &quot;World&quot;)
}
</pre><p>Interestingly, our test case didn't have to inject any values! It turns out when we wrote that code we made didn't benchmark it with arrays or objects as big as in this case.</p><p>Basically, we were iterating over each element in the input, assigning a variable to its current value, looking if we should override it with a default value and then writing it back naively. The problem was that we didn't check if the value had been actually overridden, performing a lot of unnecessary updates on the JSON payload we had received from the client. For example, this was one of the functions:</p><pre data-language="go">func (v *inputFieldDefaultInjectionVisitor) EnterVariableDefinition(ref int) {
  // Some code omitted for clarity
  // ...
  //
  variableVal, _, _, err := jsonparser.Get(v.operation.Input.Variables, v.variableName)
  // ...
  // More code omitted
  // ...
  //

  // Here v.processObjectOrListInput() might return variableVal unmodified or it
  // might return a new injected value
  newVal, err := v.processObjectOrListInput(typeRef, variableVal, v.operation)
  if err != nil {
    v.StopWithInternalErr(err)
    return
  }
  // This jsonparser.Set() call might be costly if the input variables are big and we
  // perform it unconditionally, even if the value has not changed.
  newVariables, err := jsonparser.Set(v.operation.Input.Variables, newVal, v.variableName)
  if err != nil {
    v.StopWithInternalErr(err)
    return
  }
  v.operation.Input.Variables = newVariables
}
</pre><p>We made this code slightly smarter by modifying <code>processObjectOrListInput()</code> and the functions it calls to return also whether the returned value is the same as it received or something different, allowing us to skip updating the JSON when the value hasn't changed:</p><pre data-language="go">func (v *inputFieldDefaultInjectionVisitor) EnterVariableDefinition(ref int) {
  // Some code omitted for clarity
  // ...
  //
	variableVal, _, _, err := jsonparser.Get(v.operation.Input.Variables, v.variableName)
  // ...
  // More code omitted
  // ...
  //

  newVal, err := v.processObjectOrListInput(typeRef, variableVal, v.operation)
  if err != nil {
    v.StopWithInternalErr(err)
    return
  }
  // Now we know whether v.processObjectOrListInput() returned a different value than
  // what it received
  newVal, replaced, err := v.processObjectOrListInput(typeRef, variableVal, v.operation)
  if err != nil {
    v.StopWithInternalErr(err)
    return
  }
  // If the value has not changed, we skip the potentially expensive jsonparser.Set() call
  if replaced {
    newVariables, err := jsonparser.Set(v.operation.Input.Variables, newVal, v.variableName)
    if err != nil {
      v.StopWithInternalErr(err)
      return
    }
    v.operation.Input.Variables = newVariables
  }
}
</pre><p>Once we updated our code to keep track of whether the value had changed and updated the payload only when necessary we saw <code>astnormalization (*inputFieldDefaultInjectionVisitor) processObjectOrListInput</code> go down from 4.30s to 10ms, or a 99.76% reduction.</p><h2>Profiling again</h2><p>Once we implemented those two optimizations, we ran the test again and measured its full execution time at 150ms, with 80ms accounting for sending the request upstream and awaiting the GraphQL server to send it back.</p><p>We also collected and examined a <a href="/images/blog/go_profiling/final_profile_graph.png">new profile</a> in which we determined that there was no more low-hanging fruit for us to optimize at this time.</p><h2>Notifying our customer</h2><p>Very happy with the results, we prepared a <a href="https://github.com/wundergraph/graphql-go-tools/pull/547">pull request</a>, merged it and got back to the customer in less than 48 hours to deliver the improvements.</p><p>They ran their test again, and it took 20 seconds, down from 26 minutes (or a 98.71% reduction). They thought it was so fast that they had to double-check the results to verify that all the data was sent back and forth!</p><h2>Surprisingly, the Operation is now faster when using the Gateway!</h2><p>What's even more interesting is that the operation is now faster when using the Gateway than when using the GraphQL server directly. How is that possible?</p><p>The answer is that we've built a highly optimized parser and normalization engine. Our gateway parses the GraphQL Operation and normalizes the input data out of the GraphQL Operation and rewrites it into a much smaller GraphQL Operation with the input data as part of the variables JSON. The GraphQL framework of the customer isn't optimized for this use case. By using the Gateway in front of their GraphQL server, we're able to &quot;move&quot; the input from the GraphQL Operation into the variables JSON, so the GraphQL server only has to parse a reasonably small GraphQL Operation, and JSON parsing libraries are quite fast and optimized compared to GraphQL parsing libraries.</p><h2>Conclusion</h2><p>At WunderGraph we like to use the best tools for the job, and we're incredibly confident that Go is a great choice for networked services. Not just because of the simplicity of the language, but also because of how fast we're able to iterate, test, debug and profile with its amazing built-in tools.</p></article>

78x_improvement_with_pprof.png

78x improvement with pprof

How we reduced the executing time of a huge GraphQL operation using Golang's profiling tools

Blog

MCP Scope Step-Up Authorization: From Implementation to Spec Contribution

Managing Permissions in Cosmo Just Got Easier with Groups

From Interview Task to Production Feature

Scaling GraphQL Schema Usage to billions of requests per day

Serverless GraphQL Federation Router for AWS Lambda

From 26 Minutes to 20 Seconds: Using pprof to optimize large GraphQL Operations in Go