How to improve your markdown-based docs with automatic tagging

We've been working hard recently to rebuild our entire landing page and the documentation website . I've realised that I kept adding internal links to the glossary when I was using WunderGraph specific terms. At some point I realised that there's actually a better technique to do this, it's called tagging!

So far, the results look promising. Bounce rate is down 4%, time on page almost doubled from 2min 11s to 4min 11s. The Number of pages visited per unique visitor is up from 5.04 to 8.12, an improvement of 61%.

Tagging is a simple technique to turn specific words or phrases into links. I'm super lazy, so after the first couple of links I've manually added, I thought about doing this in a more automated way.

Initially, I thought I was missing just a few links here and there. But after applying this technique, I found that I was missing links in almost all places. So, doing this manually is really not an option and would also be very time-consuming. Not to mention that you'd have to change links in all places if you'd like to change a URL.

I've first learned about tagging when I was working in the news industry a few years ago. It's common to automatically apply links to specific keywords to keep the reader on the website for some more time.

With docs, my goal is not to keep the reader on the website for too long. What I'd like to achieve is being able to write about a specific topic without having to explain every concept from scratch. This means, if I can apply automatic tagging, I can be sure that the reader will be able to find the relevant information in the docs.

Let's have a look at an example, the docs page on API Composition & Integration .

This page explains how you can use WunderGraph to compose and integrate APIs. One important concept to support this is to understand the concept of namespacing. Simply put, namespacing is a way to compose multiple APIs into a Virtual Graph without naming collisions. Now I have to explain Virtual Graph, and you know where this is going...

But with tagging, you can see that both namespacing and Virtual Graph automatically links to the relevant page.

What I didn't expect when I wrote this page initially was that each of the supported Data Sources, such as Apollo Federation, OpenAPI, PostgreSQL etc. automatically link to the page, explaining how these Data Sources work.

Luckily, our docs are part of our monorepo which is already open source, so I can easily share all the code with you. We've recently made the decision to open source our docs and add them to our monorepo. This means, we're able to use the same PR to add a new feature and document it right away.

So, how does it work? As mentioned earlier, we're using markdoc.io to convert markdoc files into HTML, but this technique might also work with other frameworks.

We're using very simple string matching, so don't expect too much. It's a good enough solution for the moment. We'll discuss later how to improve it.

That said, there's something important to note here. As you can see, the first tag Getting Started Guide is actually enclosing the second one (Getting Started). We're using simple string matching, so you have to make sure that if patterns overlap, the longer one should be higher up in the list, otherwise the AST will already be rewritten before the longer tag could match.

We've done the same thing with the CLI tag. The tag WunderGraph CLI needs to be before the tag CLI.

Once our tags are defined, we need to transform the markdown content. The code is annotated with comments to help you understand it.

We're using the transform api of markdoc . Before an AST node is passed to the render function, we're able to use the transform function to modify the AST.

What's left is to wire up the transform function and pass it to the markdoc config. We're using Next.js with markdoc, which looks automatically into the markdoc directory and picks up your config.

You can find the final solution here . If you want to try it out, clone the repo and run make and then make docs.

After implementing this, I've noticed a few things that could be improved.

One some pages, there's simply too many links, and they are sometimes repeating themselves. E.g. the 7th link about the Virutal Graph is not as helpful as the first one. So ideally, we could limit the number of links to a reasonable number.

Next, the general amount of links is sometimes too much. So, in addition to limiting the number of links to the same page, there should also be a way to limit the link frequency to not overwhelm the user.

Finally, it doesn't really make sense to link to namespacing, when you're already on the namespacing page.

How could this be improved?

Currently, we're simply string matching the tags, but we could actually pass the whole AST to the transform function and do some kind of analysis on it.

There's a question of what is good enough. I think we're already quite ok.

The initial goal was to save myself time from manually adding links to the docs. This goal was achieved, and I hope it helps some readers to better understand the concepts of WunderGraph.

I hope you've found this useful. Feel free to add this technique to your own docs to improve them. You can also just steal our docs from the monorepo. Maybe consider using some different styles. ;-)

By the way, if you're interested in working with people who care about open source and awesome docs, please join our Discord and leave a note. We're looking for Developers experienced with TypeScript, Golang and GraphQL, but our main focus is attitude and cultural fit.