Thursday, May 23, 2024
HomeProgrammingMix, Sew or Merge any Datasource — SitePoint

Mix, Sew or Merge any Datasource — SitePoint

On this article, we’ll focus on the issue of the way to fetch knowledge from a number of sources whereas nonetheless maintaining our frontend snappy, and a possible answer: utilizing a GraphQL Gateway.

As software program engineers, we’ve all confronted the problem of mixing knowledge from many methods. Even a single web page requires knowledge from a number of companies to render.

Knowledge is in all places, from CRMs to monetary methods and SaaS platforms to databases. Each enterprise inevitably buys a plethora of SaaS platforms, then needs a unified enterprise view on high of all of them. We now have to embrace this head on and combine all the things.

A GraphQL gateway combines the advantages of a conventional API gateway with GraphQL.

We’ll begin by discussing the advantages of an API gateway, after which take a look at how GraphQL suits in. Stick round to the tip of the article, the place we take a look at some frameworks for constructing our personal API gateway.

Desk of Contents

Advantages of an API Gateway

Securing our public-facing APIs towards hackers is a full-time job. Over time, organizations have advanced to create many APIs, from SOA to microservices. As an alternative of placing these APIs immediately onto the Web, organizations favor so as to add an additional layer of safety that sits in entrance of all these APIs and ensures that entry to knowledge at all times follows the identical authentication guidelines.

They do that with an API gateway.

Merchandise reminiscent of Kong or Apigee expose inner APIs from a central location. They act as a reverse proxy with options like API key administration, fee limiting, and monitoring.

The API gateway permits us to regulate who and what has entry to every service, monitoring the connections and logging entry.

Extra not too long ago, apps have been required to mix knowledge from an API gateway and different exterior SaaS suppliers. Because of this the previous centralized software — which ensured our guidelines have been adopted — is now bypassed regularly.

Think about we’re constructing an online app for our firm. We now have a job to create the consumer profile web page. Through the login course of, we have to mix knowledge from many methods:

  • Salesforce CRM: holds the overall buyer knowledge reminiscent of first and final identify.
  • Orders: latest orders are in an ordering system inside the group.
  • Notifications Service: the notification settings and up to date messages are in an app-specific database linked to a Node.js service.

The consumer would want to make three separate requests to get the info, as represented within the picture under.

making three separate requests to get the data

Within the picture above, the online consumer sends three separate API requests after which has to mix the ends in frontend code. Sending a number of requests impacts the app’s efficiency, and mixing this knowledge will increase the complexity of the code. As well as, if there’s a couple of app, now all apps have to pay attention to all backends, and a single API change in a single service can lead to updates to all of our purposes.

We will do higher. Ideally, we wish to scale back the requests from three to a single fetch. We may create a brand new service to do this — a service that orchestrates requests to backend companies. This concept has a reputation: the BFF sample.

The Backend-for-frontend (BFF) structure sample permits a single request from the frontend.

However how does it work? Let’s take a look at this sample in additional element.

The Advantages of the BFF Sample

With the BFF sample, an utility sends a single request to the API gateway. The BFF service then requests knowledge from every backend service and combines it. Lastly, the info is filtered, returning solely the info wanted for the entrance finish, decreasing the info despatched over the wire.

Representation of the BFF pattern

As illustrated within the picture above, we’ve launched an additional layer into the stack to orchestrate the request.

The consumer profile endpoint returns the info wanted for that app on the profile web page. Lowering our three requests to a single request has mounted our earlier efficiency challenge.

However we aren’t completed but.

The enterprise has determined to launch a cell app. The cell app additionally has a profile web page, however this display screen shows a lot much less profile data.

At this level, the cell group has two choices. The group may reuse the online group’s endpoint, which might imply we over-fetch the info (fetching extra knowledge than is required for the cell app). The choice is that the cell group creates their very own BFF.

Not surprisingly, the cell group determine to create their very own BFF, as they need good efficiency for his or her app.

Diagram showing the interaction of the two BFFs

As illustrated within the picture above, issues are beginning to get difficult, and we now have two new points:

  • Every group has to create a brand new BFF service for every utility they create, slowing every group down and making it onerous to standardize.
  • Every BFF must be pen-tested to make sure it’s safe.

How will we remedy these points?

We want an answer the place every app can choose the info it wants, and it ought to be a single API utilized by all purposes within the firm.

As BFFs have matured, many builders have began experimenting with GraphQL as a substitute of REST.

Let’s take a look at how this know-how might help.

The Advantages of GraphQL for a BFF

GraphQL has many strengths that make it a super know-how for a BFF:

  • Environment friendly Knowledge Fetching. GraphQL allows purchasers to request precisely the info they want and nothing extra. This improves efficiency by decreasing the info that’s transferred from the API.
  • Single Endpoint. As an alternative of exposing many endpoints by way of the gateway, GraphQL makes use of a single endpoint for all requests. This simplifies the upkeep and the necessity to model.
  • Versatile queries. Purchasers can assemble queries by combining fields and relationships right into a single request. This empowers frontend builders to optimize knowledge fetching, growing efficiency. As well as, if the necessities for the frontend change, it may be up to date to fetch totally different knowledge with out altering the backend in any manner.

Frontends can now choose solely the info they want for every request.

We will now join each our apps to the identical GraphQL server, decreasing the necessity for a second BFF service.

Diagram showing how a GraphQL BFF works

We will now share the BFF for any app within the group. We even have a single endpoint that must be pen-tested.

However once more, we’ve launched a brand new drawback! We nonetheless need to handle two methods — the API gateway and the GraphQL BFF.

What if we mixed the 2 right into a GraphQL gateway?

Subsequent, let’s take a look at how a GraphQL gateway works.

What’s a GraphQL Gateway?

A GraphQL gateway combines an API gateway with a GraphQL API to get one of the best of each applied sciences.

Let’s recap the advantages:

  • Builders have a single API endpoint to request knowledge from. This reduces over- or under-fetching, as every utility selects solely the info it wants.
  • The GraphQL gateway is shared by many apps inside the group. This implies we have now lowered our safety publicity to a single API endpoint.
  • We solely have a single service to handle, now that the BFF is mixed with the gateway.

The diagram under exhibits how the consumer profile API request works with a GraphQL gateway.

how the user profile API request works with a GraphQL gateway

Within the picture above, the consumer sends a single request to the GraphQL gateway, requesting the info it wants. The gateway makes particular person requests to every service and combines the outcomes. We now solely have a single service to handle and deploy.

Hopefully, you’re prepared to do this for your self. Subsequent, let’s take a look at how we are able to construct a GraphQL gateway.

Constructing a GraphQL Gateway

When selecting a gateway framework, we wish to search for some key options:

  • A number of Sources. The gateway should hook up with many knowledge sources — from databases to SaaS — and we should always have the ability to create our connections.
  • Routing. The gateway ought to have the ability to request knowledge from the underlying companies immediately.
  • Batching. A number of queries to the identical service are despatched in a batch, decreasing the variety of requests.
  • Safety. Authentication and authorization ought to management who can entry the linked knowledge.
  • Cross Datasource Filtering. Highly effective filters ought to be out there to not over-fetch the info wanted by the consumer.
  • Extensible. Builders ought to have the ability to lengthen the code with middleware or features to suit their wants.

There are lots of frameworks to select from, however listed here are the highest three that I like to recommend exploring additional.


Hasura has gained recognition through the years, initially as a GraphQL-over-Postgres server. Nevertheless, it has added the power to hook up with exterior methods.

We will join a “Distant Schema”, which mixes GraphQL from different servers.

There are some downsides to this method. The primary is that we have to create and handle our distant schema in a separate service, and this service should be a GraphQL endpoint. This results in the second challenge: we are able to’t join the info supply immediately.

As well as, Hasura doesn’t permit us to filter knowledge in a single knowledge supply primarily based on the values in one other. This may sound educational, nevertheless it’s really fairly frequent that we wish to categorical one thing like, “Give me orders the place the client identify is ‘ABC’.”

This provides flexibility, however on the expense of working a number of companies. Let’s take a look at an possibility that can join immediately.


StepZen permits us to hook up with the info supply immediately from the GraphQL server. This reduces the necessity to run a number of companies to create a gateway.

To attach Stepzen to an information supply, we create a GraphQL schema file like this:

kind Question {
  something(message: String): JSON
  @relaxation (
    endpoint: ""
    technique: POST
    headers: [
      {name: "User-Agent", value: "StepZen"}
      {name: "X-Api-Key", value: "12345"}
    postbody: """
        "consumer": {
          "id": "1000",
          "identify": "The Consumer"

On this instance, we’re connecting the server to a database utilizing the customized schema.

There’s another choice that you could be favor, and that could be a code-only method. Let’s take a look at that subsequent.


For the previous few years, I’ve been engaged on an open-source product known as Graphweaver, which can be utilized as a GraphQL gateway.

It connects on to our knowledge sources and creates an instantaneous GraphQL API. This API consists of all of the CRUD operations we’d anticipate to create, learn, replace, and delete. It auto-generates filters, sorting, and pagination arguments, saving time. We will lengthen the built-in operations with our code for full flexibility.

Graphweaver has out-of-the-box knowledge connectors for databases reminiscent of Postgres and Mysql and SaaS suppliers like Xero and Contentful.

Making adjustments or connecting knowledge sources entails writing Typescript code, giving us full customization.

For those who’re occupied with creating your personal GraphQL API, I extremely suggest you check out the Graphweaver GitHub code.


On this article, we’ve checked out the way to change our present API gateway and BFF sample with a single GraphQL gateway.

We’ve checked out the advantages of an API gateway and why organizations use them. Model management, fee limiting and entry administration are a number of the causes.

We additionally appeared on the BFF sample and the way it orchestrates API requests for frontend apps.

Lastly, we checked out GraphQL and the way it is a helpful know-how for BFFs.

Finally, this led us to making a GraphQL gateway, and we checked out three choices for creating our personal: Hasura, StepZen, and the product I’ve been engaged on, Graphweaver.

I hope this text has satisfied you to attempt a GraphQL gateway of your personal, and if that’s the case, that you simply’ll think about giving Graphweaver a attempt.



Please enter your comment!
Please enter your name here

Most Popular

Recent Comments