Partial caching with defer

When you're developing an API that is expensive to call or slow to return data (or both), it can be very useful to put a cache in front of that API. The cache serves the request quickly and without incurring as much of a cost, and everyone is happy. That's the promise of Grafbase edge caching.

Until now caching has been all or nothing: either the entire request is served from the cache, or the entire request runs. This covers some cases well, but if you're working with a mix of very cacheable data and uncacheable data it can be problematic. If you're not careful to split your queries into those that are cache friendly and those that aren't then you run the risk of never hitting the cache.

That's where partial caching comes in: you can explain the caching properties of your data to Grafbase, send queries without thinking about them, and we'll make sure to fetch as much from the cache as possible, while still requesting fresh data from your data sources where neccesary.

Lets run through a simple example. Say you were developing a travel website, and your API returns some recommendations and some deals to users. For the sake of this example you always want the deals to be served fresh, but recommendations are expensive to calculate so you'd like those to be cached.

To do this you can set up your schema like this:

import { config, graph } from '@grafbase/sdk'

const g = graph.Standalone()

g.query('recommendations', {
  resolver: 'fetchRecommendations',
  returns: g.ref('Recommendations').list(),
  cache: {
    maxAge: 360,
  },
})

g.query('deals', {
  resolver: 'fetchRecommendations',
  returns: g.ref('Deal').list(),
})

export default config({
  graph: g,
  experimental: {
    partialCaching: true,
  },
})

Here we've configured caching with a maxAge of 6 hours for the recommendations field and no caching at all on the deals field.

Now when a user makes a query like:

query {
  recommendations {
    name
    price
    score
  }
  deals {
    name
    price
  }
}

Grafbase will split this up behind the scenes into two different queries.

The cacheable fields:

query {
  recommendations {
    name
    price
    score
  }
}

and the non-cacheable fields:

query {
  deals {
    name
    price
  }
}

If the cacheable fields are found in the cache then we'll use those, if not they'll be added back into the query with the non-cacheable fields and resolved as normal.

Partial caching is a great way to speed things up when you get a cache hit, but responses are still returned to the client after everything has finished resolving - so a cache miss on a slow field will still return a slow response, even if some of the response can be served from the cache.

The solution to this is the @defer directive, which we've integrated into partial caching. The request above could instead be made as:

query {
  ... @defer {
    recommendations {
      name
      price
      score
    }
  }
  deals {
    name
    price
  }
}

With this request, if recommendations is not present in the cache we'll return deals immediately and resolve recommendations in the background. If recommendations is in the cache we'll return it immediately as part of the initial response, the same as you'd see if you hadn't included the defer directive.

This feature is still experimental and being actively worked on, so there's some limitations and edge cases that it doesn't quite handle yet:

1. Currently @defer is only supported on the edge runtime, and not on the Node.js runtime.
2. We don't currently support cache purging for these cache entries.
3. Stale while revalidate support is not there yet.
4. Caching parts of items inside lists will behave badly if the order of the list in the cache and the order returned from the API does not match. For now we'd recommend caching all or none of list items.

We'll be iterating on this feature over the coming weeks, so if any of these features are important to you reach out and we can adjust priories as appropriate.

We'd love to hear your feedback and ideas, so join us on Discord.