GROQ Developer Update: New Versioning Scheme and Functions

Every day, Sanity executes on average 450 million GROQ queries from the Content Lake. Developers who try it tell us it’s become their preferred way to integrate content into their frontend apps. As one developer recently said on Twitter: “I don't want loads of cms-data-wrangling logic in my frontend or on my server. GROQ lets me shape the response via the query.”

At Sanity, we believe content should be treated as data. GROQ is an example of this. It lets you query any collection of JSON documents and filter them down to exactly what you need by their properties and value. And it lets you reshape and reform that data, using projections and functions. You can think of it as SQL, but for JSON documents. This saves you time, and it opens up opportunities for using your content in creative ways you didn’t anticipate at the start of your project. To learn more about GROQ and our rationale for its invention, check out this companion post.

Today, we’re reaffirming our commitment to supporting the open-source GROQ language specification for Sanity and other apps that implement it by formalizing a versioning scheme. Developers can now more confidently rely on support for the language and associated tooling as a critical aspect of Sanity.

We’re also introducing several new GROQ functions for arrays, strings, and mathematical expressions. All of these new functions are a direct result of developer feedback, and we look forward to seeing how you’ll use them.

Since GROQ is open source, we don’t want to tie its versioning to any Sanity-specific tooling. We also have a philosophy of having APIs that don’t break. We use semantic versioning for some of our tools, like groq-js, and date-based versioning for others, like the Content Lake API, and want to avoid confusion here.

In considering how we version the GROQ language specification, we’ve taken inspiration from other well-known language specs, like HTML and SQL. These languages use major version numbers to demarcate significant changes in functionality. We also want to be able to point to specific releases that introduce incremental changes; we’ve chosen to use revisions for this purpose.

We settled on the following format:

GROQ-<major version #>.revision<#>

The current version of the specification is GROQ-1.revision1. This version does not include any breaking changes.

Further non-breaking improvements will increment the revision number, whereas breaking changes will increment the major version number. Non-breaking changes are usually those that introduce new functionality, without major changes to syntax on existing functions. The revision number resets when the major version is incremented.

Adding new functions, like the ones described below, results in non-breaking changes since existing functionality isn’t impacted.

For example, the next release with a non-breaking change will be versioned GROQ-1.revision2.

The next release that introduces a breaking change will be versioned GROQ-2.revision0. Breaking changes will rarely be introduced, and only in the case of vital syntax changes that improve the experience of GROQ for all existing developers.

For example, if you construct a query requesting a field that doesn’t exist in a document or an array, GROQ currently returns null. This is behavior GROQ users might rely on. Changing this, so the field isn’t returned at all would be considered a breaking change, but one that could provide a cleaner user experience. Especially with JavaScript projects, since explicit null values bypass default parameters for undefined values.

With this release we have the pleasure of introducing new GROQ functions based on community feedback. There are three new namespaces for functions added to the specification, which have been implemented across all GROQ tooling. These are:

• array:: functions: Perform array operations on lists, such as removing all null values, building text strings from a list of names, or generating a list of all unique document _types.
• math:: functions: Run common mathematical operators on numeric values. For example, you can add the prices across multiple products or return the maximum discount available within a cart of products.
• string:: functions: Manipulate text or validate that information matches a given prefix. For example, get a list of the articles that start with “How to” or split a comma separated string of author names into an array.

Here’s a detailed overview of the functions. You can read our documentation for more in-depth details:

array::compact(<array>) - removes all null values from an array

// Listing the subtitles for all posts in the Content Lake, 
// removing unhelpful null values 
array::compact(*[_type == "post"].subtitle)

array::join(<array>, <token>) - concatenates all array elements into one string, separated by a specified token.

// Joining author names with commas into a simple string 
*[_type == "post"]{ 
  "authors": array::join(authors[]->name, ", ") 
}

array::unique(<array>) - removes duplicate values from an array (this works for values that can be compared for equality, specifically numbers, strings, booleans, and null, and will not work for values that are arrays or objects)

// Listing all unique types for all Portable Text fields 
// named `body` in the Content Lake
array::unique(*.body[]._type)

math::avg(<array-of-numbers>) - calculates the average value (arithmetic mean) of an array of numbers.

// Calculating the average price of items in 
// the 'Winter 2022' collection.
math::avg(
	*[_type == "collection" && name == "winter2022"].items[]->price
)

math::max(<array-of-numbers>) - returns the largest numeric value of an array of numbers.

// What is the most expensive item in the kitchen category?
math::max(
	*[_type == "item" && category == "kitchen"].price
)

math::min(<array-of-numbers>) - returns the smallest numeric value of an array of numbers.

// What is the least expensive item in the kitchen category?
math::min(
	*[_type == "item" && category == "kitchen"].price
)

math::sum(<array-of-numbers>) - calculates the sum of an array of numbers.

// Calculating the total price of every item 
// in the 'Winter 2022' collection.
math::sum(
	*[_type == "collection" && name == "winter2022"].items[]->price
)

string::split(<string>, <delimiter-token>) - turns a string into an array of substrings based on a delimiting token.

// Transforming a string of collaborator names into an array
*[_type == "journalArticle"] {
	contributors: string::split(collaborators, ", ")
}

string::startsWith(<string>, <string-pattern>) - checks if a prefix string exactly matches the start of another string.

// Retrieving a count of how many post titles start with "How to"
count(
	*[_type == "post" && string::startsWith(title, "How to")]
)

You can visit groq.dev to try GROQ out in your browser.

You can use the new GROQ features with your content on the Content Lake API version v2022-03-13 and v2021-10-21. You can also download the latest Vision plugin release (it’s Studio v3 Ready!) which lets you select the latest GROQ revision.

If you are new to Sanity and want to experience a modern CMS that treats content as data, get started here.