GROQ specification compatibility, introduction of namespaces, and new functions

Content Lake is a full refactor of Sanity's backend and query engine. You can start using it by upgrading and specifying an API version for your client. Content Lake follows the GROQ specification and introduces both bug fixes and new features.

We have updated the documentation to reflect the newest version. This changelog contains the breaking changes and migration paths you need to take. You can take your time and stay on v1 for as long as you need, while you test and compare your queries against the new version. You can also specify the API version for only new queries to gradually move over. Let us know in the community if you have questions or need help.

The GROQ ^ operator now works correctly in all scopes.

This fixes the known issue where the ^ operator only worked in subqueries. In all other scopes, it returned the root of the current scope, instead of the parent scope.

*[_type=="person"]{
  name,
  someObj{
    name,
    // Old Behavior: "parentName" returns someObj.name
    // New Behavior: "parentName" returns root name value
    "parentName": ^.name
  }
}

GROQ now uses three-valued logic consistently:

• >, >=, <, <= returns null when the operands are of different types.
• &&, || handles null "as expected": null && true ⇒ null and null || true ⇒ true.

Note that == has changed slightly:

• It now always return either true or false (never null).
• You can compare against null and get the expected result: 123 == null ⇒ false and null == null ⇒ true.
• Comparisons between other types than strings/booleans/numbers always return false.

This also means that in works with null: null in foo will return true if foo is [null, 1, 2].

in null always returns null.

We have cleaned up the behavior for array traversal. For example, queries that contained multiple array traversals that don't work in v1, now work as expected in v2021-03-25 and onward.

*["link" in body[].markDefs[]._type]
// v1: No results
// v2021-03-25: An array of documents that has a link annotation in their "body" field

// Data:
[
  {
    "_type": "book",
    "authors": [
      { "names": ["MH", "Holm"] },
      { "names": ["Bob"] },
    ]
  }
]

// Query:
*[_type == "book"].authors[].names

// v1:
[null]

// v2021-03-25:
[
  [
    "MH",
    "Holm",
  ],
  [
    "Bob"
  ]
]

// You can also add an additional `[]` to flatten it completely:

// Query:
*[_type == "book"].authors[].names[]

// v2021-03-25:
[
  "MH",
  "Holm",
  "Bob"
]

If you have stored null values in your documents, these are no longer removed in projections.

You can now override attributes while using the spread operator (...).

// Data
[{"title": "A", "customTitle": "B", "_type": "post"}]

// Query:
*[_type == "post"]{..., "title": customTitle}

// Output from v1: 
[{"title": "A"}]

// Output from v2021-03-25:
[{"title": "B"}]

All numbers are now 64-bit floats. Previously we used 64-bit integer representation in certain contexts.

Previously ordering by strings would in some context use a "smart" numeric ordering instead of proper string comparison:

// Query:
["foo4", "foo12"]|order(@)

// v1:
["foo4", "foo12"]

// v2021-03-25:
["foo12", "foo4"]

// However, *|order(foo) has always (both before and after) used proper string comparison.

The is operator has been deprecated. You can use the equivalent comparison instead:

// v1:
*[_type is "post"]

// v2021-03-25:
*[_type == "post"]

Previously you could call functions without parentheses: *[length "foo" > 3]. This is now no longer possible and is a syntax error. You should use *[length(foo) > 3] instead.

You can use now() and identity() instead.

Whenever you apply the projection operator ({}) on a non-object then it returns null instead of executing the object expression.

This means that the following query *[foo == bar][0]{a} now correctly returns null if there were no results. Previously this returned {}.

Previously negative indexes were not respected: [_type == "article"][-1] was equivalent to [_type == "article"][1]. This has now been fixed and -1 returns the last document.

The following query now fails: *[_type == "bar" && @[""] == "bar"] since empty attribute keys are not allowed.

Passing | order(…) on non-arrays return a null value instead of converting it to an array.

defined(x) is now only false when x is null. Previously it was false for empty arrays and objects as well.

The pipe operator (|) used to work between nearly all access operators, filters, and slices (e.g., * | [filter], * | (foo), * | [2..4], and * | [2] were all permitted). The pipe operator remains required before the score() and order() pipe functions, and is optional before a projection, but using one before a filter or traversal syntax will throw an error.

Namespaces allow for a stronger grouping of functionality within the GROQ specification. They create dedicated spaces for global functions, as well as safer distinctions for specific implementations of GROQ. Learn more.

The score() function computes a _score for each document from how well the expression matches the document. The documents will also be sorted by score (from high to low) if no other order() function is specified. Note that all documents will be scored (even those that don't match it at all) so you typically want to add a limit. Read more about scoring and boosting.

The pt::text() function takes either a Portable Text block or an array of blocks and returns a string in which blocks are appended with a double newline character (\n\n). Text spans within a block are appended without space or newline. Read more about getting plain text from Portable Text.

The geo namespace contains a number of useful functions for creating and querying against locations in your data. Query for distance, intersections, and more.

It's now possible to filter and order efficiently on date times:

// Filter
*[_type == "post" && dateTime(publishedAt) > dateTime("2021-01-01T12:00:00")]

// Order
*[_type == "post"]|order(dateTime(publishedAt) desc)

If you have huge documents, but your queries have projections that filter away most of the data (e.g., *[_type == "post"]{title, description}) you may see performance improvements.

Expressions that use both && and || will now often be much faster (depending on how complicated they are).

*[
  _type == "post" &&
  slug == "hello" &&
  ("news" in tags || products[0].name == "Sanity")
]

// v1: Slow!
// v2021-03-25: Much faster!