Ad hoc document migration

Sanity's dataset commands (import and export) are useful, but don't give you granularity beyond document types. In cases where you only want to migrate one or a few documents between datasets, this command may be worth considering. As always, I would recommend testing this on a non-production dataset first. At the very least, run your query in Vision to ensure you're getting the data you want.

There is a feature in most capable shells called process substitution. In the iteration we'll be using here, it's used where a command is expecting a file (sanity dataset import wants a file) and lets you provide standard input instead. Here's a templated version of the command we might run:

sanity dataset import <(sanity documents query --dataset <SOURCE_DATASET> --api-version v2024-05-30 "<GROQ_QUERY>" | jq -c ".[]") <TARGET_DATASET>

Take note of <( and ). That surrounds the command we'll run to "get" our JSON data, and is the syntax that sets out the command whose output should be substituted inline. If we replace that whole segment with a filename, we get the following, which should look familiar to anyone who's run the import command before:

sanity dataset import myFile.ndjson <TARGET_DATASET>

With that in mind, let's instead look at the command inside the substitution syntax:

sanity documents query --dataset <SOURCE_DATASET> --api-version v2024-05-30 "<GROQ_QUERY>" | jq -c ".[]"

Let's consider this in two parts—separated by the pipe.

sanity documents query --dataset <SOURCE_DATASET> --api-version v2024-05-30 "<GROQ_QUERY>"

This is a command built into Sanity's CLI. We need to specify the dataset explicitly, since the goal is to migrate between datasets (there's also a project flag, which might be used to extend this beyond migrations between just datasets). We also want to specify the API version, since that's a best practice (though as of this writing, I think the command defaults to use v2022-06-01, which should be fine here). The <GROQ_QUERY> is whatever you want, really. What we want from this command is to return data in a way that we could essentially import it into Sanity. See to the side for a few examples.

The second part is to pipe (|) that output into jq, which is the best command line tool you probably don't have installed (if you do, I want to hear your best use cases).

jq -c ".[]"

The -c flag is returning compact output. I don't know if that's synonymous with NDJSON, which is what we want, but it worked in all my test cases. Send any complaints to my Hotmail address.

".[]" is jq filter syntax to index into an array. This is necessary based on how we've written the rest of the command, but can be modified as needed if you get an error.

That's it! This command isn't for everyone, but may be what you need to do that ad hoc migration where the normal CLI commands are too heavy-handed.

Note that this will not work to migrate assets. It also doesn't work as written when a document contains references (whether to an asset or another document). Consider the flags available on Sanity's import and export commands for ways to manage that.

Note also that this requires a shell like bash or zsh. Mac and Linux users should be fine, but I don't know what you can expect on the native command line in Windows.