Recommended way to separate local/dev/staging/prod data in Sanity.io

i might have asked before, but is there a recommended way to separate local/dev/staging/prod data?
obviously i want to keep the schemas the same and only update when necessary. but in the early days of the project, how do we do this? also with multiple devs? i don't want to be working on something that might change the schema and mess up my co-workers data

An initial thought is of course, source control (like git) when it comes to multiple devs working on the same file defining a schema etc., but when it comes to dataset integrity, unless you are on an enterprise plan there isn't a way to set custom access control rules for each dataset. That said, you can define your prod dataset as private as an example, and then set the staging/dev datasets as public for collaboration.
You may want to
look into roles as well, as a way to assign read-only access for example to certain people via a specific role.
Hopefully that is helpful, Jace!

thanks for the help, but i'm not sure it solved the issue we are having, and im not sure enterprise is the solution (nor do i think we should need to pay for enterprise while still on the ground floor of building/testing the technology for the site)
so ya i guess i just dont understand how to solve my problem perhaps i can ask again with more details:


Scenario:There are 3 people on my team. We have setup 1 project on Sanity with 1 dataset. All of us are set as admins. A lot of things are in flux, so things may change on a daily basis (ie changing a schema, updating the content). It's basically just a blog/marketing site, so the data/schema should be relatively simple.

But today we had the issue where one of us was updating the schema for the footer social media links. they added a field and removed a field. They need to publish the content so they can see changes and make the changes for the UI.... but what if the feature takes a day/week/month to get into main/dev branches for the rest of the team?

When the changes were made locally on my coworkers dev env, it changed the content on my local sanity env. and now it shows conflicts. I obviously don't want to remove the fields, but i also don't have the most up to date schema and wouldn't necessarily expect to get it for at least a day or more.

Furthermore, I'm concerned with syncing the data when moving to production and the process for the content team and devs to work as efficiently as possible.



Questions:1. free tier: only two datasets available. i guess paid tiers will have more?
2. Is there a process for keeping data in sync on local/dev/staging/prod?
3. Can data be moved from one dataset to another, eg content teams publishing page content on staging and exporting/importing that content to prod

again i appreciate the help , but maybe there's just some training we are missing or something.
Our team has decided to go ahead with sanity, so maybe it would be worthwhile to go ahead and get our financial team involved to get going on an enterprise license and perhaps some training for devs and content teams

I appreciate the deeper breakdown here to provide more clarity, and I do understand what you are looking to do. I reached out internally with our team to get a better answer for you and i'll follow up when I have it. Thanks for your patience!

thanks

So in general it is pretty difficult to accomplish this level of source control for lack of a better term for your schemas when they are being built and are in a severe state of flux as well. There are some other workflows and options though.
To be a bit more clear, changes to the schema or the content will never affect the other. Yes, you’ll see warnings in your studio if a field exists on your colleague’s local studio and they’ve added content to it, but the content is still “there” in your local studio, too (and all others). The only mistake a dev might then make in a case like this (with multiple different studio schemas) is acting on those warnings, which
will change the content lake.
A solution for what you are looking to do may come more from a better workflow/understanding of what “best practices” are and how the schema and the content are linked within the content lake. For example, one approach might be to have the devs working on the project export the production dataset each morning and import it to their own, working on that from their local dev studios. That said, if the ask is to simultaneously develop a studio that’s also in production, all without showing any schema warnings, I don’t think that’s possible. In general it shouldn’t be too tough to script something up that deletes the dev dataset, exports production, imports production into dev, and then lets your team go to work developing. That may be a way to create your own workflow that acts almost as source control for this problem.

In addition to this, I can loop in the business development team to reach out to you but they won’t be able to do so until after the weekend unfortunately. This way you could talk about possible other options that an enterprise plan may support (they may have other options available or in mind than I have mentioned here).

Hopefully that helps, and just let me know and I will ping someone to reach out next week to follow up as well

For example, one approach might be to have the devs working on the project export the production dataset each morning and import it to their own, working on that from their local dev studios.

By this, do you mean, they would connect to a different projectID. just each dev would setup a separate project, and like you said, export from

prod

or

staging

etc to get in sync
If that's what you meant, ya i think that's probably the best way. Everyone will setup a project themselves, and we can just have a

production

project that will be copied from whenever we need to sync up.
I think just laying it out like you did helped me get my head around it. Obviously we're new to using sanity, so figuring out this things early on is best.

I don't think there is a need to talk to business team just yet. This workflow should work for now until we are ready to start moving to production.

Thanks

Excellent! I'm glad that could help give you some more clarity, but I to clarify a bit more I mean you can use one project (and only one projectID as well) and all the devs would still connect to that same projectID, but the dataset itself would be different.
So besides the ones you may have set up like

production

staging

development

etc, you would have possibly

developerNumberOneDataset

and then the first dev would be using that as their "local" dataset that they could import from

prod

or

staging

or

development

etc each day to stay up to date (almost serving as a "feature branch" so to speak for the dataset they are modifying), and then they would export those changes back to the shared dataset like

staging

or

prod

and others would be able to then import the new changes into their "local" dataset. Each dev could have their own while all being on the same project. Hopefully that was clear? Here are the list of data set commands and there you can set them to private or public, you can import or create new datasets and then export etc.

ok, so then we just need to get off the free tier then? only 2 data sets
how many data sets for the paid tiers?

nvm i found it :
team: 4
business: 6
enterprise: ???
phase 3: profit

Ahh, yeah you may be able to use only a few datasets to accomplish something similar if you are trying to stay within the bounds of the free plan, if you want to just use

staging

and

prod

as your two datasets (and the changes in staging would be more along the lines of your daily changes and prod is your true source of truth for your data). In this way you would run into some conflicts while all trying to work on the same data possibly within the

staging

dataset but it wouldn't cause production issues as originally described.
I would defer to our sales team around specifics for plans and pricing etc and have them follow up but from what I can tell from our docs you could have 4 datasets on the Team plan which is the next tier up from the free plan.

so this is what you mean, get a team plan. 1 project - 4 datasets:

prod

dev1

dev2

staging

That would be an example yes

super duper, thanks

see you monday!!!! im out 😄

You got it Jace, have a great weekend!

Hey Jace, just wanted to follow up after the weekend and giving this some more thought/talking it over with some more team members. Although, as we talked about already, you do have other options I strongly believe that your best options would be to use env based datasets as opposed to feature branch named datasets:

production , staging, dev, test

vs
This will end up serving as a much better long term stability choice and it may already be what you had in mind after our conversation but wanted to add a bit more of my assertive opinion here and be a bit more prescriptive of what will probably be best for your team.

That said, as mentioned, you could do this similarly with only 2 datasets and stay on the free plan (

production

and

staging

for example) but if you want additionally dataset options the Team plan would be needed obviously. I believe you will want to have a bit more control, so adding in some sort of internally organized project management workflow attached to this would be best.
Hope this helps and doesn't overcomplicate but did just want to follow up

Hey just want to clarify something you said earlier...

To be a bit more clear, changes to the schema or the content will never affect the other. Yes, you’ll see warnings in your studio if a field exists on your colleague’s local studio and they’ve added content to it, but the content is still “there” in your local studio, too (and all others).

where is that content stored? say i made a couple documents locally that i want to add/share to a different studio?
is there a way to just copy/paste a couple documents, or maybe just some field data?

i guess basically just want to know if there is a mechanism to export and import a couple documents/fields at a time?