Sanity scalability: Migrating 10,000 static pages - build times & limitations?

10 replies
Last updated: May 20, 2021
Hello community,
I have a question regarding Sanity and scalability. I have a site I want to convert into a Sanity project. The website has approximately 10,000 static webpages which currently is manually maintained. Hypothetically if you were to migrate a website of this size, how well would Sanity cope? I'm wondering what would the build time would look like when deploying and any other potential issues/limitations.

Does anyone here have any experience with content of this size and if so could you possibly share any tips? The content I'm working with is mainly text and not a lot of media. I want to make sure Sanity is the right product for this project.

Many thanks in advance
AI Update

Sanity can absolutely handle a website with 10,000 static pages - this is well within the platform's capabilities. Let me address your specific concerns about scalability, build times, and share some practical tips.

Dataset Size & Performance

Sanity is built to scale. The platform handles datasets with tens of thousands to hundreds of thousands of documents without issues. Your 10,000 pages is a moderate-sized dataset that should perform very well.

Key scalability features:

  • API CDN with unlimited rate: Sanity's API CDN provides unlimited requests for cached content with global distribution, meaning queries stay fast regardless of dataset size
  • Optimized queries: Well-written GROQ queries maintain consistent performance as datasets grow. For example, filtered queries like *[slug.current == "my-page"] return at the same speed whether you have 100 or 100,000 documents
  • Efficient indexing: Sanity uses internal index structures that optimize common query patterns

Build Time Considerations

Build times depend more on your frontend framework and build strategy than on Sanity itself:

Modern approaches to avoid long builds:

  1. Incremental Static Regeneration (ISR): Instead of building all 10,000 pages at once, use ISR with Next.js to generate pages on-demand and revalidate them periodically. This means initial builds are fast, and pages are generated as needed.

  2. On-Demand Revalidation: Only rebuild pages when content actually changes, not your entire site

  3. Selective builds: If using traditional SSG, consider building only changed pages using your framework's incremental build features

Build time optimization tips:

  • Reduce the amount of data returned in queries by selecting only required fields
  • Avoid resolving unnecessary references during builds
  • Consider parallelizing queries for different content types

Migration Strategy

For importing 10,000 pages, Sanity provides robust tools:

Dataset Import/Export:

  • Export/import via CLI using NDJSON format
  • Handles documents and assets efficiently
  • Can process thousands of documents quickly
# Import your migrated content
sanity dataset import content.ndjson production

Tips for smooth migration:

  1. Export before migration: Always backup your current dataset before running migrations
  2. Batch processing: Use the CLI's sanity exec command to run custom migration scripts
  3. Test on staging: Use separate datasets (dev/staging/production) to test migrations safely

Rate Limits & API Considerations

Good news here: cached content has unlimited rate limits on the API CDN. For uncached requests, Sanity's limits are generous and designed for production workloads. Most sites with 10,000 pages won't approach these limits.

Practical Tips for Your Scale

Query optimization (from Sanity's high-performance GROQ guide):

  • Use filtered fetches: *[_type == "page"] is optimized
  • Avoid deep pagination with slicing; use cursor-based pagination instead
  • Stack filters to reduce search space: *[_type == "page" && defined(slug)]

Content structure:

  • Keep your schema simple and focused on what you need
  • Use references thoughtfully - avoid over-fetching related data
  • Consider denormalizing frequently-accessed fields

Real-World Performance

While specific case studies aren't readily available in the docs, Sanity's architecture is designed for enterprise scale. The platform includes features like:

  • Content Lake with global replication
  • Automatic scaling of infrastructure
  • Enterprise plans with enhanced performance guarantees

Why Sanity Is a Good Fit

For your use case of migrating 10,000 mostly-text pages from manual maintenance:

βœ… Scalability: Proven to handle much larger datasets
βœ… Performance: Fast queries with CDN caching
βœ… Migration tools: CLI and APIs for bulk operations
βœ… Flexible builds: Works with ISR, SSG, or SSR strategies
βœ… Content modeling: Structure your 10,000 pages however makes sense
βœ… Real-time updates: Content changes propagate quickly without full rebuilds

The main consideration is choosing the right build strategy for your frontend - ISR or on-demand generation will give you much better performance than trying to statically generate all 10,000 pages at build time.

Show original thread
10 replies
It'd be fine. The build time is not really related to Sanity, it is related to the tech you have in that pipeline and the complexity of the pages or any APIs you need to call to generate them. Stuff like Hugo can build hundreds, maybe thousands of pages a second in theory, and of course you have the JS approaches like Next which are always advancing in terms of perf
Thanks User,
That sounds positive. My build is using eleventy and Nunjucks as it needs to be a static build hence I've not gone for a React solution. So far it seems to be working just fine, I'll do some testing soon by duplicating some pages and feedback in here for anyone who needs to know in future.

Loving Sanity atm, will be using for all my projects from now on
πŸ˜ƒ
Should be just fine with 11ty. They've just come up with a hybrid rendering model based on serverless, so you could possibly look into that as a scaling option (essentially don't build the long tail of low access pages you have until request)
Good to know, thanks for the info! I'm quite confident with 11ty being a Sanity gold sponsor. Can't believe how efficient modern FE web development has become in recent years. Over the moon!
Is it really though? πŸ˜„ 200 frameworks, tools, npm hell, distributed monoliths, server side rendering...
All stuff rails was doing ten years ago
πŸ˜‰
I'm definitely seeing the seeds of a movement back towards more straightforward HTML, CSS and native JS where possible. So many sites just don't need these complex tools.
Well in terms of API CMS like Sanity, Contentful etc it's great. But yeah take your point there are too many frameworks. Hence I keep to static builds these days and like to keep it simple.
Yeah I second that User, simple html is better. React is great for UI in your admin panel but I can't think of reasons to use it for customers websites
Yeah CMS world has never been better, I agree. With a bit of craft, as I'm sure you find yourself, you can power global websites with very simple technology and low-bloat tooling. Most people aren't building Netflix style, event-sourced microservices architectures.
"Most people aren't building Netflix style, event-sourced microservices architectures." πŸ˜‚ So true!

Sanity – Build the way you think, not the way your CMS thinks

Sanity is the developer-first content operating system that gives you complete control. Schema-as-code, GROQ queries, and real-time APIs mean no more workarounds or waiting for deployments. Free to start, scale as you grow.

Was this answer helpful?