Migrating the Sanity.io codebase from Flow to TypeScript

We recently migrated the @sanity-packages written with Flow to TypeScript. It was an interesting journey and we'd like to share some experiences about how we ended up (re)configuring our build pipeline, how we dealt with CSS modules and how we managed to keep focus on compatibility in order to lay a solid foundation for the future of the Sanity.io codebase.

TypeScript and Flow both add support for static typing to JavaScript. Static typing means that the compiler (or even the code editor) will tell you about potential errors in your program, as opposed to dynamic typing where type errors are checked at execution time.

A statically typed language enables you as a developer to encode expectations about the structure of data types, variables, function arguments and return values in your code. A compiler will then tell you when these expectations are broken, thus giving early feedback about potential errors.

For example, it enables you to declare the structure of the props a component is expecting, and if you try to use a React component in a way that doesn't match these expectations, you will get feedback a lot earlier than you would have otherwise, (e.g. immediately as you write the code instead of when running it).

Below is the kind of feedback an editor can give you instantly as you type. Without static typing you would be relying on catching this error while testing or running the application, and noticing that your Hello component didn't work as expected.

Not only does this mean you get earlier feedback when developing code, but it also leads to safer, more robust and less error prone code, and increased maintainability.

The first commit of what eventually became Sanity.io can be traced back to July 2014.

This doesn't mean that the oldest components haven't been modified since it was written. Quite the opposite. When we started out, this was the current state of Web development with React:

• You made React components using React.createClass(), and you used mixins to solve shared concerns between components.
• You compiled arrow functions and experimental JavaScript language features using the Traceur compiler (Babel wasn't even 6to5 yet).
• For bundling you used browserify and a multitude of different *ify packages (for example reactify to get JSX support).
• You needed Bluebird for Promise support.
• You used JSHint (and JSXHint) for code linting.
• Mocha, expect.js and sinon.js for testing.
• And you didn't write CSS (duh!), you wrote Sass.
• You wrote tiny packages which you worked on and published in isolation.

Staying ahead of tech has always been important to us, and over the years our codebase has been modernised gradually. A lot of great tools have eased this modernisation process. Most notably the codemod scripts made available by the React community have enabled us to migrate from React.createClass() to class components, etc.

As our codebase has been growing at a faster pace than our (still small) team of developers, we started experimenting with adding static typing to our codebase a couple of years ago. At the time, Flow, the static type checker developed by Facebook, seemed like a great choice because of its great support for React and seamless integration with our existing toolchain (Babel, ESLint, etc.), so we started using it for a few packages. It helped us a lot in thinking about types, but we also spent a considerable amount of time battling configuration and dealing with third party library typings.

Since then, TypeScript has matured a lot and even though Flow is still great and has also improved over the years, it now it feels like TypeScript has won. Also, our reasons for choosing Flow over TypeScript in the first place has been voided with the advent of Babel getting TypeScript support in v7, and with the recent TypeScript support in ESLint.

We now felt the time was ripe for getting rid of Flow.

Note: We're not getting into a deeper comparison of Flow vs TypeScript, as there are other and better resources for that.

Migrating from Flow to TypeScript was a bumpy journey, but a lot of great tools helped a lot, especially the flow-to-ts tool from Khan Academy. But even after running it, it took a lot of careful effort to make everything pass TypeScript's type checker. And then there were the realisations we had along the way...

We already had some TypeScript packages, for which we were using the TypeScript compiler. All our other packages were built using Babel.

We <3 Babel and have been using it since it was called 6to5. We have been using it to compile React/JSX, and it's enabled us to use new JavaScript language features before they were supported by every browser or runtime that we officially support. It also integrates well with ESLint, Webpack, and other tools we rely on, so getting rid of it would mean that we'd miss out on some of these features.

Still, maintaining two different build pipelines isn't exactly optimal, and Babel has had TypeScript support for a while already.

So to begin with we wanted to see if we could get rid of our existing TypeScript compile pipeline...

But no. Although Babel has TypeScript support, it only supports parsing TypeScript and compiling to plain JavaScript with type information omitted. A large part of the value of TypeScript comes from the ability to ship TypeScript Declaration files with your packages. Since we couldn't get that from Babel, getting rid of the TypeScript compiler was not an option.

Since TypeScript is a superset of JavaScript, all JavaScript code is also valid TypeScript. And the TypeScript compiler can be configured to both parse and do inference based type checking on JavaScript code. You can even generate type declarations from plain JavaScript. Here's a great writeup about that if you're interested.

Still, getting rid of Babel would mean getting rid of a lot of our existing tooling. A huge part of the reason is ESLint, which we still rely on for overall code quality and consistency.

Ok, so we're left with two compilers in any case. To minimise the maintenance burden, we then decided to use Babel for compiling all source code, while keeping the TypeScript compiler only for emitting type declaration files.

It was interesting to see how much more type errors TypeScript uncovered compared to Flow. The largest part of doing the migration was actually to fix semantic issues with our existing types.

As mentioned earlier, the scope now was to get rid of Flow and improve/harmonise our TypeScript build pipeline. This meant not spending time on improvement of existing types, or adding types to new packages. A problem during this process was to figure out what to do with our part: imports, which is a special kind of module id that our plugin system is built upon. Due to TypeScript's support for declaring export types of modules, it was pretty straightforward to add declarations for just the part imports used in a package by making a d.ts file. Here's an example.

To begin with, all the exports are defined as the any type. The fact that TypeScript gives you the power to do this is great as it removes roadblocks during migration and then allows you to gradually refine the type system over time.

We use CSS modules for the Sanity Studio application, which means a few of our @sanity-packages import css files, e.g. import styles from './Component.module.css'. Since TypeScript has no support for understanding the syntax and inferring the exported types from CSS files, the stop-gap measure was to make a TypeScript Declaration file for all files ending with .css. Thanks to the wildcard support, this was just a matter of adding a file with the following:

declare module '*.css' {
  const shim: any
  export default shim
} 

Although the above would make the TypeScript compiler happy when encountering CSS imports, it declares the export of all css files as any. However, another great discovery was typescript-plugin-css-modules. This is a plugin for the TypeScript Language Service, which is a separate compiler service optimised for incremental compilation and is typically what Code editors and IDEs use to provide immediate feedback.

This means that in editors that are using the TypeScript Language Service, you will still get type feedback when using CSS-modules:

Answer: Not much (yet)! The scope of this round of work was to improve our TypeScript pipeline and to get rid of Flow. No matter how tempting it was to fix the issues uncovered by TypeScript, it was important for us to stay clear of changing any semantics of the current code, and only work on type annotations. For this reason, there were several places we had to add // @ts-ignore comments to make the TypeScript compiler pass.

// eslint-disable-next-line @typescript-eslint/ban-ts-ignore
// @ts-ignore 

Writing this hurts, but it's justifiable in a transition period 🙈

These changes lay the foundation for gradually improving typings, adding static typing to existing packages. Eventually this work will surface to our users and come with several benefits:

• Static types will help us catch a lot more errors before shipping updates.
• It will enable our team of developers to ship code faster and more confidently.
• It will make it a lot faster to refactor code and do cross-cutting changes without accidentally breaking stuff.
• It will make our codebase more accessible to both new hires and external contributors.
• It will help us in knowing what constitutes our public API, and what the community relies on, and therefore reduce our potential for accidentally breaking stuff.

These will in time lead to Sanity.io becoming an even better product.

As we move forward we will slowly but steadily be working on improving our type system. There are some interesting challenges ahead of us, for example how we can utilise static typing for document types defined by a sanity schema, and how we can provide type safe integration points for developers customising and doing integration work with Sanity.

Note: Although we will probably write more TypeScript in the future, we will not require Sanity developers to learn TypeScript in order to write plugins or integrate. TypeScript is completely optional and plain JavaScript will always work.

Our work has just begun in this area. Our long term goal is to provide TypeScript definitions for all our packages, and gradually improve our type system over time.

*) The Sanity codebase consists of a lot more than what's found in the public and Open Source sanity-io/sanity repository. This blog post is focused only on the public Open Source monorepo where we keep most of the JavaScript packages that developers interact with, and and editors rely on when using the Sanity Studio.