It’s a big decision
At takeshape, we are very focused on developer productivity. We are a small team with limited resources, so it’s worth taking the time to think about how to work together faster and more efficiently. When we recently refactored our build process, we made a big decision: we will abandon yarn and use pnpm to manage our dependencies and run our scripts. It’s a story of how we made that decision and how it has benefited us so far.
Initially, takeshape’s code base was scattered across git repositories. Each package is developed independently and depends on each other. In theory, this is the ideal setting. In practice, we find that everything is interdependent, and we really want to be able to test and release all packages at the same time. When we release a new version of one of the packages, we fail, but forget to update it in other projects that depend on it.
In the end, we realized that monorepo was the right trade-off when it came to keeping projects separate and dependent. All of our software packages (such as web client, front-end routing library and CLI) exist in a testable and deployable unit. Our bags can be used
link: syntax interdependence.
It works to a large extent, but we still find that managing our
monorepoThe part of it is boring. It’s partly because of us
monorepoEach package in manages its own dependencies with its own package manager,
jsonAnd its own
lockDocuments. Even if each package uses the same development tool chain, such as eslint, jet, typescript, and Babel, each package declares these separately
devdependencyThe toolchain must be up to date in all our packages. We decided not to use it
YarnTo solve this problem, we need to discard the workspace feature of each package
lockFile, while using a single workspace wide
Avoiding phantom dependence is also trickier than it really needs to be. When your code import is not in
package.json A phantom dependency is generated when a package is declared in. Suppose you will
Package A Add to dependent on
Package B In the project. because
Yarn Leave all packages in the
node_modules So you can import and use it
Package B Without having to put it all in the
package.json In the middle. Although not very common, this is a mistake, and it does slow down the debugging process unless you remember to check it intentionally.
monorepoIt also makes our CI pipeline more complex than needed. First, we parallelized it
CircleCI Build to speed up slow
Webpack Build. However, with the development of our country
monorepoThe cost of installing dependencies separately for each build is also increasing. Installing dependencies for each build becomes a bottleneck. In response, we use the
CircleCIScripts consolidate the build process to reduce usage. Finally, we get a set of fragile CI scripts to tailor, test, and build changes to any package.
What we really need is a way to recursively install packages, elevate all shared dependencies, and run scripts for lint, test, and build. We know that yarn can’t expand for us, so we started to consider alternatives:
- What if we keep yarn and add lerna? Adding lerna can solve some problems in CI scripting, but it can’t solve the problems caused by phantom dependencies or duplicate devdependencies.
- What if we add lerna and use yarn workspaces? The workspace solves the problem of sharing developer dependencies and automatically linking them. But all nodes_ Upgrading modules to the project root will only aggravate the risk of fantasy dependencies and cause problems with some modules and tools.
- If we upgrade to yarn 2.0 and use Other What should we do? Yarn 2.0 is really exciting. It has many cool features, including plug’n’play (PNP). PNP can solve the problem of fantasy dependencies, but it may not be compatible with some dependencies that require file access. Yarn 2.0 is not compatible with lerna; instead, it has a plug-in architecture. These plug-ins are likely to solve our need for CI scripts, but they are not mature enough to be used confidently in production.
- What if we replace yarn with pnpm? according to
pnpmIt has the saying that “[use] hard links and symbolic links to save only one version of the module on disk at a time.”. This kind of organization
node_modulesThis method can prevent phantom dependence and avoid other problems in hoisting. It can solve the same problem as ARN 2.0’s PNP, but because it only uses links, it has wider compatibility. Pnpm also contains filtering function similar to lerna.
Finally, pnpm means the most to us. We found that pnpm’s recursive command and the — filter flag eliminated our need for a separate package like lerna. By seamlessly moving shared developer dependencies like Babel, eslint, and jest to the top level of our project, we can now update these packages from a single shared source. During the switching process, we even found and fixed multiple phantom dependencies in our project!
After using pnpm, our complex circleci pipeline is simplified to one operation. As a result, we reduced the total chargeable circleci minutes by about half! We think we can get more benefits by adjusting the settings, but it’s a good start.
Of course, every decision needs to weigh the pros and cons, and pnpm is no exception. Although pnpm is actively maintained by zkochan, it is a less popular project than yarn or NPM. Although Microsoft uses pnpm, it is not as high as yarn’s direct corporate sponsorship from Facebook. And pnpm have their own lockfile format, so they are not directly compatible with yarn or NPM. It’s hard for us to know what will happen in the future, but if we need to get out of pnpm, it will be a bigger task than we hope.
As silly as it may sound, yarn doesn’t need the “run” command of a custom script, which spoils us, so we have to retrain our muscle memory. It’s a small compromise, but it does increase the cost of our team switching such a basic daily workflow.
But at the end of the day, these costs far outweigh the improvements pnpm has made to our stack. Now we can work faster and more effectively than ever, and at a lower cost.
Pnpm may not be the right tool for every project or stack, but if you encounter any of the same problems as our monorepo, please look at it and consider it as an alternative.