Transition to Yarn Workspaces

[leeyi45]

Transition to Yarn Workspaces

Yarn has supported Workspaces since Yarn 1, but it has matured over later versions and we can now use it since we have moved to using Yarn 4.

The main motivations for doing this are:

• Reduced install size. Yarn workspaces support focused installs, which means that a single team working on a single bundle or tab won't have to install the entire repository's worth of packages that won't be relevant to them
• Better Compartmentalization. There is a significant amount of code that functioned like a common library for bundles and tabs that didn't really have a proper place to live. Workspaces allow us to separate this code neatly.
• Dependency Management. Previously, if tabs relied on a specific bundle, it was down to the build scripts to determine if the tabs needed rebuilding when the bundle was rebuilt. This is more complicated than expected. Instead, workspaces allow us to rely on Yarn's dependency resolution features to determine what needs to be rebuilt and when.
• Automatic Worktree Detection. If so desired, Yarn automatically detects which bundles/tabs have changed with respect to the main git branch and won't rebuild those that haven't. This responsibility previously fell onto the build scripts, but I think Yarn's implementation is far more robust.
• More Configurability: Each bundle/tab can maintain its own set of dependencies now, separate from other bundles. This allows us to have patches local to a specific bundle (like the @jscad/modeling patch for csg). Also each bundle/tab can customize its own tsconfig etc..

Migration to vitest

This repository was migrated to vitest, which has been designed as a drop in replacement for jest. I find that getting jest to work with the wide mix of ESM and CJS syntaxes has been a major headache, and configuration for jest has always been needlessly complicated.

vitest supports TypeScript and ESM out of the box and has generally provided a far smoother experience, especially when dealing with dependencies that need to be tested (like typedoc).

Why I hate Jest

Basically some scripts relied on __dirname, which is only available in CJS, and so would not work if compiled to ESM. But the scripts also used top level await, so compiling to CJS wasn't an option. Thus, import.meta.dirname would have to be used instead, but that meant that jest couldn't transform these files unless it was run in its experimental ESM mode, which would then cause many of the CJS bundled dependencies to break because they don't use imports with extensions.

The alternatives would have been significantly bulkier jest configurations or major changes to the code, and even after trying both of those I still ended up with errors because jest wouldn't properly transform dependencies.

Also because jest insists on transforming to CJS, we can't use verbatimModuleSyntax: true in tsconfig, instead requiring some weird configuration as a workaround.

Upgrading Module Manifests

This PR is the beginning of the steps required to address #57. Bundles will now be able to specify a requires field in their manifest that specifies the minimum version of Source that the bundle can be used with.

A bundle manifest can now look like:

{
  "tabs": [],
  "requires": 1
}

Each bundle specifies its own manifest, which is then collated by the build tools automatically.

Bundles will also be able to specify versions to help work toward resolving #79. The version field in package.json gets merged with the manifest during compilation.

Vitepress Docs Server

The Modules Repo's wiki has been quite the disorganized mess (oops). While I was thinking about how to go about documenting the changes introduced by this repository I realized that we were probably going to need something more serious than the Github Wiki.

So I've begun porting the Wiki over to this new Vitepress server (found in ./docs) whilst trying to update any outdated documentation.

[leeyi45]

There are still many tasks left to be done, this is far from complete, but I thought I'd open this now for feedback.

I'm especially concerned that the structure of the repository looks a lot more complex now, so I'm wondering if there's any need for simplifying things.

[leeyi45]

RFC:

Effectively the modules repository has three kinds of documentation:

• Bundle Docs for Users (example)
• Bundle Docs for Developers (example)
• Repo Docs for Developers (the various READMEs in ./scripts that are being migrated)

Should we unify all of these under one documentation server? I'm not even sure how we would go about doing that.

I have also been experimenting with using Typedoc to generate Markdown that can be used with Vitepress to document things like our common React components library (which probably will need to become its own fully fledged thing in the future).