Next major for the ecosystem

[wooorm]

Soon (April), it’s time to switch to ESM (without CJS backup) for all packages in the ecosystem.
That’s a lot of projects switching over, and a round of majors that bubbles from the bottom through to the top.

I’ve had some experience with that recently in a push in micromark (with CJS fallback, causing issues). My recent own projects dioscuri and xdm are fully ESM. It’s going to be a lot of fun for the ecosystem 😅

Otherwise, here are some changes slate for this next bubbling:

Big breaking changes:

• archive and deprecate remark plugins that (should) have rehype equivalents, and add some docs somewhere on whether folks should use remark or rehype plugins (currently quite confusing)
  • remarkjs/remark-external-links
  • remarkjs/remark-unwrap-images
  • remarkjs/remark-highlight.js
  • remarkjs/remark-react
  • remarkjs/remark-autolink-headings
  • remark-html-katex in remarkjs/remark-math
• archive and deprecate projects w/ very few downloads:
  • remarkjs/remark-midas
  • remarkjs/remark-defsplit
  • retextjs/retext-sentiment
  • vfile/vfile-reporter-folder-json
  • remarkjs/grunt-remark
  • syntax-tree/unist-builder-blueprint
  • syntax-tree/unist-builder-blueprint-cli
  • syntax-tree/hast-util-to-snabbdom
  • syntax-tree/mdast-util-from-quill-delta
• remark-yaml-config: update js-yaml
• vfile-matter: update js-yaml
• unified-engine: update js-yaml
• vfile-message: rename location -> position (Rename location field to position vfile/vfile-message#6)
• vfile: rename contents to value (Rename contents field to value vfile/vfile#49)
• mdast-util-to-string: do not use node.title (Remove node.title support syntax-tree/mdast-util-to-string#8)
• mdast: rename depth to rank (Rename depth to rank syntax-tree/mdast#31) (I feel the other issues there plus Rename node types syntax-tree/nlcst#6 are too much break for what they bring)
• hast: change doctypes (Simplify doctypes syntax-tree/hast#19)

Small breaking changes:

• hastscript’s h and s and a JSX pragma Add support for using h, s as a JSX pragmas syntax-tree/hastscript#15
• unist-util-find-all-before’s results should be reversed probably Results are reversed syntax-tree/unist-util-find-all-before#6

For ES++ features (unifiedjs/rfcs#4), I’d say we can do that after we switch to ESM, because ESM will already be a lot of work, and it gives us a baseline of what ES features / engines to support.

[chrisrzhou]

Are there discussion threads (any links to notable discussions are good enough!) where I can read more into the motivation to move formally to using ESM? Inquiring to stay up to date with the unified ecosystem, and also getting context in the broader JS ecosystem.

[wooorm]

micromark/micromark#27 and other PRs there, but that was both ESM and CJS.
See also https://twitter.com/sindresorhus/status/1349294527350149121.
The motivation is essentially: we’ll have to move some time. This is a good time!

[nnmrts]

Great to see unified switching over as well!

@wooorm Please don't archive retextjs/retext-sentiment. I'm ready to maintain it if you want. It just seems too useful to me to just forget it.

[wooorm]

I’m also going to try to add jsdoc based types to everything (starting with ±150 of my own projects).

Another big one is to add support for plugins as ESM in unified-engine: the only way forward I see is if plugins export default? Or have a named plugin export? 🤔 I prefer the first from the unified-engine perspective, but as an author I don’t like default exports.

[chrisrzhou]

@wooorm, (for my own learning), a few questions relating to his upcoming refactor:

1. What are the benefits for jsdoc types vs .d.ts vs strictly implementing the code in TS?

The main benefits I'm inferring is decoupling the dependency and reliance on TS, so that if the ecosystem needs to be free of TS in the future, we at least have JSDocs with an easy path to decoupling in the future? I'm curious about the decision because I'm facing similar concerns with trying to keep source code as independent from tooling/dependencies as possible (flow was king at one point, typescript is currently 'king', but no one knows what will happen with the typing ecosystem in the future).

2. With the focus on ES modules, it seems pretty amazing that we don't need build tools since it's natively supported by Node and modern browsers. How do you resolve dependencies in unified ecosystem that are not migrated to ES modules? E.g. we cannot naturally rely on package.json's type: module if an upstream dependency hasn't formally migrated to ES module (even if the package supports ES modules through build tools and exporting "module: index.module.js" in package.json, Node scripts won't treat it as a formal ES module). Is this problem avoided in the ecosystem because the ecosystem has very limited external dependencies?

3. Related to 2, should packages (even microutils) in the ecosystem care about bundling/minifying or should that just be left to upstream consumers?

Would love to hear the perspectives and approaches of the team here, so I can also learn something from it!

And my (personal and likely not-important) opinion on:

Another big one is to add support for plugins as ESM in unified-engine: the only way forward I see is if plugins export default? Or have a named plugin export? 🤔 I prefer the first from the unified-engine perspective, but as an author I don’t like default exports.

I think most authors avoid default exports and use named exports (explicit named export is useful as a static guard, also great for refactoring), although I think that if we have to consider how consumers would use an API, we should make the appropriate decision based on how this library is intended to be consumed. For myself, I use 1) named exports for internal implementation all the time and 2) decide the final public export based on what makes sense for the consumer.

[chrisrzhou]

Didn't realize Github does not support comment threads, so it's kind of annoying keeping track of discussions and responses :P (this comment further adds to the problem)