Compile Ohm grammars to WebAssembly

[pdubroy]

Hi everyone!

A couple months ago, I started prototyping a new feature: the ability to compile an Ohm grammar to WebAssembly, so that it can be used from languages other than JS. You can find more background (use cases and implementation details) in #503.

The MVP is complete, and this feature is now available for early testing (with some limitations). We're very interested to get feedback and hear what uses cases people have for this.

Although it wasn't the main motivation for this work, early benchmarks are showing ~10x improvement in parse times. Take this with a grain of salt — since the implementation isn't yet complete, it's hard to know what the final performance will look like. But it looks like this will be a significant performance win for real-world grammars!

Note

We are seeking funding to make it possible to ship a production-ready version. If you or your company could benefit from the portability and performance improvements of this work and could potentially help fund it, please get in touch with @pdubroy.

What's supported

• Given an existing Ohm grammar, you can use the ohm2wasm command — from the new @ohm-js/wasm package — to compile the grammar to a .wasm blob.
• To use that grammar, you need a miniohm package for your host language (i.e., the language in which you'll be parsing the input). Right now there are two supported languages: Go, and JavaScript. (The JS package is not yet documented but it's only 160 LoC, so it shouldn't be too hard to figure out how to use it.)

Limitations

There are currently a bunch of limitations, including:

• no implicit space skipping
• max 256 rules in the grammar
• max 64k of input
• parameterized rules don't support >3 params
• only direct* left recursion is supported.

• Direct also includes things like AddExp = AddExp '+' number -- plus.

For a more up to date list, see the TODO.md.

What's next

With the MVP complete, the remaining work is being tracked in #511. We are looking for a small amount of funding to continue this work. We estimate that there is ~8–10w of effort remaining; if/when we can secure funding, it will be my primary focus. Until then, I will try to make progress on it in my spare time, but it won't be my main focus.