Mini-RFC: --no-pack flag

[ashleygwilliams]

Feature Description
when calling wasm-pack build a user can optionally pass --no-pack and wasm-pack will build your wasm, generate js, and not build a package.json

Example

wasm-pack build --target no-modules --no-pack

Outstanding Questions

• Mode? should we stick this in as a mode? or should it be toplevel? docs for mode:
  https://rustwasm.github.io/docs/wasm-pack/commands/build.html#mode
• the default outdir is called pkg.. i'd... probably keep this the default for --no-pack but it will certainly be Weird

[ashleygwilliams]

cc @Pauan rustwasm/wasm-bindgen#1674 (comment)

[alexcrichton]

@Pauan can you elaborate a bit more on the motivation here as well? Are you worried about some diagnostics coming out about package.json that may be easily run into when they shouldn't?

I personally would agree that we should keep the same output folder, although it seems reasonable to restructure it slightly. For example we may not need to add a .gitignore, but just check if the top-level .gitignore ignores pkg (or something like that)

[Pauan]

@ashleygwilliams Mode? should we stick this in as a mode?

I don't think it should use --mode, since that's orthogonal.

As for whether it should have a "mode style" flag or not, personally, I think --no-pack or --no-package is fine, but I don't have strong feelings about it.

the default outdir is called pkg.. i'd... probably keep this the default for --no-pack

I agree that pkg should still be the default. It can be changed with --out-dir, so that's not an issue.

---

@alexcrichton can you elaborate a bit more on the motivation here as well? Are you worried about some diagnostics coming out about package.json that may be easily run into when they shouldn't?

The motivations are:

• Avoid doing unnecessary checks and unnecessary work.

  As an example, it wouldn't need to handle npm packages at all, it wouldn't need to merge package.json, it wouldn't need to output the extra files, etc. This improves performance.

• Avoid unnecessary warnings (e.g. warnings for missing README, warnings for missing LICENSE, etc.)

For example we may not need to add a .gitignore, but just check if the top-level .gitignore ignores pkg (or something like that)

After looking into it, I don't have much of an opinion about .gitignore: it's a small file that just contains *, and it doesn't require much extra work, so I'm fine with continuing to output it.

[Pauan]

This is slightly off-topic, but we should also start thinking about how to handle npm dependencies with applications.

Right now wasm-pack generates a pkg/package.json file which contains the "dependencies" for all of the Rust crates (well, at least that's the intention).

That works fine if you just publish that package.json, but not if you are creating an application (such as with the rust-webpack-template).

The issue is that there are now two package.jsons: the top-level package.json for the app, and the pkg/package.json which contains the Rust dependencies.

That means that the pkg/package.json dependencies aren't installed, so the developer has to manually install them.

So we need to somehow fix that so the pkg/package.json dependencies are automatically installed.

This is another reason for the --no-package flag: since we will need to handle npm dependencies differently, it makes sense that --no-package should trigger that special handling.

This should probably be filed as a different issue, but it is related to --no-package.

Come to think of it, given the above, maybe we should call the flag --application instead, that seems closer to the intent.

[alexcrichton]

I don't really personally buy the argument about performance in the sense that you're not wrong but we're talking about a handful of milliseconds at most here. I do think it's compelling to cater to the use case of removing extraneous warnings or extraneous checks that users could stumble over (such as generating an error or a warning that doesn't make sense in the "no pack" context).

I'm wondering if we could perhaps change a few defaults and basically cater to your use case @Pauan. I suspect we could continue to just go ahead and generate package.json by default (with a flag to turn it off), and wasm-pack would likely be already adequate minus it generating an extra file.

[Pauan]

I don't really personally buy the argument about performance in the sense that you're not wrong but we're talking about a handful of milliseconds at most here.

I have tried modifying my loader to use wasm-pack, and it has a significant performance cost:

• with wasm-pack: 2100ms
• with wasm-bindgen only: 1630ms

The above numbers were taken on a tiny "hello world" project which had already been compiled, so cargo build only took 130ms, which means most of the time is spent in wasm-bindgen (1500ms), followed by 470ms spent in wasm-pack.

The above is only for a single crate. It will naturally scale linearly with the number of crates. I have some projects which have 6 crates, so I expect wasm-pack will take 12600ms compared to 9780ms for wasm-bindgen, a difference of 2820ms.

Now, you might be right that the package.json handling itself isn't slow (I have to do some profiling to find out), but it seems to me that all of the checks and processing that wasm-pack does is indeed slow, and will only get slower in the future (when #606 is implemented).

An individual operation might only take a handful of milliseconds (a single disk I/O takes several ms), but when you're doing dozens or hundreds of operations, it adds up.

I suspect we could continue to just go ahead and generate package.json by default (with a flag to turn it off), and wasm-pack would likely be already adequate minus it generating an extra file.

I'm confused: my argument is that there should be a flag to prevent it from generating package.json, and you argue against that, but then you say that there should be a flag to prevent it from generating package.json?

[alexcrichton]

Er ok so lemme explain a bit on the performance part. I've worked extensively on Cargo, rustc, etc, and I've worked quite a lot on those with respect to performance. When I say wasm-pack should take a handful of milliseconds, I mean it. I know disk I/O can be slow but the fact of the matter is that it basically isn't. Cargo does orders of magnitude more I/O than wasm-pack and wasm-bindgen (like, multiple orders), so there's absolutely no reason that wasm-pack/wasm-bindgen should be slow.

If you're seeing 1500ms in wasm-bindgen for a hello world crate, that's bad! If you're seeing a 500ms performance overhead using wasm-pack, that's bad! What I'm asserting above is obviously modulo bugs and such, but I really want to drive home that this is something we care about and a 500ms performance difference is not acceptable. It's something that needs to be diagnosed, fixed, and upstreamed.

Can you provide a sample project with reproduction steps to see the discrepancy? If wasm-bindgen takes 1500ms that almost sounds like you're doing a debug build. If wasm-pack takes 500ms extra then I have no idea where that time is going but we need to profile, figure out where it's going, and figure out how to solve it.

These are all surmountable issues and we have so much to benefit by staying consistent on the same toolchain (e.g. like Cargo for Rust) so we need to not be distracted by small bugs which make it seem like the whole toolchain needs to be updated.

I'm confused: my argument is that there should be a flag to prevent it from generating package.json, and you argue against that, but then you say that there should be a flag to prevent it from generating package.json?

Sorry what I mean is that I don't believe there's a reason to avoid generating package.json for performance. If performance is the only reason, there's no reason that flag should exist. For functionality, however, it seems like a reasonable thing to add.