[SR-9307] RFE: WebAssembly Support

[ddunbar]

Previous ID	SR-9307
Radar	None
Original Reporter	@ddunbar
Type	New Feature

Additional Detail from JIRA

Votes	20
Component/s
Labels	New Feature, WebAssembly
Assignee	@MaxDesiatov
Priority	Medium

md5: 0f339f235efbb21077c1ec85b26beb6a

Sub-Tasks:

• SR-15168 Set up Swift Community CI for WebAssembly target

relates to:

• SR-15168 Set up Swift Community CI for WebAssembly target

Issue Description:

Swift should support compiling to WebAssembly.

[ddunbar]

This is a fairly large project, and it isn't clear to me yet exactly what concrete steps are involved.

At a high level, what I think we would want is that it is very straightforward to build Swift packages targeting WASM, and including good tooling for integrating the result into a web application. This has several immediate high-level implications:

1. Given Swift's natural bridging with C, we should expect that Swift WASM interoperates with Clang-based WASM, so that we can easily have mixed-language projects that work. Via its C family target support, this also will make SwiftPM a reasonable platform for building C-based WASM projects.

2. When used in a web context, the WASM will sometimes want to be integrated with the server. This means SwiftPM ideally would support building and integrating results built with multiple deployment targets.

At a more concrete level, these are some of the required tasks:

1. Add support for a WASM target triple to the compiler.

   1. This is trivial, and will let swiftc target `wasm{32,64}` using LLVM's existing backend.

2. Disable or error on unsupported constructs.

   1. Update the compiler to remove the use of unsupported constructs (swiftcc calling convention) or to error on things which won't work yet.

3. Add support for building a WASM version of the stdlib.

   1. This is complex, because of the runtime requirements of the standard library. We may want to start by building a thinned down version of the stdlib with the things that are easy (e.g. "Int", hopefully), and add parts back in as the compiler support matures.

   2. This should be enough to allow creation of non-trivial WASM fragments which doing something interesting, and can be manually loaded and tested.

4. Add support for building LLVM bitcode packages + LTO.

   1. In lieu of a mature WebAssembly linking standard (???), the easiest way to get complex packages into WASM is to build everything to LLVM bitcode, use LLVM's module linking to link into a single LLVM module (potentially with optimization and dead code stripping), and then emit WASM from that.

   2. This is a useful feature to add to Swift anyway, as it enables LTO for Swift in other contexts, which can bring some performance and code size improvements (especially for contexts where the intra-package ABI does not need to be stable, such as most server-side applications and some parts of client-side applications).

5. Define the mechanism/ABI/tooling etc. by which Swift WASM programs are loaded in JavaScript.

   1. In order to support development of a full runtime, we need a well-defined interface with the JavaScript loader environment. It isn't yet clear to me if there is a clear standard for this; but it will be needed in order to support integrating a full runtime library (e.g. object allocation) with a built Swift program. As such, this probably is required before the full standard library can be supported.

   2. Ideally, this would just amount to adoption of an existing standard by which language runtimes are combined with language built programs. I have not yet investigated what other toolchains are doing to manage this.

[ddunbar]

WIP PRs, these are enough to get swiftc to emit a trivial wasm file:

[-apple/swift-clang#235]

(rejected, landed on Clang TOT)

apple/swift-clang#236

(rejected, instead will support the swiftcc convention in WASM code generator)

#20684

(pending other changes, and an end-to-end test)

#20687

(merged)

#20688

(rejected, part of swiftcc alternate strategy above)

[ddunbar]

The next major effort here (beyond merging/reworking some of the above PRs) is to land enough CMake support so that the standard library can be (attempted to be) built for WASM. I pushed some minimal efforts in that direction here:

https://github.com/ddunbar/swift/tree/wasm-cmake

The major direction that needs to be figured out is whether or not Swift+WASM should depend on Emscripten or not. Using Emscripten has a number of advantages in that it is (IIUC) the most mature toolchain for targeting WASM from C++ code, but it has the downside that it brings in a very hefty and not really sensible dependency (for example, it would be very unlikely we would ship it inside of a Swift toolchain, which would rule out self-contained Swift+WASM support).

My tendency is to start by instead trying to make the WASM support be self-contained, relying on the WebAssembly support which is inside of the Clang that Swift already has inside of it. This will presumably become more mature over time. The difficulty with this strategy is that, on its own, that provides no solution for dealing with the (a) necessary C++ SDK parts (standard library, standard library headers, etc), (b) the interaction between the JS runtime and the WASM code – problems which Emscripten has already solved. However, the upside of this approach is that a standard Swift OSS toolchain could potentially come with WASM support out of the box (once complete).

I don't actually understand the WASM ecosystem enough yet to understand the true implications of either of the above choices (and maybe there are more!).

[belkadan]

I think the main reason this hasn't been taken on is because the stdlib is still several megabytes. A WASM version of Swift is either going to have a large startup time or is going to be missing features.

[ddunbar]

Several megabytes of WASM isn't actually particularly uncommon for the kinds of things people are compiling to it & asmjs. WASM has a number of features to potentially ameliorate this, although as you might imagine integrating them with Swift is a rather non-trivial thing.

For reference, the smallest Go WASM files right now are 2MB (https://github.com/golang/go/wiki/WebAssembly#reducing-the-size-of-wasm-files).

At this point I think the size of the WASM is the least of our problems. 🙂

[ddunbar]

Clang ABIInfo patch posted here, since that code has been upstreamed to Clang: https://reviews.llvm.org/D56188

[jckarter]

If the goal is to deploy Swift to web clients, another high-level approach that might make sense is to target Javascript instead of WASM, potentially by lowering SIL to Javascript without involving LLVM. The Javascript runtime already provides a lot of functionality that the Swift runtime needs to build up from scratch, like method dispatch, memory management, value type strings, etc., and a Swift runtime environment built on top of Javascript could potentially be smaller overall to deploy than a WASM binary. We could also have access to the full web platform without being trapped in the wasm box.

[ddunbar]

I think that is very fair. I was sort of hoping that the WebAssembly approach would be significantly less work (and relatively easy to maintain), but its not obvious that that is true.

On the flip side, using WebAssembly means there is some potential for transparently supporting Swift packages that also include C/C++ source code.

[jckarter]

You're probably right that getting wasm minimally working would be much less work, but my concern would be that there's a lot of follow-up work in making it practical as well, since as Jordan noted, a naive port of Swift to wasm would have to carry around a lot of fairly heavyweight dependencies with it. The Swift runtime also transitively depends on a lot of POSIX-ish and C++ standard library functionality that's already "free" on traditional platforms but not necessarily so in wasm, so the dependency problem might be even worse than it looks by naively looking at the size of the Swift dylibs on macOS or Linux. OTOH, I know there's a lot of excitement around wasm as a general binary deployment technology for things like server VMs too, and concerns like download size, Javascript and web platform interop, etc. are less of an issue for those use cases.

[swift-ci]

Comment by Mark Malstrom (JIRA)

The Swift runtime also transitively depends on a lot of POSIX-ish and C++ standard library functionality that's already "free" on traditional platforms but not necessarily so in wasm

Most, if not all, of this issue should be solved by the new WASI standard now: https://hacks.mozilla.org/2019/03/standardizing-wasi-a-webassembly-system-interface/

OTOH, I know there's a lot of excitement around wasm as a general binary deployment technology for things like server VMs too, and concerns like download size, Javascript and web platform interop, etc. are less of an issue for those use cases.

This is what I've observed as the focus of most of the excitement around WebAssembly and the biggest reason I, personally, would like to see WebAssembly compilation support for Swift.

[MaxDesiatov]

Embedded Wasm target has been available in preview in development snapshots off main for a couple of months now. WASI Preview 1 support in core libraries such as swift-corelibs-foundation and swift-corelibs-xctest has been merged over the course of recent weeks. I would prefer remaining outstanding issues to be tracked separately, but otherwise this one seems to be more or less completed.