Can mill native launcher be statically linked?

[jackkoenig]

In the hardware industry, it's common to have an older Linux distro, e.g. RHEL8 is really common which will gives the following linkage error when trying out Mill 1.0.0.

/scratch/koenig/.cache/mill/download/1.0.0-native-linux-amd64: /lib64/libc.so.6: version `GLIBC_2.32' not found (required by /scratch/koenig/.cache/mill/download/1.0.0-native-linux-amd64)
/scratch/koenig/.cache/mill/download/1.0.0-native-linux-amd64: /lib64/libc.so.6: version `GLIBC_2.34' not found (required by /scratch/koenig/.cache/mill/download/1.0.0-native-linux-amd64)

This can be worked around by using the JVM version of the client, but it would be nice if the client could be statically linked.

[lihaoyi]

Mill delegates the native image building to Graal. Perhaps you could try following https://www.graalvm.org/latest/reference-manual/native-image/guides/build-static-executables/ and adding --static-nolibc in dist/package.mill's nativeImageOptions, running ./mill show dist.native.executable to generate a statically linked executable? If it works, we could make it the default, or available with a version suffix like -static

[lihaoyi]

Building a native executable with --static-nolibc seems to work

$ git diff
diff --git a/dist/package.mill b/dist/package.mill
index fc069e6357c..5012e87e684 100644
--- a/dist/package.mill
+++ b/dist/package.mill
@@ -306,7 +306,8 @@ object `package` extends MillJavaModule with DistModule {
     def nativeImageOptions = Seq(
       "--no-fallback",
       "--enable-url-protocols=https",
-      "-Os"
+      "-Os",
+      "--static-nolibc"
       // Enable JVisualVM support
       // https://www.graalvm.org/latest/tools/visualvm/#using-visualvm-with-graalvm-native-executables
       // "--enable-monitoring=jvmstat,heapdump"

@jackkoenig probably need you to try it out and see if it fixes the problem in your environment, if it does then we can just turn it on by default

[jackkoenig]

Hey @lihaoyi, I did some digging and here is what I figured out and think.

I think --static-nolibc is a good idea regardless. It looks like the only non-system library is zlib (at least on arm64 linux) so that's the only thing that will be statically linked when you add that option. It's still a good idea to not require Mill users to have zlib installed just in case they don't.

That doesn't actually solve my original problem though, as it specifically is the system libraries like libc that cause issues on older Linux distributions. For circt/firtool we fully statically link using musl libc¹. I am not certain if a fully statically linked binary should be the default as there is disagreement about that, but it's worth considering.

Another option that might be simpler would be to have the Mill boostrap script check the system's glibc version (at least on Linux). If it's new enough then it can fetch the native binary, if it isn't, then it can fall back to the JVM client.

My main concern is Chisel users (who as I mentioned are often on older Linux distros) following default Mill instructions and being confused. I could add -jvm to the Mill version in all Chisel documentation and our template which may be good enough, but I also do like using the native client on my Macbook 🙂.

Footnotes

1. by building in an Alpine Linux Docker container which is much easier than trying to use other distros. Be warned if you try this that the default thread stack size is 128KiB, we increase it to 8MiB for firtool. ↩

[lihaoyi]

@jackkoenig sounds like Graal's --static should do what you want right? That includes libc, so the binary should work regardless of what libc version the user has installed