Skip to content

Conversation

@lihaoyi
Copy link
Member

@lihaoyi lihaoyi commented Feb 16, 2025

Pulls in com-lihaoyi/os-lib#359 from upstream to fix a problem with leaked processes causing hangs

@lihaoyi lihaoyi marked this pull request as ready for review February 16, 2025 01:46
lihaoyi added a commit that referenced this pull request Feb 16, 2025
Should fix the rebootstrap problems found in
#4545 and
#4570 that seems to be caused by
a mistake in this backport
#4504. Not sure why this causes
a hang, but for now just fixing it to revert to the previous semantics

Tested manually via

```bash
./mill dist.installLocal
git checkout main
./mill-assembly.jar 'integration.invalidation[selective-execution].local.fork.test' mill.integration.SelectiveExecutionWatchTests.watch.changed-code
```

Doing this manually on `0.12.x` hangs, whereas doing it on this PR
succeeds
@lihaoyi
Copy link
Member Author

lihaoyi commented Feb 16, 2025

Seems like the problem is this stderr pumper thread refusing to exit

"/Users/lihaoyi/Library/Caches/Coursier/arc/https/cdn.azul.com/zulu/bin/zulu17.54.21-ca-jdk17.0.13-macosx_aarch64.tar.gz/zulu17.54.21-ca-jdk17.0.13-macosx_aarch64/zulu-17.jdk/Contents/Home/bin/java -DMILL_SCALA_2_13_VERSION=2.13.15 -DMILL_SCALA_2_12_VERSION=2.12.20 -DTEST_SCALA_2_13_VERSION=2.13.15 -DTEST_SCALA_2_13_VERSION_FOR_SCALANATIVE_4_2=2.13.8 -DTEST_SCALA_2_12_VERSION=2.12.6 -DTEST_SCALA_3_2_VERSION=3.2.0 -DTEST_SCALA_3_3_VERSION=3.3.1 -DTEST_SCALAJS_VERSION=1.18.1 -DTEST_SCALANATIVE_0_5_VERSION=0.5.6 -DTEST_UTEST_VERSION=0.8.5 -DTEST_SCALATEST_VERSION=3.2.19 -DTEST_TEST_INTERFACE_VERSION=1.0 -DTEST_ZIOTEST_VERSION=2.1.14 -DTEST_ZINC_VERSION=1.10.7 -DTEST_KOTLIN_VERSION=2.0.21 -Djna.nosys=true -cp /Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/scala-library/2.13.15/scala-library-2.13.15.jar:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/com/lihaoyi/mill-moduledefs_2.13/0.11.2/mill-moduledefs_2.13-0.11.2.jar:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/com/lihaoyi/utest_2.13/0.8.5/utest_2.13-0.8.5.jar:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/com/lihaoyi/sourcecode_2.13/0.4.2/sourcecode_2.13-0.4.2.jar:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/scala-compiler/2.13.15/scala-compiler-2.13.15.jar:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/io/github/alexarchambault/windows-ansi/windows-ansi/0.0.6/windows-ansi-0.0.6.jar:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/io/get-coursier/interface/1.0.27/interface-1.0.27.jar:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/com/lihaoyi/mainargs_2.13/0.7.6/mainargs_2.13-0.7.6.jar:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/com/lihaoyi/requests_2.13/0.9.0/requests_2.13-0.9.0.jar:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/ch/qos/logback/logback-classic/1.5.16/logback-classic-1.5.16.jar:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/jgrapht/jgrapht-core/1.4.0/jgrapht-core-1.4.0.jar:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/guru/nidi/graphviz-java-min-deps/0.18.1/graphviz-java-min-deps-0.18.1.jar:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/scala-sbt/test-interface/1.0/test-interface-1.0.jar:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/portable-scala/portable-scala-reflect_2.13/1.1.3/portable-scala-reflect_2.13-1.1.3.jar:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/scala-reflect/2.13.15/scala-reflect-2.13.15.jar:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/io/get-coursier/coursier_2.13/2.1.24/coursier_2.13-2.1.24.jar:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/io/get-coursier/coursier-jvm_2.13/2.1.24/coursier-jvm_2.13-2.1.24.jar:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/jline/jline/3.28.0/jline-3.28.0.jar:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/scalameta/scalafmt-dynamic_2.13/3.8.5/scalafmt-dynamic_2.13-3.8.5.jar:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/modules/scala-xml_2.13/2.3.0/scala-xml_2.13-2.3.0.jar:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/ow2/asm/asm-tree/9.7.1/asm-tree-9.7.1.jar:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/com/lihaoyi/os-lib_2.13/0.11.4-M6/os-lib_2.13-0.11.4-M6.jar:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/com/lihaoyi/pprint_2.13/0.9.0/pprint_2.13-0.9.0.jar:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/io/github/java-diff-utils/java-diff-utils/4.12/java-diff-utils-4.12.jar:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/net/java/dev/jna/jna/5.16.0/jna-5.16.0.jar:/Users/lihaoyi/.ivy2/local/org.fusesource.jansi/jansi/2.4.1/jars/jansi.jar:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/slf4j/slf4j-api/2.0.16/slf4j-api-2.0.16.jar:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/modules/scala-collection-compat_2.13/2.12.0/scala-collection-compat_2.13-2.12.0.jar:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/com/lihaoyi/geny_2.13/1.1.1/geny_2.13-1.1.1.jar:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/ch/qos/logback/logback-core/1.5.16/logback-core-1.5.16.jar:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/jheaps/jheaps/0.11/jheaps-0.11.jar:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/guru/nidi/graphviz-java/0.18.1/graphviz-java-0.18.1.jar:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/net/java/dev/jna/jna-platform/5.16.0/jna-platform-5.16.0.jar:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/com/eed3si9n/jarjarabrams/jarjar-abrams-core_2.13/1.14.0/jarjar-abrams-core_2.13-1.14.0.jar:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/com/lihaoyi/scalaparse_2.13/3.1.1/scalaparse_2.13-3.1.1.jar:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/com/lihaoyi/upickle_2.13/3.3.1/upickle_2.13-3.3.1.jar:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/com/lihaoyi/fansi_2.13/0.5.0/fansi_2.13-0.5.0.jar:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/io/get-coursier/dependency_2.13/0.3.2/dependency_2.13-0.3.2.jar:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/com/github/plokhotnyuk/jsoniter-scala/jsoniter-scala-core_2.13/2.13.5.2/jsoniter-scala-core_2.13-2.13.5.2.jar:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/io/get-coursier/coursier-core_2.13/2.1.24/coursier-core_2.13-2.1.24.jar:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/io/get-coursier/coursier-cache_2.13/2.1.24/coursier-cache_2.13-2.1.24.jar:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/io/get-coursier/coursier-proxy-setup/2.1.24/coursier-proxy-setup-2.1.24.jar:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/io/get-coursier/coursier-env_2.13/2.1.24/coursier-env_2.13-2.1.24.jar:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/scalameta/scalafmt-interfaces/3.8.5/scalafmt-interfaces-3.8.5.jar:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/scalameta/scalafmt-sysops_2.13/3.8.5/scalafmt-sysops_2.13-3.8.5.jar:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/com/typesafe/config/1.4.3/config-1.4.3.jar:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/ow2/asm/asm/9.7.1/asm-9.7.1.jar:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/guru/nidi/com/eclipsesource/j2v8/j2v8_macosx_x86_64/4.6.0/j2v8_macosx_x86_64-4.6.0.jar:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/com/google/code/findbugs/jsr305/3.0.2/jsr305-3.0.2.jar:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/slf4j/jcl-over-slf4j/1.7.30/jcl-over-slf4j-1.7.30.jar:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/slf4j/jul-to-slf4j/1.7.30/jul-to-slf4j-1.7.30.jar:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/com/eed3si9n/jarjar/jarjar/1.14.0/jarjar-1.14.0.jar:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/com/lihaoyi/fastparse_2.13/3.1.1/fastparse_2.13-3.1.1.jar:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/com/lihaoyi/ujson_2.13/3.3.1/ujson_2.13-3.3.1.jar:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/com/lihaoyi/upack_2.13/3.3.1/upack_2.13-3.3.1.jar:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/com/lihaoyi/upickle-implicits_2.13/3.3.1/upickle-implicits_2.13-3.3.1.jar:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/io/github/alexarchambault/concurrent-reference-hash-map/1.1.0/concurrent-reference-hash-map-1.1.0.jar:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/io/get-coursier/coursier-util_2.13/2.1.24/coursier-util_2.13-2.1.24.jar:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/io/github/alexarchambault/is-terminal/0.1.1/is-terminal-0.1.1.jar:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/io/get-coursier/jniutils/windows-jni-utils/0.3.3/windows-jni-utils-0.3.3.jar:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/codehaus/plexus/plexus-archiver/4.10.0/plexus-archiver-4.10.0.jar:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/codehaus/plexus/plexus-container-default/2.1.1/plexus-container-default-2.1.1.jar:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/virtuslab/scala-cli/config_2.13/1.1.3/config_2.13-1.1.3.jar:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/io/get-coursier/cache-util/2.1.24/cache-util-2.1.24.jar:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/modules/scala-parallel-collections_2.13/1.1.0/scala-parallel-collections_2.13-1.1.0.jar:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/ow2/asm/asm-commons/9.6/asm-commons-9.6.jar:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/apache/commons/commons-lang3/3.16.0/commons-lang3-3.16.0.jar:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/com/lihaoyi/upickle-core_2.13/3.3.1/upickle-core_2.13-3.3.1.jar:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/javax/inject/javax.inject/1/javax.inject-1.jar:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/codehaus/plexus/plexus-utils/4.0.1/plexus-utils-4.0.1.jar:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/codehaus/plexus/plexus-io/3.5.0/plexus-io-3.5.0.jar:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/commons-io/commons-io/2.18.0/commons-io-2.18.0.jar:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/apache/commons/commons-compress/1.27.1/commons-compress-1.27.1.jar:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/io/airlift/aircompressor/0.27/aircompressor-0.27.jar:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/tukaani/xz/1.9/xz-1.9.jar:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/com/github/luben/zstd-jni/1.5.6-3/zstd-jni-1.5.6-3.jar:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/codehaus/plexus/plexus-classworlds/2.6.0/plexus-classworlds-2.6.0.jar:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/apache/xbean/xbean-reflect/3.7/xbean-reflect-3.7.jar:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/virtuslab/scala-cli/specification-level_2.13/1.1.3/specification-level_2.13-1.1.3.jar:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/commons-codec/commons-codec/1.17.1/commons-codec-1.17.1.jar:/Users/lihaoyi/Github/mill/testkit/compile-resources:/Users/lihaoyi/Github/mill/testkit/resources:/Users/lihaoyi/Github/mill/out/testkit/compile.dest/classes:/Users/lihaoyi/Github/mill/main/resolve/compile-resources:/Users/lihaoyi/Github/mill/main/resolve/resources:/Users/lihaoyi/Github/mill/out/main/resolve/compile.dest/classes:/Users/lihaoyi/Github/mill/main/util/compile-resources:/Users/lihaoyi/Github/mill/main/util/resources:/Users/lihaoyi/Github/mill/out/main/util/compile.dest/classes:/Users/lihaoyi/Github/mill/main/client/compile-resources:/Users/lihaoyi/Github/mill/main/client/resources:/Users/lihaoyi/Github/mill/out/main/client/buildInfoResources.dest:/Users/lihaoyi/Github/mill/out/main/client/compile.dest/classes:/Users/lihaoyi/Github/mill/main/api/compile-resources:/Users/lihaoyi/Github/mill/main/api/resources:/Users/lihaoyi/Github/mill/out/main/api/buildInfoResources.dest:/Users/lihaoyi/Github/mill/out/main/api/compile.dest/classes:/Users/lihaoyi/Github/mill/main/define/compile-resources:/Users/lihaoyi/Github/mill/main/define/resources:/Users/lihaoyi/Github/mill/out/main/define/compile.dest/classes:/Users/lihaoyi/Github/mill/main/eval/compile-resources:/Users/lihaoyi/Github/mill/main/eval/resources:/Users/lihaoyi/Github/mill/out/main/eval/compile.dest/classes:/Users/lihaoyi/Github/mill/main/compile-resources:/Users/lihaoyi/Github/mill/main/resources:/Users/lihaoyi/Github/mill/out/main/buildInfoResources.dest:/Users/lihaoyi/Github/mill/out/main/compile.dest/classes:/Users/lihaoyi/Github/mill/main/test/compile-resources:/Users/lihaoyi/Github/mill/main/test/resources:/Users/lihaoyi/Github/mill/out/main/test/compile.dest/classes:/Users/lihaoyi/Github/mill/runner/client/compile-resources:/Users/lihaoyi/Github/mill/runner/client/resources:/Users/lihaoyi/Github/mill/out/runner/client/buildInfoResources.dest:/Users/lihaoyi/Github/mill/out/runner/client/compile.dest/classes:/Users/lihaoyi/Github/mill/main/server/compile-resources:/Users/lihaoyi/Github/mill/main/server/resources:/Users/lihaoyi/Github/mill/out/main/server/compile.dest/classes:/Users/lihaoyi/Github/mill/main/codesig/compile-resources:/Users/lihaoyi/Github/mill/main/codesig/resources:/Users/lihaoyi/Github/mill/out/main/codesig/compile.dest/classes:/Users/lihaoyi/Github/mill/runner/linenumbers/compile-resources:/Users/lihaoyi/Github/mill/runner/linenumbers/resources:/Users/lihaoyi/Github/mill/out/runner/linenumbers/compile.dest/classes:/Users/lihaoyi/Github/mill/bsp/compile-resources:/Users/lihaoyi/Github/mill/bsp/resources:/Users/lihaoyi/Github/mill/out/bsp/buildInfoResources.dest:/Users/lihaoyi/Github/mill/out/bsp/compile.dest/classes:/Users/lihaoyi/Github/mill/pythonlib/compile-resources:/Users/lihaoyi/Github/mill/pythonlib/resources:/Users/lihaoyi/Github/mill/out/pythonlib/compile.dest/classes:/Users/lihaoyi/Github/mill/javascriptlib/compile-resources:/Users/lihaoyi/Github/mill/javascriptlib/resources:/Users/lihaoyi/Github/mill/out/javascriptlib/compile.dest/classes:/Users/lihaoyi/Github/mill/scalanativelib/worker-api/compile-resources:/Users/lihaoyi/Github/mill/scalanativelib/worker-api/resources:/Users/lihaoyi/Github/mill/out/scalanativelib/worker-api/compile.dest/classes:/Users/lihaoyi/Github/mill/scalanativelib/compile-resources:/Users/lihaoyi/Github/mill/scalanativelib/resources:/Users/lihaoyi/Github/mill/out/scalanativelib/compile.dest/classes:/Users/lihaoyi/Github/mill/scalajslib/worker-api/compile-resources:/Users/lihaoyi/Github/mill/scalajslib/worker-api/resources:/Users/lihaoyi/Github/mill/out/scalajslib/worker-api/compile.dest/classes:/Users/lihaoyi/Github/mill/scalajslib/compile-resources:/Users/lihaoyi/Github/mill/scalajslib/resources:/Users/lihaoyi/Github/mill/out/scalajslib/buildInfoResources.dest:/Users/lihaoyi/Github/mill/out/scalajslib/compile.dest/classes:/Users/lihaoyi/Github/mill/kotlinlib/worker/compile-resources:/Users/lihaoyi/Github/mill/kotlinlib/worker/resources:/Users/lihaoyi/Github/mill/out/kotlinlib/worker/compile.dest/classes:/Users/lihaoyi/Github/mill/kotlinlib/compile-resources:/Users/lihaoyi/Github/mill/kotlinlib/resources:/Users/lihaoyi/Github/mill/out/kotlinlib/buildInfoResources.dest:/Users/lihaoyi/Github/mill/out/kotlinlib/compile.dest/classes:/Users/lihaoyi/Github/mill/testrunner/entrypoint/compile-resources:/Users/lihaoyi/Github/mill/testrunner/entrypoint/resources:/Users/lihaoyi/Github/mill/out/testrunner/entrypoint/compile.dest/classes:/Users/lihaoyi/Github/mill/testrunner/compile-resources:/Users/lihaoyi/Github/mill/testrunner/resources:/Users/lihaoyi/Github/mill/out/testrunner/compile.dest/classes:/Users/lihaoyi/Github/mill/scalalib/api/compile-resources:/Users/lihaoyi/Github/mill/scalalib/api/resources:/Users/lihaoyi/Github/mill/out/scalalib/api/buildInfoResources.dest:/Users/lihaoyi/Github/mill/out/scalalib/api/compile.dest/classes:/Users/lihaoyi/Github/mill/scalalib/compile-resources:/Users/lihaoyi/Github/mill/scalalib/resources:/Users/lihaoyi/Github/mill/out/scalalib/compile.dest/classes:/Users/lihaoyi/Github/mill/runner/compile-resources:/Users/lihaoyi/Github/mill/runner/resources:/Users/lihaoyi/Github/mill/out/runner/compile.dest/classes:/Users/lihaoyi/Github/mill/integration/invalidation/selective-execution/compile-resources:/Users/lihaoyi/Github/mill/integration/invalidation/selective-execution/resources:/Users/lihaoyi/Github/mill/out/integration/invalidation/selective-execution/compile.dest/classes:/Users/lihaoyi/Github/mill/out/integration/invalidation/selective-execution/runClasspath.dest:/Users/lihaoyi/.ivy2/local/com.lihaoyi/mill-testrunner-entrypoint/0.12.8-1-46e216/jars/mill-testrunner-entrypoint.jar:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/scala-sbt/test-interface/1.0/test-interface-1.0.jar mill.testrunner.entrypoint.TestRunnerMain file:/Users/lihaoyi/.ivy2/local/com.lihaoyi/mill-scalalib_2.13/0.12.8-1-46e216/jars/mill-scalalib_2.13.jar,file:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/scalameta/scalafmt-dynamic_2.13/3.8.5/scalafmt-dynamic_2.13-3.8.5.jar,file:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/modules/scala-xml_2.13/2.3.0/scala-xml_2.13-2.3.0.jar,file:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/scala-library/2.13.16/scala-library-2.13.16.jar,file:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/com/lihaoyi/mill-moduledefs_2.13/0.11.2/mill-moduledefs_2.13-0.11.2.jar,file:/Users/lihaoyi/.ivy2/local/com.lihaoyi/mill-main_2.13/0.12.8-1-46e216/jars/mill-main_2.13.jar,file:/Users/lihaoyi/.ivy2/local/com.lihaoyi/mill-scalalib-api_2.13/0.12.8-1-46e216/jars/mill-scalalib-api_2.13.jar,file:/Users/lihaoyi/.ivy2/local/com.lihaoyi/mill-testrunner_2.13/0.12.8-1-46e216/jars/mill-testrunner_2.13.jar,file:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/scalameta/scalafmt-interfaces/3.8.5/scalafmt-interfaces-3.8.5.jar,file:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/scalameta/scalafmt-sysops_2.13/3.8.5/scalafmt-sysops_2.13-3.8.5.jar,file:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/io/get-coursier/interface/1.0.27/interface-1.0.27.jar,file:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/com/typesafe/config/1.4.3/config-1.4.3.jar,file:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/com/lihaoyi/sourcecode_2.13/0.4.2/sourcecode_2.13-0.4.2.jar,file:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/scala-compiler/2.13.15/scala-compiler-2.13.15.jar,file:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/io/github/alexarchambault/windows-ansi/windows-ansi/0.0.6/windows-ansi-0.0.6.jar,file:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/com/lihaoyi/mainargs_2.13/0.7.6/mainargs_2.13-0.7.6.jar,file:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/com/lihaoyi/requests_2.13/0.9.0/requests_2.13-0.9.0.jar,file:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/ch/qos/logback/logback-classic/1.5.16/logback-classic-1.5.16.jar,file:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/jgrapht/jgrapht-core/1.4.0/jgrapht-core-1.4.0.jar,file:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/guru/nidi/graphviz-java-min-deps/0.18.1/graphviz-java-min-deps-0.18.1.jar,file:/Users/lihaoyi/.ivy2/local/com.lihaoyi/mill-main-eval_2.13/0.12.8-1-46e216/jars/mill-main-eval_2.13.jar,file:/Users/lihaoyi/.ivy2/local/com.lihaoyi/mill-main-resolve_2.13/0.12.8-1-46e216/jars/mill-main-resolve_2.13.jar,file:/Users/lihaoyi/.ivy2/local/com.lihaoyi/mill-main-client/0.12.8-1-46e216/jars/mill-main-client.jar,file:/Users/lihaoyi/.ivy2/local/com.lihaoyi/mill-main-api_2.13/0.12.8-1-46e216/jars/mill-main-api_2.13.jar,file:/Users/lihaoyi/.ivy2/local/com.lihaoyi/mill-main-util_2.13/0.12.8-1-46e216/jars/mill-main-util_2.13.jar,file:/Users/lihaoyi/.ivy2/local/com.lihaoyi/mill-testrunner-entrypoint/0.12.8-1-46e216/jars/mill-testrunner-entrypoint.jar,file:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/modules/scala-parallel-collections_2.13/1.1.0/scala-parallel-collections_2.13-1.1.0.jar,file:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/slf4j/slf4j-api/2.0.16/slf4j-api-2.0.16.jar,file:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/scala-reflect/2.13.15/scala-reflect-2.13.15.jar,file:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/io/github/java-diff-utils/java-diff-utils/4.12/java-diff-utils-4.12.jar,file:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/jline/jline/3.28.0/jline-3.28.0.jar,file:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/net/java/dev/jna/jna/5.16.0/jna-5.16.0.jar,file:/Users/lihaoyi/.ivy2/local/org.fusesource.jansi/jansi/2.4.1/jars/jansi.jar,file:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/modules/scala-collection-compat_2.13/2.12.0/scala-collection-compat_2.13-2.12.0.jar,file:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/com/lihaoyi/geny_2.13/1.1.1/geny_2.13-1.1.1.jar,file:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/ch/qos/logback/logback-core/1.5.16/logback-core-1.5.16.jar,file:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/jheaps/jheaps/0.11/jheaps-0.11.jar,file:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/guru/nidi/graphviz-java/0.18.1/graphviz-java-0.18.1.jar,file:/Users/lihaoyi/.ivy2/local/com.lihaoyi/mill-main-define_2.13/0.12.8-1-46e216/jars/mill-main-define_2.13.jar,file:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/com/lihaoyi/os-lib_2.13/0.11.4-M6/os-lib_2.13-0.11.4-M6.jar,file:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/com/lihaoyi/upickle_2.13/3.3.1/upickle_2.13-3.3.1.jar,file:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/com/lihaoyi/pprint_2.13/0.9.0/pprint_2.13-0.9.0.jar,file:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/com/lihaoyi/fansi_2.13/0.5.0/fansi_2.13-0.5.0.jar,file:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/scala-sbt/test-interface/1.0/test-interface-1.0.jar,file:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/io/get-coursier/coursier_2.13/2.1.24/coursier_2.13-2.1.24.jar,file:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/io/get-coursier/coursier-jvm_2.13/2.1.24/coursier-jvm_2.13-2.1.24.jar,file:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/guru/nidi/com/eclipsesource/j2v8/j2v8_macosx_x86_64/4.6.0/j2v8_macosx_x86_64-4.6.0.jar,file:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/com/google/code/findbugs/jsr305/3.0.2/jsr305-3.0.2.jar,file:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/slf4j/jcl-over-slf4j/1.7.30/jcl-over-slf4j-1.7.30.jar,file:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/slf4j/jul-to-slf4j/1.7.30/jul-to-slf4j-1.7.30.jar,file:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/net/java/dev/jna/jna-platform/5.16.0/jna-platform-5.16.0.jar,file:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/com/eed3si9n/jarjarabrams/jarjar-abrams-core_2.13/1.14.0/jarjar-abrams-core_2.13-1.14.0.jar,file:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/com/lihaoyi/scalaparse_2.13/3.1.1/scalaparse_2.13-3.1.1.jar,file:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/com/lihaoyi/ujson_2.13/3.3.1/ujson_2.13-3.3.1.jar,file:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/com/lihaoyi/upack_2.13/3.3.1/upack_2.13-3.3.1.jar,file:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/com/lihaoyi/upickle-implicits_2.13/3.3.1/upickle-implicits_2.13-3.3.1.jar,file:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/io/get-coursier/dependency_2.13/0.3.2/dependency_2.13-0.3.2.jar,file:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/com/github/plokhotnyuk/jsoniter-scala/jsoniter-scala-core_2.13/2.13.5.2/jsoniter-scala-core_2.13-2.13.5.2.jar,file:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/io/get-coursier/coursier-core_2.13/2.1.24/coursier-core_2.13-2.1.24.jar,file:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/io/get-coursier/coursier-cache_2.13/2.1.24/coursier-cache_2.13-2.1.24.jar,file:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/io/get-coursier/coursier-proxy-setup/2.1.24/coursier-proxy-setup-2.1.24.jar,file:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/io/get-coursier/coursier-env_2.13/2.1.24/coursier-env_2.13-2.1.24.jar,file:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/com/eed3si9n/jarjar/jarjar/1.14.0/jarjar-1.14.0.jar,file:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/com/lihaoyi/fastparse_2.13/3.1.1/fastparse_2.13-3.1.1.jar,file:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/com/lihaoyi/upickle-core_2.13/3.3.1/upickle-core_2.13-3.3.1.jar,file:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/io/github/alexarchambault/concurrent-reference-hash-map/1.1.0/concurrent-reference-hash-map-1.1.0.jar,file:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/io/get-coursier/coursier-util_2.13/2.1.24/coursier-util_2.13-2.1.24.jar,file:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/io/github/alexarchambault/is-terminal/0.1.1/is-terminal-0.1.1.jar,file:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/io/get-coursier/jniutils/windows-jni-utils/0.3.3/windows-jni-utils-0.3.3.jar,file:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/codehaus/plexus/plexus-archiver/4.10.0/plexus-archiver-4.10.0.jar,file:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/codehaus/plexus/plexus-container-default/2.1.1/plexus-container-default-2.1.1.jar,file:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/virtuslab/scala-cli/config_2.13/1.1.3/config_2.13-1.1.3.jar,file:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/io/get-coursier/cache-util/2.1.24/cache-util-2.1.24.jar,file:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/ow2/asm/asm/9.6/asm-9.6.jar,file:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/ow2/asm/asm-commons/9.6/asm-commons-9.6.jar,file:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/apache/commons/commons-lang3/3.14.0/commons-lang3-3.14.0.jar,file:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/javax/inject/javax.inject/1/javax.inject-1.jar,file:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/codehaus/plexus/plexus-utils/4.0.1/plexus-utils-4.0.1.jar,file:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/codehaus/plexus/plexus-io/3.5.0/plexus-io-3.5.0.jar,file:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/commons-io/commons-io/2.16.1/commons-io-2.16.1.jar,file:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/apache/commons/commons-compress/1.26.2/commons-compress-1.26.2.jar,file:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/io/airlift/aircompressor/0.27/aircompressor-0.27.jar,file:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/tukaani/xz/1.9/xz-1.9.jar,file:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/com/github/luben/zstd-jni/1.5.6-3/zstd-jni-1.5.6-3.jar,file:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/codehaus/plexus/plexus-classworlds/2.6.0/plexus-classworlds-2.6.0.jar,file:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/apache/xbean/xbean-reflect/3.7/xbean-reflect-3.7.jar,file:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/virtuslab/scala-cli/specification-level_2.13/1.1.3/specification-level_2.13-1.1.3.jar,file:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/ow2/asm/asm-tree/9.6/asm-tree-9.6.jar,file:/Users/lihaoyi/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/commons-codec/commons-codec/1.17.0/commons-codec-1.17.0.jar /Users/lihaoyi/Github/mill/out/integration/invalidation/selective-execution/local/fork/test.dest/testargs stderr thread" #41 prio=5 os_prio=31 cpu=1.45ms elapsed=22.00s tid=0x000000014c808200 nid=0xb403 runnable  [0x0000000173b8a000]
   java.lang.Thread.State: RUNNABLE
	at java.io.FileInputStream.readBytes([email protected]/Native Method)
	at java.io.FileInputStream.read([email protected]/FileInputStream.java:276)
	at java.io.BufferedInputStream.fill([email protected]/BufferedInputStream.java:244)
	at java.io.BufferedInputStream.read1([email protected]/BufferedInputStream.java:284)
	at java.io.BufferedInputStream.read([email protected]/BufferedInputStream.java:343)
	- locked <0x0000000617e08c70> (a java.lang.ProcessImpl$ProcessPipeInputStream)
	at java.io.FilterInputStream.read([email protected]/FilterInputStream.java:106)
	at mill.main.client.InputPumper.run(InputPumper.java:48)
	at java.lang.Thread.run([email protected]/Thread.java:840)

@lihaoyi
Copy link
Member Author

lihaoyi commented Feb 16, 2025

The issue appears to be the Mill subprocess spawned in the test in

Future {
eval(
("--watch", "{foo.fooCommand,bar.barCommand}"),
check = true,
stdout = os.ProcessOutput.Readlines(line => output0 = output0 :+ line),
stderr = os.Inherit
)
}

In client-server mode, when the test is done, the process is terminated by the IntegrationTester#close code in

override def close(): Unit = {
if (clientServerMode) {
// try to stop the server
os.call(
cmd = (millExecutable, "--no-build-lock", "shutdown"),
cwd = workspacePath,
stdin = os.Inherit,
stdout = os.Inherit,
stderr = os.Inherit,
env = millTestSuiteEnv,
check = false
)
}
removeServerIdFile()
}

However in fork mode, there isn't any mechanism to terminate the Mill subprocess, which continues running in that Future indefinitely.

This probably only become an issue in #4456 due to subtle changes about how the Jvm.* APIs deal with leaked subprocesses; probably before we were just leaking, and now we are instead hanging waiting for the leaked subprocess to complete. Somehow, now the TestRunnerMain subprocess is exiting, but the child MillMain process that it spawned remains hanging around, and this causes the parent MillMain process to also hang around. But still haven't figured out the exact cause

@lihaoyi
Copy link
Member Author

lihaoyi commented Feb 16, 2025

One bug is the bincompat forwarders upstream in OS-Lib have the wrong default set for destroyOnExit. This should be fixed separately, but even passing in destroyOnExit = true explicitly in IntegrationTester#eval doesn't seem to help

https://github.com/com-lihaoyi/os-lib/blob/3aea8be1b75b564a06dfdd1ab9bab5acb4a06c77/os/src/ProcessOps.scala#L74

@lihaoyi
Copy link
Member Author

lihaoyi commented Feb 16, 2025

Upstream fix for OS-Lib com-lihaoyi/os-lib#357

@lihaoyi
Copy link
Member Author

lihaoyi commented Feb 18, 2025

Seems the underlying problem was bugs in destroyOnExit, which when fixed in com-lihaoyi/os-lib#359 seems to make things pass

@lihaoyi lihaoyi closed this Feb 18, 2025
@lihaoyi lihaoyi reopened this Feb 18, 2025
@lihaoyi lihaoyi merged commit 1faa3e3 into com-lihaoyi:main Feb 18, 2025
39 checks passed
lihaoyi added a commit that referenced this pull request Feb 19, 2025
…, use it in integration test cleanup (#4587)

This attempts to fix some of the residual flakiness encountered in
#4570

Somehow the recursive process termination we pulled in from upstream
com-lihaoyi/os-lib#359 only works some of the
time, and other times the processes get leaked. This PR extends the
`serverId`-file-deleted shutdown logic we already used for client-server
mode and enables it for `--no-server` mode as well. This lets us put an
additional guardrail in our `IntegrationTester#close` to delete any such
files, to try and force any Mill processes to terminate.

Added an additional integration test to exercise this behavior

Also fixed a bug in `ExampleTester` not honoring the `clientServerMode`
flag, and update `testkit.test` to assert on those behaviors
@lefou lefou added this to the 0.13.0 milestone Mar 2, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants