[aurora-au] Update to version 0.4.0 and add test port

[JonLiu1993]

Fixes #41119 (comment)

Update aurora-au to the latest version 0.4.0

Tested usage successfully by aurora-au:x64-windows

• Changes comply with the maintainer guide.
• SHA512s are updated for each updated download.
• The "supports" clause reflects platforms that may be fixed by this new version.
• Any fixed CI baseline entries are removed from that file.
• Any patches that are no longer applied are deleted from the port's directory.
• The version database is fixed by rerunning ./vcpkg x-add-version --all and committing the result.
• Only one version is added to each modified port's versions file.

[chiphogg]

Thanks for putting this PR together so soon after the release --- very exciting! I had one question about whether we can get rid of the patch file.

[WangWeiLin-MV]

The port installation tests pass with the following triplets:

• x64-windows

[chiphogg]

We did just find one issue with 0.4.0. The impact is that a project using Au as a dependency, via CMake, will likely be broken. One of the new files wasn't added to CMakeLists.txt. See aurora-opensource/au#359 for more details.

Here is a patch file that should fix the issue: https://github.com/chiphogg/conan-center-index/blob/62558ad3b32f5c742c588123bc3e1ec2ff254b73/recipes/au/all/patches/0001-v0.4.0-add-static-cast-checkers-to-cmake.patch

Is there any kind of "automated package test" we could add that would catch this? The conan recipe has a "test package" that acts as a small client library, and builds some simple code. (It's how I caught this error in the first place.) If we can't do this in an automated fashion, then the next best thing would be some kind of manual version. In either case, I'd be surprised if we don't need to add this patch!

Sorry about this issue! It made me realize that although we had CMake tests in our CI, we didn't have any tests for client libraries using CMake. We've now closed this testing gap with a new CI job, and I confirmed that it would have caught this error, so we should be in better shape going forwards.

And at least we caught it before the vcpkg update landed!

[andre-nguyen]

@JonLiu1993 Thank you for taking the time to do the update to 0.4

@chiphogg afaik its basically like a unit test, there isn't really a way to automatically test all of the exposed package functionality.

[chiphogg]

@chiphogg afaik its basically like a unit test, there isn't really a way to automatically test all of the exposed package functionality.

I'm sorry --- I wasn't able to understand what you were saying here. 😅 What is "basically like a unit test"? The tests that this comment referred to?

What I was getting at was that we should make sure that a vcpkg project that uses Au 0.4.0 can successfully include "au/au.hh" and compile, before we land this PR. I expect this to fail in the current state, but pass if we add a patch that's equivalent to the one I linked. Of course it'd be great if we could automate it, but I think the most important thing is to simply perform this check by any means, manual or automatic.

[dg0yt]

This kind of testing can be implemented as a test port. Example:
https://github.com/microsoft/vcpkg/tree/master/scripts/test_ports/vcpkg-ci-blas

[JavierMatosD]

@JonLiu1993 in the meantime, it looks like we should add the patch that @chiphogg is suggesting. Either that or wait for upstream to mint a new release.

[chiphogg]

This kind of testing can be implemented as a test port. Example: https://github.com/microsoft/vcpkg/tree/master/scripts/test_ports/vcpkg-ci-blas

That looks like exactly what we would want! I tried copying the pattern, but I have no idea how to run it or test whether it works:
cea4949

Maybe somebody who knows how to run these CI jobs can try running this test package. The expected outcome is that it should fail because static_cast_checkers.hh can't be found: that would be good. Once we can get to that point, we can add the patch file I shared earlier, and that should be enough to get to a passing state. Then the CI job and patch can all be landed as part of this current PR.

[dg0yt]

These test ports are included via --overlay-ports=scripts/test_ports.

So:

vcpkg install --overlay-ports=scripts/test_ports vcpkg-ci-aurora-au

[JonLiu1993]

@chiphogg, Thanks for the patch, but when I add the vcpkg-ci-aurora-au test port locally, make the patch you provided, and run the command vcpkg install --overlay-ports=scripts/test_ports vcpkg-ci-aurora-au
I will encounter a new error:

PS F:\test\vcpkg> ./vcpkg install --overlay-ports=scripts/test_ports vcpkg-ci-aurora-au
Computing installation plan...
The following packages will be built and installed:
  * aurora-au:[email protected]
    vcpkg-ci-aurora-au:x64-windows@ci -- F:\test\vcpkg\scripts/test_ports\vcpkg-ci-aurora-au
Additional packages (*) will be modified to complete this operation.
Detecting compiler hash for triplet x64-windows...
Compiler found: C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.41.34120/bin/Hostx64/x64/cl.exe
Restored 0 package(s) from C:\Users\v-zhli17\AppData\Local\vcpkg\archives in 269 us. Use --debug to see more details.
Installing 1/2 aurora-au:[email protected]...
Building aurora-au:[email protected]...
-- Using cached aurora-opensource-au-0.4.0.tar.gz.
-- Extracting source F:/test/vcpkg/downloads/aurora-opensource-au-0.4.0.tar.gz
-- Applying patch fix-usage.patch
-- Using source at F:/test/vcpkg/buildtrees/aurora-au/src/0.4.0-d8793eab4a.clean
-- Found external ninja('1.11.0').
-- Configuring x64-windows
-- Building x64-windows-rel
-- Installing: F:/test/vcpkg/packages/aurora-au_x64-windows/share/aurora-au/copyright
-- Installing: F:/test/vcpkg/packages/aurora-au_x64-windows/share/aurora-au/usage
-- Performing post-build validation
Stored binaries in 1 destinations in 1 s.
Elapsed time to handle aurora-au:x64-windows: 7.9 s
aurora-au:x64-windows package ABI: 027788a1455d9076f8483a48a2b2088a16a8b3d8dc1fe06dbd97ea0dfca26383
Installing 2/2 vcpkg-ci-aurora-au:x64-windows@ci...
Building vcpkg-ci-aurora-au:x64-windows@ci...
F:\test\vcpkg\scripts/test_ports\vcpkg-ci-aurora-au: info: installing overlay port from here
-- Using cached msys2-mingw-w64-x86_64-pkgconf-1~2.3.0-1-any.pkg.tar.zst.
-- Using cached msys2-msys2-runtime-3.5.4-2-x86_64.pkg.tar.zst.
-- Using msys root at F:/test/vcpkg/downloads/tools/msys2/21caed2f81ec917b
-- Found external ninja('1.11.0').
-- Configuring x64-windows
CMake Warning at installed/x64-windows/share/vcpkg-cmake/vcpkg_cmake_configure.cmake:344 (message):
  The following variables are not used in CMakeLists.txt:

      PKG_CONFIG_EXECUTABLE

  Please recheck them and remove the unnecessary options from the
  `vcpkg_cmake_configure` call.

  If these options should still be passed for whatever reason, please use the
  `MAYBE_UNUSED_VARIABLES` argument.
Call Stack (most recent call first):
  scripts/test_ports/vcpkg-ci-aurora-au/portfile.cmake:5 (vcpkg_cmake_configure)
  scripts/ports.cmake:196 (include)


-- Building x64-windows-dbg
CMake Error at scripts/cmake/vcpkg_execute_build_process.cmake:134 (message):
    Command failed: F:/test/vcpkg/downloads/tools/cmake-3.30.1-windows/cmake-3.30.1-windows-i386/bin/cmake.exe --build . --config Debug -- -v -j9
    Working Directory: F:/test/vcpkg/buildtrees/vcpkg-ci-aurora-au/x64-windows-dbg
    See logs for more information:
      F:\test\vcpkg\buildtrees\vcpkg-ci-aurora-au\build-x64-windows-dbg-out.log

Call Stack (most recent call first):
  installed/x64-windows/share/vcpkg-cmake/vcpkg_cmake_build.cmake:74 (vcpkg_execute_build_process)
  scripts/test_ports/vcpkg-ci-aurora-au/portfile.cmake:10 (vcpkg_cmake_build)
  scripts/ports.cmake:196 (include)


error: building vcpkg-ci-aurora-au:x64-windows failed with: BUILD_FAILED
See https://learn.microsoft.com/vcpkg/troubleshoot/build-failures?WT.mc_id=vcpkg_inproduct_cli for more information.
Elapsed time to handle vcpkg-ci-aurora-au:x64-windows: 2.7 s

Change Dir: 'F:/test/vcpkg/buildtrees/vcpkg-ci-aurora-au/x64-windows-dbg'

Run Build Command(s): "C:/Program Files/Microsoft Visual Studio/2022/Community/Common7/IDE/CommonExtensions/Microsoft/CMake/Ninja/ninja.exe" -v -v -j9
[1/2] C:\PROGRA~1\MIB055~1\2022\COMMUN~1\VC\Tools\MSVC\1441~1.341\bin\Hostx64\x64\cl.exe   /TP  -external:IF:\test\vcpkg\installed\x64-windows\include -external:W0 /nologo /DWIN32 /D_WINDOWS /utf-8 /GR /EHsc /MP  /D_DEBUG /MDd /Z7 /Ob0 /Od /RTC1  -MDd /showIncludes /FoCMakeFiles\test_au_client.dir\test_au_client.cc.obj /FdCMakeFiles\test_au_client.dir\ /FS -c F:\test\vcpkg\scripts\test_ports\vcpkg-ci-aurora-au\project\test_au_client.cc
FAILED: CMakeFiles/test_au_client.dir/test_au_client.cc.obj 
C:\PROGRA~1\MIB055~1\2022\COMMUN~1\VC\Tools\MSVC\1441~1.341\bin\Hostx64\x64\cl.exe   /TP  -external:IF:\test\vcpkg\installed\x64-windows\include -external:W0 /nologo /DWIN32 /D_WINDOWS /utf-8 /GR /EHsc /MP  /D_DEBUG /MDd /Z7 /Ob0 /Od /RTC1  -MDd /showIncludes /FoCMakeFiles\test_au_client.dir\test_au_client.cc.obj /FdCMakeFiles\test_au_client.dir\ /FS -c F:\test\vcpkg\scripts\test_ports\vcpkg-ci-aurora-au\project\test_au_client.cc
F:\test\vcpkg\installed\x64-windows\include\au/constant.hh(59): error C2131: expression did not evaluate to a constant
F:\test\vcpkg\installed\x64-windows\include\au/constant.hh(59): note: failure was caused by a read of a variable outside its lifetime
F:\test\vcpkg\installed\x64-windows\include\au/constant.hh(59): note: see usage of 'u'
F:\test\vcpkg\installed\x64-windows\include\au/constant.hh(59): note: the template instantiation context (the oldest one first) is
F:\test\vcpkg\scripts\test_ports\vcpkg-ci-aurora-au\project\test_au_client.cc(14): note: see reference to function template instantiation 'auto au::Constant<au::detail::SpeedOfLightUnit>::as<int,au::SymbolFor<au::UnitProduct<T,au::Pow<B,-1>>>>(OtherUnit) const' being compiled
        with
        [
            T=au::Meters,
            B=au::Seconds,
            OtherUnit=au::SymbolFor<au::UnitProduct<au::Meters,au::Pow<au::Seconds,-1>>>
        ]
ninja: build stopped: subcommand failed.

Could you please take a look? thanks!

[chiphogg]

Could you please take a look? thanks!

Yep, I think I made some progress --- thanks @dg0yt for showing me how to run the CI script!

At some point, I felt like I was getting stale results, so to be sure my cache was clear, I ran rm -Rf installed/* from the base vcpkg directory. A little clumsy, perhaps, but after I did this, everything was reliably reflecting the latest changes. So, assume that I did this between every step.

Now for the progress.

First, my copy-paste approach had an error, that would have prevented the CI package from working even if there wasn't a library error. I had copied something that evidently, we didn't need and didn't want. I fixed that in chiphogg@c9900e2.

At this point, when I ran the command, I got the hoped-for error about including "au/static_cast_checkers.hh" (to see the error cat the logfile that it prints out). I brought in the patch (chiphogg@7544fba), with some slight tweaks to match the format of the previous vcpkg patch, and re-ran. Success!

Summary:

• Applying chiphogg@c9900e2 and running rm -Rf installed/* && vcpkg install --overlay-ports=scripts/test_ports vcpkg-ci-aurora-au demonstrates the desired error.
• Applying chiphogg@7544fba and running rm -Rf installed/* && vcpkg install --overlay-ports=scripts/test_ports vcpkg-ci-aurora-au produces a successful outcome.

[chiphogg]

Also, FWIW, this was all run on Linux. From the error above, it looks like you were running the command on Windows. I've found an MSVC-only bug based on the compiler error you shared.

This should only affect constants... which I just happened to pick for my little "demonstration". 😅 Applying chiphogg@894bb23 should give you a more MSVC-friendly test for 0.4.0.

I'll make sure we add MSVC coverage for Constant, and fix this for 0.4.1.

[dg0yt]

At some point, I felt like I was getting stale results, so to be sure my cache was clear, I ran rm -Rf installed/* from the base vcpkg directory.

Well, there is also a binary package cache, in addition to the installed tree. There is a --no-binarycaching option, and more.

[chiphogg]

Thanks for the latest round of updates, @JonLiu1993!

It looks like the builds are failing. The logs on the azure dev jobs don't say why; they direct the reader to another log file, which I don't see how to access.

Regardless, this isn't entirely unexpected: the current version of the test hits a newly-found MSVC-only bug, which is in 0.4.0 and won't be fixed until 0.4.1. If you try making these changes (chiphogg@894bb23) to the test, would the remote jobs pass then?

[chiphogg]

I just remembered that we can make suggestions as comments, which you can then commit directly! Let's see if this will fix the remote builds (it's the changes from chiphogg@894bb23).

Suggested change

	#include "au/constants/speed_of_light.hh"
	#include "au/io.hh"
	#include "au/units/meters.hh"
	#include "au/units/seconds.hh"
	using ::au::SPEED_OF_LIGHT;
	using ::au::symbols::m;
	using ::au::symbols::s;
	int main(int argc, char **argv) {
	std::cout << "Speed of light in m/s: " << SPEED_OF_LIGHT.as<int>(m / s)
	<< std::endl;
	return 0;
	}
	#include "au/io.hh"
	#include "au/units/hours.hh"
	#include "au/units/meters.hh"
	#include "au/units/miles.hh"
	using ::au::symbols::h;
	using ::au::symbols::mi;
	constexpr auto km = ::au::kilo(::au::symbols::m);
	int main(int argc, char **argv) {
	constexpr auto v = 65.0 * mi / h;
	std::cout << v << ", in km/h, rounded to nearest integer, is "
	<< round_as(km / h, v) << std::endl;
	return 0;
	}

[matthew-reynolds]

@chiphogg @JonLiu1993 Does it make sense to just skip 0.4.0 and update this PR to target the newly-released 0.4.1 rather than trying to carry patches on top of the official 0.4.0 release?

[chiphogg]

@chiphogg @JonLiu1993 Does it make sense to just skip 0.4.0 and update this PR to target the newly-released 0.4.1 rather than trying to carry patches on top of the official 0.4.0 release?

This present PR, taking us to 0.4.0, is now ready to land immediately AFAICS. I think it makes sense to land it as-is, both because the work is already done (and the CI is green), and because it adds layers of protection (the new CI tests) that will help ensure things work smoothly going forward.

I've been planning to make a follow-on PR myself to take us from 0.4.0 to 0.4.1. That's still my plan. Should be fun! 🙂

[WangWeiLin-MV]

The port installation tests pass with the following triplets:

• x64-windows