Use CoreFoundation from Swift?

[jeanthom]

Hi,

You can find a CoreFoundation implementation inside Swift sources (which was forked by the PD folks): https://github.com/PureDarwin/CoreFoundation

It looks more complete than the one currently integrated in airyx. Should we migrate to this CoreFoundation or is there a reason for keeping using "ours"?

Also, can we reuse code licensed under the ASPL license?

[mszoek]

Awesome, thanks for the pointer! I'll have to read the APSL carefully but it is probably fine. There are other CF implementations as well that are more complete/recent than what I have now.

Do you think you could try to get it to build against the Airyx frameworks?

[eleanxr]

I did a quick investigation, and I think the following will work:

1. Address the first item noted in Swift build on FreeBSD #209 (comment) since the CF implementation depends on libdispatch as well.
2. Decouple the Swift-dependent portions and CF portions of the CMake build in swift-corelibs-foundation (quick POC I tested on Linux just to verify the CF portion makes it through CMake+build without Swift available: https://github.com/deleanor/swift-corelibs-foundation/tree/poc-noswift-build). Probably need the second patch noted in Swift build on FreeBSD #209 as well. Long term, I know Swift is desired, but this might decouple the two issues.

As long as there are no objections, I'll see if I can get libdispatch going with the (unmerged) patch noted in #209 next.

[eleanxr]

I managed to get the swift CF build to complete. I pulled the CoreFoundation subdirectory from swift-corelibs-foundation repository swift-5.7-RELEASE tag into ravynos/Frameworks and made my changes on top of that base so there'd be a clean diff. I need to go through the diff and re-evaluate everything now that I've made it through the build once, but I think many of the changes are OK. There are probably some I can simplify or remove given what I know now.

I did not end up changing ravynos's libdispatch as suggested in #209 . I think that still might be good to do, but with a complete CF in place, that can proceed independently of other things that require CF. I did use the patch against swift-corelibs-foundation suggested in #209 (swiftlang/swift-corelibs-foundation#3004) to implement CF RunLoop with Kevent. That PR suggests the referenced dispatch patch, but does not require it. Note that even though that PR says it was merged, it was later reverted by Apple because it caused tests to timeout on Windows.

Here is the (certainly incomplete) list of things I still need to do:

1. Go through the diffs and commits and try and clean up. There are a few changes I marked as questionable that I need to re-evaluate. Some of the macro guards could be cleaned up with changes to some of the platform macros in include/TargetConditionals.h (maybe), and some might need to be changed to be more declarative.
2. It'd be nice to locate an appropriate test suite for CF. swift-corelibs-foundation ships with a test suite for Foundation (but not CF), but it's implemented in Swift, so we'd need Swift to run even that. If someone else knows where an existing test suite is, that'd be much appreciated.
3. Get the other frameworks building against CF and get it to install to /System/Library/Frameworks properly.

Not sure if this was the right place to record this, but I wanted to write it down while it's still in my head.

Work-in-progress branch at https://github.com/eleanxr/ravynos/tree/swift-cf-RunLoopWithKevent but the actual changes against upstream are easier to see when diffed against the import-swift-cf-swift-5.7-RELEASE tag. I'm definitely open to any feedback on something I should be doing differently or missing here, but it's definitely not ready for a PR yet.

[mszoek]

This is fantastic progress! Thanks so much for taking it on.

[eleanxr]

All of the Frameworks and CoreServices build successfully now, and I'm tracking down a few runtime undefined dynamic symbol errors that occur on application startup before we get to main in the services that use the frameworks. However, once I complete that I'm not sure how to best test the system. AFAICT, WindowServer and bin/open are the main things that rely on the frameworks at the moment, and I don't currently have hardware that can host the UI (I've just been working in a tty on a VM). So I can make sure we get to main in those, but I can't really test anything after that. Are there any other suggested tests I can run to just make sure some basic things that are expected to work still work?

[lin72h]

So exciting to know your progress. I'd happy to test it once your PR get merged, I've already found the internal libdispatch has some bugs need to test too

[eleanxr]

I was able to debug further than I thought I'd be able to be without starting the GUI and am working through a number of runtime errors now during WindowServer startup. It's looking like we're going to need a Foundation that supports bridged CF types, so I'm trying to decide if it's better to try and adapt PureDarwin's Foundation (which has an incomplete set of bridged types that at least somewhat work against Swift's CF implementation) or adapt the current (Cocotron?) Foundation implementation to support bridged types. The main driver for this is that Foundation types are propagating down into CF, and CF expects them to be castable to CFTypeRef. I'm still learning about how bridging works, so if someone believes this is the wrong path, or I'm misunderstanding something, please feel free to let me know!

[eleanxr]

Rather than taking on a major change to introduce bridging into Foundation as part of this issue, I was able to introduce a shim to use the Cocotron implementations of the CF functions called from Foundation that are expecting bridged types to get execution to make it further along. It's still probably a good idea to go back and evaluate bridging the CF types appropriately in a future issue, but it's looking like the shim layer successfully decouples this task from a major update to Foundation.

There are still more runtime errors to work through, but at least for now the shim is only needing to provide alternative implementations for 7 CF functions. The main ones are CFHash, CFRetain, and CFRelease. I separated these into a separate header and implementation file (Foundation/CFBaseShim.{hm}) to centralize them for future removal if/when necessary.

[mszoek]

This sounds amazing and I'm excited to give it a spin. Do you have a branch or fork I can build from?

[eleanxr]

Yep! It's on the swift-cf-RunLoopWithKevent branch in my fork of the ravynos repo: https://github.com/eleanxr/ravynos/tree/swift-cf-RunLoopWithKevent

Last night, I fixed a couple more things to get WindowServer to make it to the same place where it currently stops on the main branch for me, line 326 of WindowServer.m where it goes into the wait for /var/run/windowserver. Prior to that, it makes it through a number of CF and Foundation dependent operations, including bundle initializations.

I think I should be able to get a PR out within the next few days. I just want to go through and revisit a couple of implementation decisions I flagged to check out before submission and maybe clean up the commit history a little.

[eleanxr]

Just in case anyone checked out/forked the branch, I committed a major fix to CFRunLoop. At least a simple CFRunLoop program runs as expected now (https://github.com/eleanxr/ravynos-cf-test/blob/main/runloop.m).