Unit Testing Best Practices?

[tifroz]

In a pure (modern) Xcode project, on would write Swift Tests or XCTests suites with injected mock dependencies using a dependency-injection library like swift-dependencies to avoid unwanted side effects (hitting servers...) + provide consistency + control the scope of the test.

When unit-testing native skip modules

This approach isn't an option because the XCTest suite is transpiled, and no dependency-injection library will travel across transpiled / native boundaries.

Possible workaround strategies:

• Old fashioned direct dependency injection via parameters (tons of boilerplate in the feature code)
• XCTest suite calls dedicated test method that injects dependencies before calling the feature method to be tested (brittle, tons of boilerplate - but at least boilerplate doesn't affect feature code)
• Only unit-test on iOS platform, hope that everything works the same and the android platform (no parity testing)

When unit-testing transpiled modules,

The modern approach is not an option either because I am not aware of a transpiled dependency-injection library

Possible workaround strategies:

• Old fashioned direct dependency injection via parameters (tons of boilerplate in the feature code)
• Roll your own dependency transpiled injection library (ugh)

Question

Has anyone figured out the a strategy mix that works for them? (we just started a new project and would like to nail the unit-testing early if we can)

[marcprux]

That's a great question, and you outlined the current considerations and tradeoffs very well. I unfortunately don't have a great solution to dependency injection for you, but I'll add that the testing limitations do vary depending on what mode you are using Skip in.

Skip Lite (transpiled)

As you point out (and as mentioned in our testing docs), Skip Lite currently only supports legacy XCTest style test cases, and not the newer Testing framework. This could be something we enhance in the future, but we'll be fairly limited by the fact that Skip's transpilation doesn't support macros, which the Testing framework uses heavily. XCTest, OTOH, can convert to Android JUnit tests in a fairly straightforward way.

In pure Skip Lite mode, you would probably have to come up with your own dependency injection solution; we would be interested in collaborating on something that could be used by the wider community if you have any ideas you would like the share about the best way to implement it.

Skip Fuse (compiled)

Compiled (Fuse) mode is a different story. By default, a compiled package created will Skip will still use transpiled mode for the testing, and set up the module to enable bridging. This can actually be useful for testing out the Swift-to-Kotlin bridging side of your project (in case that is something you are using), but still suffers from the limitations that transpiled testing is limited to XCTest-style test cases. Take a look at https://github.com/skiptools/skip-bridge/tree/main/Tests for some of the variations of testing that can be accomplished with native modules + transpiled testing.

Native Android Testing

However, if the module you are testing is a pure-native module and doesn't require any Java/Kotlin bridging, then you could simply use native SwiftPM testing along with the Testing framework, rather than Skip's test harness. Skip supports running these kinds of tests on your Android device or emulator using skip android test (as opposed to the transpiled tests with skip test; yes, this is confusing). Running tests in this mode will be able to use any native dependencies you want (possibly including swift-dependencies, which allegedly builds for Android), and you will be free of any of Skip's testing constraints. This is the solution we use for https://github.com/skiptools/swift-android-native, which you will notice doesn't actually have any dependency on Skip at all, but tests can still be run with skip android test (which is just a fancy front-end to SwiftPM+adb; observe skip android test --verbose to witness the sausage-making).

The downside to this mode is that skip android test (or using the CI unit testing in https://github.com/marketplace/actions/swift-android-action) is that tests are run against "bare metal" with adb shell and not in the context of an actual Android app, which means that any integration with any of your Java/Kotlin dependencies will simply not work (as there is no ambient Java environment or other Android APK affordances). This is a limitation that we are aware of and would like to rectify soon.

Hopefully this provides a bit more clarity to the landscape of cross-platform testing with Skip. We'd love to chat more about this, since testing is a passion of ours 😊

[tifroz]

Thanks! Your Native Android section was eye-opening.

I am guessing (hoping) that our app and most real-world apps will have few transpiled modules, which will be mostly leaf modules with few dependencies.

If so, we can get really good unit-test coverage & minimal downside
1/ Using native android testing + swift test to unit-test our native modules, injecting mock dependencies to substitute for dependencies that require an android runtime.
2/ Selective parameter-injection for transpiled modules

I'll have to experiment a bit so I can better understand how practical that would be in the real world. Will report back!