feat: rustls-tls and rustls-native-roots feature flags

[c-thiel]

What changes are included in this PR?

Add rustls-tls and rustls-tls-native-roots" feature flags for rest catalog & core.
reqwest is not directly used in other crates

Are these changes tested?

No - just pass through a well-tested rustls functionality.

[liurenjie1024]

Hi, @c-thiel Could you elaborate what problem this pr is solving? I don't get much why we need this flag.

[c-thiel]

Hi @liurenjie1024,
by default reqwests uses the openssl library of the system (vendored openssl) to connect to servers via TLS / HTTPS.
This creates a runtime dependency on the underlying system to have a compatible openssl library installed.

rustls is rust-native implementation of openssl which removes this runtime dependency. This is very helpful to build platform agnostic binaries as well as building minimal docker images. Lakekeeper for example is build on the google distroless images stack. The minimal distroless incompatible is incompatible with iceberg-rust due to the missing runtime depdendency of openssl. To remove this runtime dependency, we currently hard-coded rustls-tls in our fork: https://github.com/lakekeeper/iceberg-rust/blob/a8b6509775b139c92772a999b0cbca637e274a03/Cargo.toml#L100

[Xuanwo]

I'm wondering if it would be better for us not to rely on reqwest with the default features enabled, and instead let users decide which features to include. This way, we wouldn't need to wrap those feature flags ourselves.

[c-thiel]

Hm, I see in the workspace we are already disabling default features for reqwest.
We still pulled the vendored-openssl in somehow last time I checked (~4 months ago). Let me double check if we actually need this.

[liurenjie1024]

Hi @liurenjie1024, by default reqwests uses the openssl library of the system (vendored openssl) to connect to servers via TLS / HTTPS. This creates a runtime dependency on the underlying system to have a compatible openssl library installed.

rustls is rust-native implementation of openssl which removes this runtime dependency. This is very helpful to build platform agnostic binaries as well as building minimal docker images. Lakekeeper for example is build on the google distroless images stack. The minimal distroless incompatible is incompatible with iceberg-rust due to the missing runtime depdendency of openssl. To remove this runtime dependency, we currently hard-coded rustls-tls in our fork: https://github.com/lakekeeper/iceberg-rust/blob/a8b6509775b139c92772a999b0cbca637e274a03/Cargo.toml#L100

Thanks @c-thiel for explaination. How about making rustls-tls and rustls-tls-native-roots default? This way our user don't rely on system openssl by default. We could add a feature for openssl.

[Xuanwo]

Thanks @c-thiel for explaination. How about making rustls-tls and rustls-tls-native-roots default? This way our user don't rely on system openssl by default. We could add a feature for openssl.

I don't need to need to introduce any of those flags. We are using default = false and users can tune features based on their own needs.

I think we can close this PR now.

[liurenjie1024]

I agree with @Xuanwo that user just need to add following to their Cargo.toml

[features]
default = ["reqwest/rustls-tls"]