TLS: "Bad Protocol Version" error with reqwest client, curl works great, axum tls rust-tls server, mkcert self-signed certs

[ptdecker]

I have exhasted all avenues I can find (including, ugh, ChatGPT) and am unable to solve an issue. I have created minimum implementations of both a client and server and am able to reproduce the issue. The issue does not occur with curl nor browser hitting an endpoint. But, reqwest bombs with a hyper issue. I'm unsure where to post this. Seems like reqwest discussion is the best place. I appreciate any and all help.

In the code below, I've redacted and adjusted some paths and such for privacy of my employeer.

The server is a minimal axum server with TLS serving up a single GET endpoint that returns a JSON payload.

Cargo.toml

[package]
name = "test-server"
version = "0.1.0"
edition = "2021"

[dependencies]
axum = { version = "0.7.5", features = ["macros"] }
serde = { version = "1.0.209", features = ["derive"] }
axum-server = { version = "0.7.1", features = ["tls-rustls", "rustls-pemfile"] }
tokio = { version = "1.40.0", features = ["full"] }

main.rs

use axum::{routing::get, Router, Json};
use serde::{Deserialize, Serialize};
use std::{net::SocketAddr, path::PathBuf};
use axum_server::tls_rustls::RustlsConfig;

#[derive(Serialize, Deserialize)]
struct Account {
    id: u32,
    name: String,
}

async fn get_account() -> Json<Account> {
    Json(Account {
        id: 12,
        name: "Patricia".to_string(),
    })
}

#[tokio::main]
async fn main() {
    let app = Router::new().route("/api/accounts/mocked/12", get(get_account));


    let config = RustlsConfig::from_pem_file(
        PathBuf::from("[redacted]/local.pem"),
        PathBuf::from("[redacted]/local-key.pem"),
    )
        .await
        .unwrap();

    let addr = SocketAddr::from(([127, 0, 0, 1], 3002));

    axum_server::bind_rustls(addr, config)
        .serve(app.into_make_service())
        .await
        .unwrap();
}

The client is a minimal reqwest-based client hard coded to hit that endpoint.

Cargo.toml:

[package]
name = "test-client"
version = "0.1.0"
edition = "2021"

[dependencies]
reqwest = { version = "0.12.7", features = ["json", "rustls-tls"] }
tokio = { version = "1.40.0", features = ["full"] }

main.rs:

use reqwest::Client;
use std::error::Error;

#[tokio::main]
async fn main() -> Result<(), Box<dyn Error>> {
    // Create a reqwest client with disabled certificate verification (for localhost)
    let client = Client::builder()
        .danger_accept_invalid_certs(true)
        .build()?;

    // Make the GET request to the specified endpoint
    let url = "https://localhost:3002/api/accounts/mocked/12";
    let response = client.get(url).send().await?;

    // Check if the request was successful
    if response.status().is_success() {
        // Print the JSON response
        let body = response.text().await?;
        println!("Response: {}", body);
    } else {
        eprintln!("Request failed with status: {}", response.status());
    }

    Ok(())
}

I'm using a self-signed certificate created with mkcert.

Curl works fine but the client bombs:

$ cd test-server
$ cargo run &
...

$ cd ../test-client
$ curl -v https://localhost:3002/api/accounts/mocked/12
* Host localhost:3002 was resolved.
* IPv6: ::1
* IPv4: 127.0.0.1
*   Trying [::1]:3002...
* Connected to localhost (::1) port 3002
* ALPN: curl offers h2,http/1.1
* (304) (OUT), TLS handshake, Client hello (1):
*  CAfile: /etc/ssl/cert.pem
*  CApath: none
* (304) (IN), TLS handshake, Server hello (2):
* (304) (IN), TLS handshake, Unknown (8):
* (304) (IN), TLS handshake, Certificate (11):
* (304) (IN), TLS handshake, CERT verify (15):
* (304) (IN), TLS handshake, Finished (20):
* (304) (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / AEAD-CHACHA20-POLY1305-SHA256 / [blank] / UNDEF
* ALPN: server accepted h2
* Server certificate:
*  subject: O=mkcert development certificate; [email protected] (Todd Decker)
*  start date: Mar 31 19:05:34 2023 GMT
*  expire date: Jul  1 19:05:34 2025 GMT
*  subjectAltName: host "localhost" matched cert's "localhost"
*  issuer: O=mkcert development CA; [email protected] (Todd Decker); CN=mkcert [email protected] (Todd Decker)
*  SSL certificate verify ok.
* using HTTP/2
* [HTTP/2] [1] OPENED stream for https://localhost:3002/api/accounts/mocked/12
* [HTTP/2] [1] [:method: GET]
* [HTTP/2] [1] [:scheme: https]
* [HTTP/2] [1] [:authority: localhost:3002]
* [HTTP/2] [1] [:path: /api/accounts/mocked/12]
* [HTTP/2] [1] [user-agent: curl/8.7.1]
* [HTTP/2] [1] [accept: */*]
> GET /api/accounts/mocked/12 HTTP/2
> Host: localhost:3002
> User-Agent: curl/8.7.1
> Accept: */*
>
* Request completely sent off
< HTTP/2 200
< content-type: application/json
< content-length: 329
< vary: origin, access-control-request-method, access-control-request-headers
< access-control-allow-origin: *
< date: Fri, 20 Sep 2024 22:32:53 GMT
<
* Connection #0 to host localhost left intact
{"id":1,"name":"Patricia"}

$ cargo run
...
Error: reqwest::Error { kind: Request, url: "https://localhost:3002/api/accounts/mocked/12", source: hyper_util::client::legacy::Error(Connect, Error { code: -9836, message: "bad protocol version" }) }

$

[seanmonstar]

The error code being negative hints to me it's from the OS. You have the default features of reqwest still on, and it will use the default tls if you don't tell the builder otherwise. Try .use_rustls_tls().

[ptdecker]

OMG! That was it! I guess I missed that in the docs. That will teach me to RTFM... TWICE!

Thank you

[ptdecker]

For other readers of this issue, it is pretty black and white in the docs right under the big orange box:

https://docs.rs/reqwest/0.12.7/reqwest/tls/index.html#default-tls

Since Cargo features are additive, other crates in your dependency tree can cause the default backend to be enabled. If you wish to ensure your Client uses a specific backend, call the appropriate builder methods (such as [use_rustls_tls()](https://docs.rs/reqwest/0.12.7/reqwest/struct.ClientBuilder.html#method.use_rustls_tls)).

[mominul]

I have also faced the same issues on a Macbook M1 Air (macOS Ventura 13.4.1) and using the .use_rustls_tls() method fixed it.

But why does this happen? It gives a bad experience and needs to be investigated. But have my heartiest thanks for developing and maintaining these crates @seanmonstar ❤️

[ptdecker]

Sean answered this here:

#2428 (comment)