calling into_async_read panics whereas into_stream works well

[bmatthieu3]

Hi,

I encounter some issues calling into_async_read on a ReadableStream:

Uncaught (in promise) Error: already locked to a reader, or not a readable byte stream

On the other hand, calling into_stream on a ReadableStream works perfectly and allows me to iterate over the chunk of data (as like in the example of the repo).

Here is my rust code:

        wasm_bindgen_futures::spawn_local(async move {
            use wasm_streams::ReadableStream;
            use web_sys::Response;
            use web_sys::window;
            use wasm_bindgen_futures::JsFuture;
            use futures::StreamExt;
            use wasm_bindgen::JsCast;
            use futures::TryStreamExt;
    
            let window = window().unwrap_throw();
            let resp_value = JsFuture::from(window.fetch_with_str(&url))
                .await
                .map_err(|_| "fetch failed").unwrap_throw();
            let resp: Response = resp_value.dyn_into().unwrap_throw();
        
            // Get the response's body as a JS ReadableStream
            let raw_body = resp.body().unwrap_throw();
            let body = ReadableStream::from_raw(raw_body.dyn_into().unwrap_throw());

            // Converting to an IntoAsyncRead panics
            let mut async_read = body.into_async_read();
            // While converting the JS ReadableStream to a Rust stream does work
            //let mut stream = body.into_stream();
            // Consume the stream, logging each individual chunk
            /*while let Some(Ok(chunk)) = stream.next().await {
                al_core::log(&format!("{:?}", chunk));
            }*/
        });

Thank you very much for your help,

[MattiasBuelens]

Readable byte stream support in fetch() is still very new. The standard was only recently updated in whatwg/fetch#1593, and at the time of writing only Firefox supports it. So this example won't work in e.g. Chrome, which I assume is what you're testing in.

Luckily, the futures crate has a solution for this. You can convert any Stream into an AsyncRead using TryStreamExt::into_async_read. You can use that as a fall back when ReadableStream::try_into_async_read() returns an error. Note that you must first convert the raw JsValues into Uint8Arrays and then into &[u8]s before you can call into_async_read.

Here's a more complete example:

use futures_util::future::Either;

// Convert the JS ReadableStream to a Rust AsyncRead
let mut async_read = match body.try_into_async_read() {
    Ok(async_read) => Either::Left(async_read),
    Err((_err, body)) => Either::Right(
        body
            .into_stream()
            .map_ok(|js_value| js_value.dyn_into::<Uint8Array>().unwrap_throw().to_vec())
            .map_err(|_js_error| std::io::Error::new(std::io::ErrorKind::Other, "failed to read"))
            .into_async_read(),
    ),
};

// Consume the reader
let mut buf = [0u8; 4];
assert_eq!(async_read.read(&mut buf).await.unwrap_throw(), 4);
assert_eq!(&buf, b"\x89PNG");

[bmatthieu3]

Thank you for your complete answer, I was not aware about how new it is.
Concerning the conversion from a JS readable stream to a rust stream body.into_stream(), do you know if it is currently compatible with most of the browsers (safari, chrome mainly) ?

Maybe I should provide a way, if it is not the case, to use the older non async JS methods FileReader.readAsArrayBuffer().

[MattiasBuelens]

Concerning the conversion from a JS readable stream to a rust stream body.into_stream(), do you know if it is currently compatible with most of the browsers (safari, chrome mainly) ?

ReadableStream::into_stream() works in all browsers. Behind the scenes, it uses a ReadableStreamDefaultReader, which is supported everywhere. So you shouldn't need to fall back to readAsArrayBuffer(). 🙂

Maybe I should put that fallback for into_async_read() into wasm-streams itself. Then you don't need to worry about browser support at all... 🤔

[bmatthieu3]

Thanks for your answer. I think I will close the issue :)

For putting it inside wasm-streams I do not know. That trick seems to not be needed anymore in the future (because of a better support from chrome and other browsers) so wasm-streams will have to be changed to remove it again. But maybe that future is far from now so the change would make sense.

At least I would think an example with that trick for converting a JS stream to a Rust async reader would be great :)

[extrawurst]

@bmatthieu3 I run into this too in today's safari and it seem they are not planning to ever support BYOB. i found your polyfill. should that work aswell when running wasm using wasm-streams that use by default BYOB under the hood for into_async_read?

[MattiasBuelens]

@extrawurst The polyfill will work together with wasm-streams. However, the way the polyfill currently works, it won't "add" BYOB support to Safari's ReadableStream implementation. Instead, it'll completely replace the global ReadableStream class with its own version, and you'd first have to "convert" any native stream to a polyfilled stream using ReadableStream.from(nativeReadableStream). Also, in the current specification, ReadableStream.from() always returns a "default" readable stream (i.e. not a readable byte stream), so any converted stream won't support BYOB either... 😕

I would recommend not to use the polyfill in this case, and instead use the code snippet above. That will always get you a working AsyncRead, although it won't be truly zero-copy behind the scenes if the browser doesn't support BYOB.