not Send due to await retainment

[Mark-Simulacrum]

• root: updns-0.0.6: start v. end

[pnkfelix]

triage: P-high. Nominating for discussion.

[pnkfelix]

self-assigned for investigation in short term (as in, next two or three days). However I suspect someone from @rust-lang/wg-async-await will be a better choice to investigate this.

[nikomatsakis]

We should try to bisect this

[pnkfelix]

used cargo-bisect-rustc.

Determined:

searched nightlies: from nightly-2019-11-26 to nightly-2020-01-03
regressed nightly: nightly-2019-12-01
searched commits: from 25d8a94 to d8bdb3fdcbd88eb16\
e1a6669236122c41ed2aed3
regressed commit: 9081929

[pnkfelix]

9081929 is a rollup commit of the following PR's:

 - #66379 (Rephrase docs in for ptr)
 - #66589 (Draw vertical lines correctly in compiler error messages)
 - #66613 (Allow customising ty::TraitRef's printing behavior)
 - #66766 (Panic machinery comments and tweaks)
 - #66791 (Handle GlobalCtxt directly from librustc_interface query system)
 - #66793 (Record temporary static references in generator witnesses)
 - #66808 (Cleanup error code)
 - #66826 (Clarifies how to tag users for assigning PRs)
 - #66837 (Clarify `{f32,f64}::EPSILON` docs)
 - #66844 (Miri: do not consider memory allocated by caller_location leaked)
 - #66872 (Minor documentation fix)

I'm guessing this is due to PR #66793

[pnkfelix]

I'm still working on reducing this down to something that I could put into the playpen.

However, one interesting thing I've found locally during my reduction:

This version of the fn proxy will signal an analogous compilation error:

async fn proxy(buf: &[u8]) -> Result<Vec<u8>> {
    let tv = unsafe { TIMEOUT };
    let dur = Duration::from_millis(unsafe { TIMEOUT });
    timeout(Duration::from_millis(unsafe { TIMEOUT }),
            async { Result::Ok(vec![0u8]) })
        .await?
        ;

    Result::Ok(vec![0u8])
}

but if I lift the first argument out to a separate let-binding (i.e. change it to use dur above, or the sub-expression unsafe { TIMEOUT } out to a separate let-binding (i.e. change the argument expression to use tv above), then the error goes away.

I'll admit that I don't know all the details of how the fn-transformation works to support async fn, but I strongly suspect that if this kind of local code change can make an error go away, then that represents a weakness somewhere (either in the fn-transformation, or in the borrowck).

[pnkfelix]

Okay in the details block is a standalone version suitable for the playground:

#![crate_type="lib"]

use std::future::Future;
use std::io::Result;
use std::pin::Pin;
use std::task::{Context, Poll};
use std::time::Duration;

pub fn timeout<T>(_duration: Duration, _future: T) -> Timeout<T>
where
    T: Future,
{
    loop { }
}

#[derive(Debug)]
pub struct Elapsed(());

impl std::fmt::Display for Elapsed {
    fn fmt(&self, w: &mut std::fmt::Formatter) -> std::fmt::Result {
        write!(w, "Elapsed")
    }
}

impl std::error::Error for Elapsed {}

impl From<Elapsed> for std::io::Error {
    fn from(_err: Elapsed) -> std::io::Error {
        std::io::ErrorKind::TimedOut.into()
    }
}

pub struct Timeout<T> {
    _value: T,
}

impl<T> Future for Timeout<T>
where
    T: Future,
{
    type Output = std::result::Result<T::Output, Elapsed>;

    fn poll(self: Pin<&mut Self>, _: &mut Context<'_>) -> Poll<Self::Output> {
        loop { }
    }
}

static mut TIMEOUT: u64 = 2000;

use std::marker::PhantomData;

pub struct JoinHandle<T> {
    _p: PhantomData<T>,
}

pub fn spawn<T>(_task: T) -> JoinHandle<T::Output>
where
    T: Future + Send + 'static,
    T::Output: Send + 'static,
{
    loop { }
}

pub async fn my_main() {
    spawn(proxy());
}

// This version of `fn proxy` demonstrates the error
pub async fn proxy() -> Result<Vec<u8>> {
    timeout
        (Duration::from_millis(unsafe { TIMEOUT }),
         async { Result::Ok(vec![0u8]) })
        .await?
        .unwrap()
        ;

    Result::Ok(vec![0u8])
}

// This variant does not generate an error
pub async fn proxy_2() -> Result<Vec<u8>> {
    let dur = Duration::from_millis(unsafe { TIMEOUT });

    timeout
        (dur,
         async { Result::Ok(vec![0u8]) })
        .await?
        .unwrap()
        ;

    Result::Ok(vec![0u8])
}

// This variant does not generate an error
pub async fn proxy_3() -> Result<Vec<u8>> {
    let tv = unsafe { TIMEOUT };

    timeout
        (Duration::from_millis(tv),
         async { Result::Ok(vec![0u8]) })
        .await?
        .unwrap()
        ;

    Result::Ok(vec![0u8])
}

and the error it generates:

error: future cannot be sent between threads safely
  --> src/lib.rs:65:5
   |
56 | pub fn spawn<T>(_task: T) -> JoinHandle<T::Output>
   |        -----
57 | where
58 |     T: Future + Send + 'static,
   |                 ---- required by this bound in `spawn`
...
65 |     spawn(proxy());
   |     ^^^^^ future returned by `proxy` is not `Send`
   |
   = help: within `impl std::future::Future`, the trait `std::marker::Send` is not implemented for `*mut u64`
note: future is not `Send` as this value is used across an await
  --> src/lib.rs:70:5
   |
70 | /     timeout
71 | |         (Duration::from_millis(unsafe { TIMEOUT }),
   | |                                         ------- has type `*mut u64`
72 | |          async { Result::Ok(vec![0u8]) })
73 | |         .await?
   | |______________^ await occurs here, with `TIMEOUT` maybe used later
74 |           .unwrap()
75 |           ;
   |           - `TIMEOUT` is later dropped here

error: aborting due to previous error

[pnkfelix]

So this does seem like somewhat clear (expected?) fallout from PR #67611: a use of a static mut X: T in a generator witness will be recorded as a unsafe pointer *mut T, and then *mut T is never Send, but the spawn method requires that the task you give it is Send.

I believe we can readily update the crate in question to avoid using the static mut in the expression under the RHS of await.

But I do wonder whether we can/should improve the diagnostics here to try to guide the user towards such a solution ...?