Cargo git dependency downloading with broken path

[JasonHise]

I needed to use a private crate from a second crate, and decided to try using a dependency based on git instead of using a local path.

Cargo.toml:

[dependencies.my_private_library]
git = "https://github.com/MyUserName/my_private_library"
branch = "master"

Problem
When I attempted to build, I got this error:

error: failed to load source for a dependency on `my_private_library`
Caused by:
  Unable to update https://github.com/MyUserName/my_private_library#61bcf82d
Caused by:
  Could not find `Cargo.toml` in `C:\Users\my_user_name\.cargo\git\checkouts\my_private_library-0f9a102462bf3bba\my_private_library`

I opened up C:\Users\my_user_name\.cargo\git\checkouts\my_private_library-0f9a102462bf3bba and found that there was a subfolder named 61bcf82 instead of my_private_library.

Notes
Manually renaming the folder FIXED the issue locally, though a new copy of 61bcf82 appeared with duplicate files in the process.

[ehuss]

@JasonHise I'm unable to think of a way to reproduce your error. It is normal for Cargo to check out git projects into .cargo/git/checkouts/PKGNAME-HASH/SHORT_HASH/. Modifying cargo's internal directory structure may cause problems.

Can you create a reproduction? Is there anything special about my_private_library? Does it have its own git or path dependencies?

[JasonHise]

I've provided you access to the private library in question. https://github.com/JasonHise/associative_symbols The project that is using it is a static library with an exposed C API. Relevant parts of Cargo.toml: [lib] crate-type = ["staticlib"] [dependencies] libc = "0.2.51" [dependencies.associative_symbols] git = "https://github.com/JasonHise/associative_symbols" branch = "master"

…

On Wed, May 1, 2019 at 12:43 PM Eric Huss ***@***.***> wrote: @JasonHise <https://github.com/JasonHise> I'm unable to think of a way to reproduce your error. It is normal for Cargo to check out git projects into .cargo/git/checkouts/PKGNAME-HASH/SHORT_HASH/. Modifying cargo's internal directory structure may cause problems. Can you create a reproduction? Is there anything special about my_private_library? Does it have its own git or path dependencies? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#6882 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AIQX3GQDFGNKREMNO3VUKUTPTHXFHANCNFSM4HIZHQJQ> .

[JasonHise]

I tried with a different private library and it worked, so it appears there is something about the associative_symbols library specifically that is causing the issue.

[ehuss]

I followed up with @JasonHise over email with the problem with his repo. There are two bugs that conspired to make this difficult to diagnose.

1. A relative path dependency like ../repo_name where the name of the repo is in the path results in a confusing error message for git dependencies. This is because Cargo checks out the repo with a short-hash name (instead of the repo's name). The error message should be clearer, making it easier to figure out what's wrong. Here is a test to demonstrate.

#[test]
fn git_relative_path_root_name() {
    // A path dependency inside a git dependency where the path includes the
    // root name of the git repo.
    let git_project = git::new("dep1", |project| {
        project
            .file(
                "Cargo.toml",
                r#"
                [package]
                name = "dep1"
                version = "0.1.0"

                [dependencies]
                dep2 = { path = "../dep1/dep2" }
                "#,
            )
            .file(
                "src/lib.rs",
                r#"
                extern crate dep2;
                pub fn f() { dep2::f(); }
                "#,
            )
            .file("dep2/Cargo.toml", &basic_manifest("dep2", "0.1.0"))
            .file("dep2/src/lib.rs", "pub fn f() {}")
    })
    .unwrap();

    // This is here just to demonstrate that with a normal git checkout,
    // everything works.
    git_project.cargo("check").run();

    let p = project()
        .file(
            "Cargo.toml",
            &format!(
                r#"
                [package]
                name = "foo"
                version = "0.1.0"

                [dependencies.dep1]
                git = "{}"
                "#,
                git_project.url()
            ),
        )
        .file("src/lib.rs", "extern crate dep1;")
        .build();

    // TODO: This fails with an error that is very difficult to understand.
    p.cargo("build")
        .with_status(101)
        .with_stderr(
            "\
[UPDATING] git repository `file:///[..]/dep1`
[ERROR] failed to load source for a dependency on `dep1`

Caused by:
  Unable to update file:///[..]/dep1

Caused by:
  Could not find `Cargo.toml` in `[..]/dep1/dep2`
",
        )
        .run();
}

2. Unrelated packages in a git dependency are normally ignored if they don't parse correctly. However, if they contain a bad path dependency, then it will cause a hard error. I think Cargo should probably ignore all manifest errors. Here is an example (similar to 1).

EDIT: Fixed via #7947.

#[test]
fn git_unrelated_bad_dependency_error() {
    // A git repo with multiple packages. Parsing errors on Cargo.toml files
    // should be ignored. However, in this case, there is a dependency error
    // that should be ignored.
    // dep1 = a normal, good package
    // dep2 = an unrelated package with an error in its dependencies
    let git_project = git::new("dep1", |project| {
        project
            .file("Cargo.toml", &basic_manifest("dep1", "0.1.0"))
            .file("src/lib.rs", "")
            .file(
                "dep2/Cargo.toml",
                r#"
                [package]
                name = "dep2"
                version = "0.1.0"

                [dependencies]
                bar = {path = "../nonexistent"}
                "#,
            )
            .file("dep2/src/lib.rs", "")
    })
    .unwrap();

    let p = project()
        .file(
            "Cargo.toml",
            &format!(
                r#"
                [package]
                name = "foo"
                version = "0.1.0"

                [dependencies.dep1]
                git = "{}"
                "#,
                git_project.url()
            ),
        )
        .file("src/lib.rs", "extern crate dep1;")
        .build();

    // TODO: This shouldn't fail, but it currently does.
    p.cargo("build").run();
}

[ehuss]

I think this is more or less fixed by #7947, so I'm going to close. Cargo is a little wonky with path dependencies in a git repo (it ignores the path, and just searches the entire repo), and occasionally has bad error messages. #7784 tracks improving those. EDIT: nevermind, that error is resolved, too.