Panic when compiling dft library v0.5.3

[0x7CFE]

It all started when I tried to use clap library in my project. I added clap = "2.19.3" to Cargo.toml and tried to rebuild:

$ cargo build
    Updating registry `https://github.com/rust-lang/crates.io-index`
 Downloading kernel32-sys v0.2.2
 Downloading unicode-segmentation v0.1.3
 Downloading winapi v0.2.8
^C

…Wait, what? I'm on Linux, why is it trying to download Windows specific libraries?! So I hit ^C and rechecked the setup. I found nothing very special and concluded that there are some weird dependencies in the libraries, so I tried to continue.

Unfortunately, there was another problem:

$ cargo build
 Downloading term_size v0.2.1
unable to get packages from source

Caused by:
  failed to parse manifest at `/home/korvin/.cargo/registry/src/github.colasdn.workers.dev-88ac128001ac3a9a/term_size-0.2.1/Cargo.toml`

Caused by:
  could not parse input as TOML
/home/korvin/.cargo/registry/src/github.colasdn.workers.dev-88ac128001ac3a9a/term_size-0.2.1/Cargo.toml:16:9 expected a key but found an empty string
/home/korvin/.cargo/registry/src/github.colasdn.workers.dev-88ac128001ac3a9a/term_size-0.2.1/Cargo.toml:16:9-16:10 expected `.`, but found `'`

At that time I realized that my compiler is rather new (built from 39c9f184 Sat Dec 10) but my cargo binary is from the very old Rust package 1.6.0 (c30b771ad 2016-01-19) so it may not be able to parse some new syntax.

Of course I've tried to upgrade my Rust installation, so I went to the installation site and used now available rustup:

$ curl https://sh.rustup.rs -sSf | sh
info: downloading installer
warning: it looks like you have an existing installation of Rust
warning: rustup cannot be installed alongside Rust. Please uninstall first
warning: run `/usr/local/lib/rustlib/uninstall.sh` as root to uninstall Rust
error: cannot install while Rust is installed
rustup: command failed: /tmp/tmp.RKZb4gnf47/rustup-init

$ sudo /usr/local/lib/rustlib/uninstall.sh
install: uninstalling component 'rust-docs'
install: uninstalling component 'rust-std-x86_64-unknown-linux-gnu'
install: uninstalling component 'rustc'

    Rust is uninstalled.

$ curl https://sh.rustup.rs -sSf | sh
info: downloading installer

Welcome to Rust!                                                                                                                                                                                                                             

This will download and install the official compiler for the Rust programming 
language, and its package manager, Cargo.

It will add the cargo, rustc, rustup and other commands to Cargo's bin 
directory, located at:

  /home/korvin/.cargo/bin

This path will then be added to your PATH environment variable by modifying the
profile file located at:

  /home/korvin/.profile

You can uninstall at any time with rustup self uninstall and these changes will
be reverted.

Current installation options:

   default host triple: x86_64-unknown-linux-gnu
     default toolchain: stable
  modify PATH variable: yes

1) Proceed with installation (default)
2) Customize installation
3) Cancel installation


info: syncing channel updates for 'stable-x86_64-unknown-linux-gnu'
info: downloading component 'rustc'
 47.1 MiB /  47.1 MiB (100 %)   5.6 MiB/s ETA:   0 s                
info: downloading component 'rust-std'
 56.8 MiB /  56.8 MiB (100 %)   8.4 MiB/s ETA:   0 s                
info: downloading component 'cargo'
  4.0 MiB /   4.0 MiB (100 %) 998.4 KiB/s ETA:   0 s                
info: installing component 'rustc'
info: installing component 'rust-std'
info: installing component 'cargo'
info: default toolchain set to 'stable'

  stable installed - rustc 1.14.0 (e8a012324 2016-12-16)


Rust is installed now. Great!

To get started you need Cargo's bin directory in your PATH environment 
variable. Next time you log in this will be done automatically.

To configure your current shell run source $HOME/.cargo/env
$ cargo --version
bash: /home/korvin/bin/cargo: No such file or directory
$ source $HOME/.cargo/env
$ cargo --version
cargo 0.15.0-nightly (298a012 2016-12-20)

All seemed working as expected, so I tried to recompile again:

$ cargo build
    Updating registry `https://github.com/rust-lang/crates.io-index`
 Downloading byteorder v0.5.3
 Downloading dft v0.5.3
 Downloading num-traits v0.1.36
 Downloading clap v2.19.3
 Downloading bit-vec v0.4.3
 Downloading num-complex v0.1.35
 Downloading rustc-serialize v0.3.22
 Downloading strsim v0.5.2
 Downloading unicode-width v0.1.4
 Downloading bitflags v0.7.0
 Downloading libc v0.2.18
 Downloading unicode-segmentation v0.1.3
 Downloading ansi_term v0.9.0
 Downloading term_size v0.2.1
 Downloading vec_map v0.6.0
   Compiling libc v0.2.18
   Compiling num-traits v0.1.36
   Compiling vec_map v0.6.0
   Compiling unicode-width v0.1.4
   Compiling bitflags v0.7.0
   Compiling bit-vec v0.4.3
   Compiling rustc-serialize v0.3.22
   Compiling byteorder v0.5.3
   Compiling term_size v0.2.1
   Compiling strsim v0.5.2
   Compiling ansi_term v0.9.0
   Compiling unicode-segmentation v0.1.3
   Compiling clap v2.19.3
   Compiling num-complex v0.1.35
   Compiling dft v0.5.3
error: internal compiler error: unexpected panic

note: the compiler unexpectedly panicked. this is a bug.

note: we would appreciate a bug report: https://github.com/rust-lang/rust/blob/master/CONTRIBUTING.md#bug-reports

thread 'rustc' panicked at 'index out of bounds: the len is 538825 but the index is 538825', ../src/libserialize/leb128.rs:47
note: Run with `RUST_BACKTRACE=1` for a backtrace.

Build failed, waiting for other jobs to finish...
error: Could not compile `dft`.

To learn more, run the command again with --verbose.

Oops. Looks like something went terribly wrong. So, the complete compiler output is:

$ rustc --version --verbose
rustc 1.14.0 (e8a012324 2016-12-16)
binary: rustc
commit-hash: e8a0123241f0d397d39cd18fcc4e5e7edde22730
commit-date: 2016-12-16
host: x86_64-unknown-linux-gnu
release: 1.14.0
LLVM version: 3.9

$ RUST_BACKTRACE=1 cargo build
   Compiling dft v0.5.3
error: internal compiler error: unexpected panic

note: the compiler unexpectedly panicked. this is a bug.

note: we would appreciate a bug report: https://github.com/rust-lang/rust/blob/master/CONTRIBUTING.md#bug-reports

note: run with `RUST_BACKTRACE=1` for a backtrace

thread 'rustc' panicked at 'index out of bounds: the len is 538825 but the index is 538825', ../src/libserialize/leb128.rs:47
stack backtrace:
   1:     0x7f752cfe87aa - std::sys::imp::backtrace::tracing::imp::write::h917062bce4ff48c3
                        at /buildslave/rust-buildbot/slave/stable-dist-rustc-linux/build/obj/../src/libstd/sys/unix/backtrace/tracing/gcc_s.rs:42
   2:     0x7f752cffa94f - std::panicking::default_hook::{{closure}}::h0bacac31b5ed1870
                        at /buildslave/rust-buildbot/slave/stable-dist-rustc-linux/build/obj/../src/libstd/panicking.rs:247
   3:     0x7f752cff71a5 - std::panicking::default_hook::h5897799da33ece67
                        at /buildslave/rust-buildbot/slave/stable-dist-rustc-linux/build/obj/../src/libstd/panicking.rs:257
   4:     0x7f752cff78b7 - std::panicking::rust_panic_with_hook::h109e116a3a861224
                        at /buildslave/rust-buildbot/slave/stable-dist-rustc-linux/build/obj/../src/libstd/panicking.rs:451
   5:     0x7f752cff7744 - std::panicking::begin_panic::hbb38be1379e09df0
                        at /buildslave/rust-buildbot/slave/stable-dist-rustc-linux/build/obj/../src/libstd/panicking.rs:413
   6:     0x7f752cff7669 - std::panicking::begin_panic_fmt::h26713cea9bce3ab0
                        at /buildslave/rust-buildbot/slave/stable-dist-rustc-linux/build/obj/../src/libstd/panicking.rs:397
   7:     0x7f752cff75f7 - rust_begin_unwind
                        at /buildslave/rust-buildbot/slave/stable-dist-rustc-linux/build/obj/../src/libstd/panicking.rs:373
   8:     0x7f752d0482fd - core::panicking::panic_fmt::hcfbb59eeb7f27f75
                        at /buildslave/rust-buildbot/slave/stable-dist-rustc-linux/build/obj/../src/libcore/panicking.rs:69
   9:     0x7f752d0482a8 - core::panicking::panic_bounds_check::h155238769b791565
                        at /buildslave/rust-buildbot/slave/stable-dist-rustc-linux/build/obj/../src/libcore/panicking.rs:56
  10:     0x7f752ac5d940 - rustc_metadata::decoder::<impl rustc_metadata::cstore::MetadataBlob>::get_root::heab36214de4017e1
                        at /buildslave/rust-buildbot/slave/stable-dist-rustc-linux/build/obj/../src/libserialize/leb128.rs:47
                        at /buildslave/rust-buildbot/slave/stable-dist-rustc-linux/build/obj/../src/libserialize/opaque.rs:164
                        at /buildslave/rust-buildbot/slave/stable-dist-rustc-linux/build/obj/../src/librustc_metadata/decoder.rs:200
                        at /buildslave/rust-buildbot/slave/stable-dist-rustc-linux/build/obj/../src/librustc_metadata/decoder.rs:235
                        at /buildslave/rust-buildbot/slave/stable-dist-rustc-linux/build/obj/../src/libserialize/serialize.rs:746
                        at /buildslave/rust-buildbot/slave/stable-dist-rustc-linux/build/obj/../src/libserialize/serialize.rs:201
                        at /buildslave/rust-buildbot/slave/stable-dist-rustc-linux/build/obj/../src/librustc_metadata/schema.rs:163
                        at /buildslave/rust-buildbot/slave/stable-dist-rustc-linux/build/obj/../src/libserialize/serialize.rs:195
                        at /buildslave/rust-buildbot/slave/stable-dist-rustc-linux/build/obj/../src/librustc_metadata/schema.rs:163
                        at /buildslave/rust-buildbot/slave/stable-dist-rustc-linux/build/obj/../src/librustc_metadata/decoder.rs:143
                        at /buildslave/rust-buildbot/slave/stable-dist-rustc-linux/build/obj/../src/librustc_metadata/decoder.rs:430
  11:     0x7f752ac84530 - rustc_metadata::locator::Context::extract_one::hdd6b5dcb63616da9
                        at /buildslave/rust-buildbot/slave/stable-dist-rustc-linux/build/obj/../src/librustc_metadata/locator.rs:625
                        at /buildslave/rust-buildbot/slave/stable-dist-rustc-linux/build/obj/../src/librustc_metadata/locator.rs:571
  12:     0x7f752ac81c8e - rustc_metadata::locator::Context::find_library_crate::h9a3af5cdf4d6637c
                        at /buildslave/rust-buildbot/slave/stable-dist-rustc-linux/build/obj/../src/librustc_metadata/locator.rs:498
  13:     0x7f752ac71d2b - rustc_metadata::creader::CrateLoader::load::hd504a966af845abd
                        at /buildslave/rust-buildbot/slave/stable-dist-rustc-linux/build/obj/../src/librustc_metadata/locator.rs:309
                        at /buildslave/rust-buildbot/slave/stable-dist-rustc-linux/build/obj/../src/librustc_metadata/creader.rs:372
  14:     0x7f752ac711ed - rustc_metadata::creader::CrateLoader::resolve_crate::h7fd34532ed681d97
                        at /buildslave/rust-buildbot/slave/stable-dist-rustc-linux/build/obj/../src/librustc_metadata/creader.rs:349
  15:     0x7f752ac890cd - rustc_metadata::creader::CrateLoader::resolve_crate_deps::{{closure}}::h11d774d27d4fc496
                        at /buildslave/rust-buildbot/slave/stable-dist-rustc-linux/build/obj/../src/librustc_metadata/creader.rs:447
  16:     0x7f752abfda93 - <std::collections::hash::map::HashMap<K, V, S> as core::iter::traits::FromIterator<(K, V)>>::from_iter::hd3118d5550540ebc
                        at /buildslave/rust-buildbot/slave/stable-dist-rustc-linux/build/obj/../src/libcore/ops.rs:2645
                        at /buildslave/rust-buildbot/slave/stable-dist-rustc-linux/build/obj/../src/libcore/option.rs:383
                        at /buildslave/rust-buildbot/slave/stable-dist-rustc-linux/build/obj/../src/libcore/iter/mod.rs:982
                        at /buildslave/rust-buildbot/slave/stable-dist-rustc-linux/build/obj/../src/libstd/collections/hash/map.rs:1988
                        at /buildslave/rust-buildbot/slave/stable-dist-rustc-linux/build/obj/../src/libstd/collections/hash/map.rs:1977
  17:     0x7f752ac6f64d - rustc_metadata::creader::CrateLoader::register_crate::h8f61e965dcdd115a
                        at /buildslave/rust-buildbot/slave/stable-dist-rustc-linux/build/obj/../src/libcore/iter/iterator.rs:1196
                        at /buildslave/rust-buildbot/slave/stable-dist-rustc-linux/build/obj/../src/librustc_metadata/creader.rs:445
                        at /buildslave/rust-buildbot/slave/stable-dist-rustc-linux/build/obj/../src/librustc_metadata/creader.rs:289
  18:     0x7f752ac71522 - rustc_metadata::creader::CrateLoader::resolve_crate::h7fd34532ed681d97
                        at /buildslave/rust-buildbot/slave/stable-dist-rustc-linux/build/obj/../src/librustc_metadata/creader.rs:365
  19:     0x7f752ac7969c - <rustc_metadata::creader::CrateLoader<'a> as rustc::middle::cstore::CrateLoader>::process_item::h157dddb3a57c65a4
                        at /buildslave/rust-buildbot/slave/stable-dist-rustc-linux/build/obj/../src/librustc_metadata/creader.rs:1015
  20:     0x7f752c5cac4e - rustc_resolve::build_reduced_graph::<impl rustc_resolve::Resolver<'b>>::build_reduced_graph_for_item::h96b728d66b4c8381
                        at /buildslave/rust-buildbot/slave/stable-dist-rustc-linux/build/obj/../src/librustc_resolve/build_reduced_graph.rs:233
  21:     0x7f752c5d1951 - <rustc_resolve::build_reduced_graph::BuildReducedGraphVisitor<'a, 'b> as syntax::visit::Visitor>::visit_item::ha580ced821dad236
                        at /buildslave/rust-buildbot/slave/stable-dist-rustc-linux/build/obj/../src/librustc_resolve/build_reduced_graph.rs:702
  22:     0x7f752c5d1f1e - <rustc_resolve::build_reduced_graph::BuildReducedGraphVisitor<'a, 'b> as syntax::visit::Visitor>::visit_item::ha580ced821dad236
                        at /buildslave/rust-buildbot/slave/stable-dist-rustc-linux/build/obj/../src/libsyntax/visit.rs:140
                        at /buildslave/rust-buildbot/slave/stable-dist-rustc-linux/build/obj/../src/libsyntax/visit.rs:59
                        at /buildslave/rust-buildbot/slave/stable-dist-rustc-linux/build/obj/../src/libsyntax/visit.rs:256
                        at /buildslave/rust-buildbot/slave/stable-dist-rustc-linux/build/obj/../src/librustc_resolve/build_reduced_graph.rs:703
  23:     0x7f752c5c650e - rustc_resolve::macros::<impl syntax::ext::base::Resolver for rustc_resolve::Resolver<'a>>::visit_expansion::h5453348ed0bfa716
                        at /buildslave/rust-buildbot/slave/stable-dist-rustc-linux/build/obj/../src/libsyntax/ext/expand.rs:93
                        at /buildslave/rust-buildbot/slave/stable-dist-rustc-linux/build/obj/../src/librustc_resolve/macros.rs:110
  24:     0x7f75298439be - syntax::ext::expand::MacroExpander::collect_invocations::h9384ac47fc8e9be0
                        at /buildslave/rust-buildbot/slave/stable-dist-rustc-linux/build/obj/../src/libsyntax/ext/expand.rs:313
  25:     0x7f75298402b8 - syntax::ext::expand::MacroExpander::expand::hde16e53a7123afb4
                        at /buildslave/rust-buildbot/slave/stable-dist-rustc-linux/build/obj/../src/libsyntax/ext/expand.rs:224
  26:     0x7f752983fdcd - syntax::ext::expand::MacroExpander::expand_crate::hd4343a1ada5fbe2e
                        at /buildslave/rust-buildbot/slave/stable-dist-rustc-linux/build/obj/../src/libsyntax/ext/expand.rs:208
  27:     0x7f752d3c381f - rustc_driver::driver::phase_2_configure_and_expand::{{closure}}::h06fe772c22a36b93
                        at /buildslave/rust-buildbot/slave/stable-dist-rustc-linux/build/obj/../src/librustc_driver/driver.rs:683
  28:     0x7f752d379096 - rustc_driver::driver::phase_2_configure_and_expand::h86e0a42d337e36fb
                        at /buildslave/rust-buildbot/slave/stable-dist-rustc-linux/build/obj/../src/librustc/util/common.rs:38
                        at /buildslave/rust-buildbot/slave/stable-dist-rustc-linux/build/obj/../src/librustc_driver/driver.rs:646
  29:     0x7f752d372f63 - rustc_driver::driver::compile_input::h66a0adaf05fab85d
                        at /buildslave/rust-buildbot/slave/stable-dist-rustc-linux/build/obj/../src/librustc_driver/driver.rs:122
  30:     0x7f752d39ed30 - rustc_driver::run_compiler::hafe7bbfedf95a825
                        at /buildslave/rust-buildbot/slave/stable-dist-rustc-linux/build/obj/../src/librustc_driver/lib.rs:222
  31:     0x7f752d2d3b48 - std::panicking::try::do_call::h5ff5e568cb9cce51
                        at /buildslave/rust-buildbot/slave/stable-dist-rustc-linux/build/obj/../src/librustc_driver/lib.rs:1138
                        at /buildslave/rust-buildbot/slave/stable-dist-rustc-linux/build/obj/../src/librustc_driver/lib.rs:138
                        at /buildslave/rust-buildbot/slave/stable-dist-rustc-linux/build/obj/../src/librustc_driver/lib.rs:1072
                        at /buildslave/rust-buildbot/slave/stable-dist-rustc-linux/build/obj/../src/libstd/panic.rs:295
                        at /buildslave/rust-buildbot/slave/stable-dist-rustc-linux/build/obj/../src/libstd/panicking.rs:356
  32:     0x7f752d00285a - __rust_maybe_catch_panic
                        at /buildslave/rust-buildbot/slave/stable-dist-rustc-linux/build/obj/../src/libpanic_unwind/lib.rs:97
  33:     0x7f752d2f37a8 - <F as alloc::boxed::FnBox<A>>::call_box::h92e2f4ed0df1d5b6
                        at /buildslave/rust-buildbot/slave/stable-dist-rustc-linux/build/obj/../src/libstd/panicking.rs:332
                        at /buildslave/rust-buildbot/slave/stable-dist-rustc-linux/build/obj/../src/libstd/panic.rs:351
                        at /buildslave/rust-buildbot/slave/stable-dist-rustc-linux/build/obj/../src/libstd/thread/mod.rs:287
                        at /buildslave/rust-buildbot/slave/stable-dist-rustc-linux/build/obj/../src/liballoc/boxed.rs:595
  34:     0x7f752cff6474 - std::sys::imp::thread::Thread::new::thread_start::ha102a6120fc52763
                        at /buildslave/rust-buildbot/slave/stable-dist-rustc-linux/build/obj/../src/liballoc/boxed.rs:605
                        at /buildslave/rust-buildbot/slave/stable-dist-rustc-linux/build/obj/../src/libstd/sys_common/thread.rs:21
                        at /buildslave/rust-buildbot/slave/stable-dist-rustc-linux/build/obj/../src/libstd/sys/unix/thread.rs:84
  35:     0x7f7525f5b6b9 - start_thread
  36:     0x7f752ccb182c - clone
  37:                0x0 - <unknown>

error: Could not compile `dft`.

My guess is that there is some problem with caches versioning or incremental compilation. Should I reset the caches somehow?

P.S.: I intentionally tried to write as detailed as possible because it looks like a problem related to environment and compiler changes. Hope this will help.

[0x7CFE]

@nagisa adviced me to remove the target directory and then compilation succeeded.

However I still treat it as a bug, mainly because IMO, compiler should track versions of all relevant data such as caches, targets and so on.

[sanxiyn]

This seems to be #37803, which is fixed.

While this won't happen in the future, you can still hit the bug when changing the compiler from buggy version to fixed version. Do you still want that fixed, or is this good enough?

As I understand, the bug was never in any release.

[0x7CFE]

Well, I think it's not me who should decide. It is definitely not a blocker for me, because there is an easy solution: just do cargo clean.

However, I think this bug may affect some non-trivial setups like compile farms or online compilers where one may eventually face some strange compilation errors.

[0x7CFE]

Closing as not actual