Incremental compilation fails in all cases on SystemZ (s390x)

[Erk-]

Incremental compilation always fails on s390x even with no changes that should affect the code.

Setting CARGO_INCREMENTAL=0 causes it to not happen.

Code

fn main() {
    println!("Hello, world!");
}

and then any change to make it recompile, for example

fn main() {
    println!("Hello, world!");
}
//

and then recompile without cleaning.

Meta

Happens on both beta and nightly, but not on stable.

rustc --version --verbose:

rustc 1.58.0-nightly (efd048394 2021-10-20)
binary: rustc
commit-hash: efd0483949496b067cd5f7569d1b28cd3d5d3c72
commit-date: 2021-10-20
host: s390x-unknown-linux-gnu
release: 1.58.0-nightly
LLVM version: 13.0.0

and

rustc 1.57.0-beta.1 (d4647278c 2021-10-18)
binary: rustc
commit-hash: d4647278cb2948e76b51e8cd7aa7d31ba6478a08
commit-date: 2021-10-18
host: s390x-unknown-linux-gnu
release: 1.57.0-beta.1
LLVM version: 13.0.0

Error output

[linux1@rusting rustbug2]$ cargo +beta run
   Compiling rustbug2 v0.1.0 (/home/linux1/dev/rustbug2)
thread 'rustc' panicked at 'called `Option::unwrap()` on a `None` value', /rustc/d4647278cb2948e76b51e8cd7aa7d31ba6478a08/compiler/rustc_hir/src/definitions.rs:452:14
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

error: internal compiler error: unexpected panic

note: the compiler unexpectedly panicked. this is a bug.

note: we would appreciate a bug report: https://github.com/rust-lang/rust/issues/new?labels=C-bug%2C+I-ICE%2C+T-compiler&template=ice.md

note: rustc 1.57.0-beta.1 (d4647278c 2021-10-18) running on s390x-unknown-linux-gnu

note: compiler flags: -C embed-bitcode=no -C debuginfo=2 -C incremental --crate-type bin

note: some of the compiler flags provided by cargo are hidden

query stack during panic:
#0 [hir_module_items] HIR module items in ``
#1 [analysis] running analysis passes on this crate
end of query stack
error: could not compile `rustbug2`

Backtrace Nightly

[linux1@rusting rustbug2]$ cargo clean
[linux1@rusting rustbug2]$ cat src/main.rs 
fn main() {
    println!("Hello, world!");
}
[linux1@rusting rustbug2]$ cargo run
   Compiling rustbug2 v0.1.0 (/home/linux1/dev/rustbug2)
    Finished dev [unoptimized + debuginfo] target(s) in 0.72s
     Running `target/debug/rustbug2`
Hello, world!
[linux1@rusting rustbug2]$ echo "//" >> src/main.rs 
[linux1@rusting rustbug2]$ RUST_BACKTRACE=full cargo run
   Compiling rustbug2 v0.1.0 (/home/linux1/dev/rustbug2)
thread 'rustc' panicked at 'called `Option::unwrap()` on a `None` value', /rustc/efd0483949496b067cd5f7569d1b28cd3d5d3c72/compiler/rustc_hir/src/definitions.rs:452:14
stack backtrace:
   0:      0x3ff74834786 - std::backtrace_rs::backtrace::libunwind::trace::h6b75d52adcf7c9b5
                               at /rustc/efd0483949496b067cd5f7569d1b28cd3d5d3c72/library/std/src/../../backtrace/src/backtrace/libunwind.rs:93:5
   1:      0x3ff74834786 - std::backtrace_rs::backtrace::trace_unsynchronized::h41f3dac57859ea32
                               at /rustc/efd0483949496b067cd5f7569d1b28cd3d5d3c72/library/std/src/../../backtrace/src/backtrace/mod.rs:66:5
   2:      0x3ff74834786 - std::sys_common::backtrace::_print_fmt::h8094d49aad352b0d
                               at /rustc/efd0483949496b067cd5f7569d1b28cd3d5d3c72/library/std/src/sys_common/backtrace.rs:67:5   3:      0x3ff74834786 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::h241d4643436be0a9
                               at /rustc/efd0483949496b067cd5f7569d1b28cd3d5d3c72/library/std/src/sys_common/backtrace.rs:46:22
   4:      0x3ff748b6dee - core::fmt::write::h1f6b8010036c2a46
                               at /rustc/efd0483949496b067cd5f7569d1b28cd3d5d3c72/library/core/src/fmt/mod.rs:1163:17
   5:      0x3ff7481f762 - std::io::Write::write_fmt::h6997ea0259d085f3
                               at /rustc/efd0483949496b067cd5f7569d1b28cd3d5d3c72/library/std/src/io/mod.rs:1696:15
   6:      0x3ff74839892 - std::sys_common::backtrace::_print::h17df086e798ed7ea
                               at /rustc/efd0483949496b067cd5f7569d1b28cd3d5d3c72/library/std/src/sys_common/backtrace.rs:49:5   7:      0x3ff74839892 - std::sys_common::backtrace::print::hcce0d091aadaacb8
                               at /rustc/efd0483949496b067cd5f7569d1b28cd3d5d3c72/library/std/src/sys_common/backtrace.rs:36:9   8:      0x3ff74839892 - std::panicking::default_hook::{{closure}}::h460a54625135dc7f
                               at /rustc/efd0483949496b067cd5f7569d1b28cd3d5d3c72/library/std/src/panicking.rs:210:50
   9:      0x3ff74838fc4 - std::panicking::default_hook::h1090968d62468ee9
                               at /rustc/efd0483949496b067cd5f7569d1b28cd3d5d3c72/library/std/src/panicking.rs:227:9
  10:      0x3ff75740c74 - rustc_driver::DEFAULT_HOOK::{{closure}}::{{closure}}::hb265a347070c7a36
  11:      0x3ff7483a29c - std::panicking::rust_panic_with_hook::hd668ece5cc8b54b4
                               at /rustc/efd0483949496b067cd5f7569d1b28cd3d5d3c72/library/std/src/panicking.rs:606:17
  12:      0x3ff74839c54 - std::panicking::begin_panic_handler::{{closure}}::h122e6a6bbfbfebaa
                               at /rustc/efd0483949496b067cd5f7569d1b28cd3d5d3c72/library/std/src/panicking.rs:497:13
  13:      0x3ff74834fa6 - std::sys_common::backtrace::__rust_end_short_backtrace::he2f203a922dc9a17
                               at /rustc/efd0483949496b067cd5f7569d1b28cd3d5d3c72/library/std/src/sys_common/backtrace.rs:139:18
  14:      0x3ff74839bc0 - rust_begin_unwind
                               at /rustc/efd0483949496b067cd5f7569d1b28cd3d5d3c72/library/std/src/panicking.rs:495:5
  15:      0x3ff747e43f8 - core::panicking::panic_fmt::h35b22c7cf69ca5eb
                               at /rustc/efd0483949496b067cd5f7569d1b28cd3d5d3c72/library/core/src/panicking.rs:107:14
  16:      0x3ff747e433c - core::panicking::panic::h4022f17bb7d648a4
                               at /rustc/efd0483949496b067cd5f7569d1b28cd3d5d3c72/library/core/src/panicking.rs:50:5
  17:      0x3ff7bbf8ee0 - rustc_middle::ty::context::TyCtxt::def_path_hash_to_def_id::ha371a19fc338d5c2
  18:      0x3ff7bd3b762 - <rustc_query_system::dep_graph::dep_node::DepNode<rustc_middle::dep_graph::dep_node::DepKind> as rustc_middle::dep_graph::dep_node::DepNodeExt>::extract_def_id::h95db7a56a8859e12
  19:      0x3ff7a89ca06 - rustc_query_impl::query_callbacks::supported_target_features::force_from_dep_node::h4286f7e39ca564ad
  20:      0x3ff7bbcf1f2 - rustc_middle::dep_graph::<impl rustc_query_system::dep_graph::DepContext for rustc_middle::ty::context::TyCtxt>::try_force_from_dep_node::h0117631fb731d4fb
  21:      0x3ff7a7e98b4 - rustc_query_system::dep_graph::graph::DepGraph<K>::try_mark_previous_green::h3b3fff9f935d411e
  22:      0x3ff7a7e988a - rustc_query_system::dep_graph::graph::DepGraph<K>::try_mark_previous_green::h3b3fff9f935d411e
  23:      0x3ff7a76b700 - rustc_query_system::dep_graph::graph::DepGraph<K>::try_mark_green::h1b8eb63f1aaa78b5
  24:      0x3ff7a34dc80 - rustc_query_system::query::plumbing::ensure_must_run::h9b4a204eca719c08
  25:      0x3ff7a73e1b8 - <rustc_query_impl::Queries as rustc_middle::ty::query::QueryEngine>::check_mod_attrs::hdacff3f8e43a4c4d
  26:      0x3ff75a28a72 - <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once::hd0a9667c1a58c0d7
  27:      0x3ff75a5cd94 - rustc_session::utils::<impl rustc_session::session::Session>::time::hc57ba44d7490d71d
  28:      0x3ff75927f08 - rustc_interface::passes::analysis::hf45ba60e0b2ab581
  29:      0x3ff7a7f4248 - rustc_query_system::dep_graph::graph::DepGraph<K>::with_task::h35d17c798a73be3d
  30:      0x3ff7a6bba5e - rustc_data_structures::stack::ensure_sufficient_stack::h16775ec94c7298ed
  31:      0x3ff7a36bb9e - rustc_query_system::query::plumbing::try_execute_query::h241facde616e5b1f
  32:      0x3ff7a739566 - <rustc_query_impl::Queries as rustc_middle::ty::query::QueryEngine>::analysis::h63ff5ad8e2498f34
  33:      0x3ff757e8ed0 - rustc_interface::passes::QueryContext::enter::h91c796cd8c4d929d
  34:      0x3ff757b44aa - rustc_interface::queries::<impl rustc_interface::interface::Compiler>::enter::hb9dd22df9b06bc04
  35:      0x3ff7575acb6 - rustc_span::with_source_map::heeacd22d733666ab
  36:      0x3ff757b2846 - scoped_tls::ScopedKey<T>::set::hc0435551d81e9855
  37:      0x3ff7575cba0 - std::sys_common::backtrace::__rust_begin_short_backtrace::h838a110968871325
  38:      0x3ff757f1e46 - core::ops::function::FnOnce::call_once{{vtable.shim}}::h3001046e3fef7ac9
  39:      0x3ff748459fa - <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once::hfd026960d52bfea8
                               at /rustc/efd0483949496b067cd5f7569d1b28cd3d5d3c72/library/alloc/src/boxed.rs:1691:9
  40:      0x3ff748459fa - <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once::h0eebc9a5d7760782
                               at /rustc/efd0483949496b067cd5f7569d1b28cd3d5d3c72/library/alloc/src/boxed.rs:1691:9
  41:      0x3ff748459fa - std::sys::unix::thread::Thread::new::thread_start::h0a35bb13b5b2b618
                               at /rustc/efd0483949496b067cd5f7569d1b28cd3d5d3c72/library/std/src/sys/unix/thread.rs:106:17
  42:      0x3ff744080b6 - start_thread
  43:      0x3ff745fbd3e - <unknown>
  44:                0x0 - <unknown>

error: internal compiler error: unexpected panic

note: the compiler unexpectedly panicked. this is a bug.

note: we would appreciate a bug report: https://github.com/rust-lang/rust/issues/new?labels=C-bug%2C+I-ICE%2C+T-compiler&template=ice.md

note: rustc 1.58.0-nightly (efd048394 2021-10-20) running on s390x-unknown-linux-gnu

note: compiler flags: -C embed-bitcode=no -C debuginfo=2 -C incremental --crate-type bin

note: some of the compiler flags provided by cargo are hidden

query stack during panic:
#0 [analysis] running analysis passes on this crate
end of query stack
error: could not compile `rustbug2`
[linux1@rusting rustbug2]$ cargo clean
[linux1@rusting rustbug2]$ cargo run
   Compiling rustbug2 v0.1.0 (/home/linux1/dev/rustbug2)
    Finished dev [unoptimized + debuginfo] target(s) in 0.72s
     Running `target/debug/rustbug2`
Hello, world!

Backtrace Beta

[linux1@rusting rustbug2]$ RUST_BACKTRACE=full cargo +beta run
   Compiling rustbug2 v0.1.0 (/home/linux1/dev/rustbug2)
thread 'rustc' panicked at 'called `Option::unwrap()` on a `None` value', /rustc/d4647278cb2948e76b51e8cd7aa7d31ba6478a08/compiler/rustc_hir/src/definitions.rs:452:14
stack backtrace:
   0:      0x3ff75bb4716 - std::backtrace_rs::backtrace::libunwind::trace::hd2f28627750044dd
                               at /rustc/d4647278cb2948e76b51e8cd7aa7d31ba6478a08/library/std/src/../../backtrace/src/backtrace/libunwind.rs:90:5
   1:      0x3ff75bb4716 - std::backtrace_rs::backtrace::trace_unsynchronized::h610d3878094d1a62
                               at /rustc/d4647278cb2948e76b51e8cd7aa7d31ba6478a08/library/std/src/../../backtrace/src/backtrace/mod.rs:66:5
   2:      0x3ff75bb4716 - std::sys_common::backtrace::_print_fmt::h5fabec786a928395
                               at /rustc/d4647278cb2948e76b51e8cd7aa7d31ba6478a08/library/std/src/sys_common/backtrace.rs:67:5   3:      0x3ff75bb4716 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::h2a6f5b4849caf945
                               at /rustc/d4647278cb2948e76b51e8cd7aa7d31ba6478a08/library/std/src/sys_common/backtrace.rs:46:22
   4:      0x3ff75c36d7e - core::fmt::write::h809197c519a783d7
                               at /rustc/d4647278cb2948e76b51e8cd7aa7d31ba6478a08/library/core/src/fmt/mod.rs:1163:17
   5:      0x3ff75b9f842 - std::io::Write::write_fmt::hedb8d8adb1236c10
                               at /rustc/d4647278cb2948e76b51e8cd7aa7d31ba6478a08/library/std/src/io/mod.rs:1696:15
   6:      0x3ff75bb9822 - std::sys_common::backtrace::_print::h0379b554f6e808df
                               at /rustc/d4647278cb2948e76b51e8cd7aa7d31ba6478a08/library/std/src/sys_common/backtrace.rs:49:5   7:      0x3ff75bb9822 - std::sys_common::backtrace::print::hbde6ca29aea6855e
                               at /rustc/d4647278cb2948e76b51e8cd7aa7d31ba6478a08/library/std/src/sys_common/backtrace.rs:36:9   8:      0x3ff75bb9822 - std::panicking::default_hook::{{closure}}::h86c51e35fce6f5b1
                               at /rustc/d4647278cb2948e76b51e8cd7aa7d31ba6478a08/library/std/src/panicking.rs:210:50
   9:      0x3ff75bb8f54 - std::panicking::default_hook::h870f35d3c73fb8ba
                               at /rustc/d4647278cb2948e76b51e8cd7aa7d31ba6478a08/library/std/src/panicking.rs:227:9
  10:      0x3ff76ae34a4 - rustc_driver::DEFAULT_HOOK::{{closure}}::{{closure}}::he11e6fb89998111f
  11:      0x3ff75bba22c - std::panicking::rust_panic_with_hook::h969a9096b409d685
                               at /rustc/d4647278cb2948e76b51e8cd7aa7d31ba6478a08/library/std/src/panicking.rs:628:17
  12:      0x3ff75bb9be4 - std::panicking::begin_panic_handler::{{closure}}::hb335f3f8709ad419
                               at /rustc/d4647278cb2948e76b51e8cd7aa7d31ba6478a08/library/std/src/panicking.rs:519:13
  13:      0x3ff75bb4f06 - std::sys_common::backtrace::__rust_end_short_backtrace::h115999872cd19e9a
                               at /rustc/d4647278cb2948e76b51e8cd7aa7d31ba6478a08/library/std/src/sys_common/backtrace.rs:139:18
  14:      0x3ff75bb9b50 - rust_begin_unwind
                               at /rustc/d4647278cb2948e76b51e8cd7aa7d31ba6478a08/library/std/src/panicking.rs:517:5
  15:      0x3ff75b64388 - core::panicking::panic_fmt::h1b6c5aafe462252b
                               at /rustc/d4647278cb2948e76b51e8cd7aa7d31ba6478a08/library/core/src/panicking.rs:100:14
  16:      0x3ff75b642cc - core::panicking::panic::h7bb71df1ebba1b62
                               at /rustc/d4647278cb2948e76b51e8cd7aa7d31ba6478a08/library/core/src/panicking.rs:50:5
  17:      0x3ff7bc224dc - <rustc_query_impl::on_disk_cache::OnDiskCache as rustc_middle::ty::context::OnDiskCache>::def_path_hash_to_def_id::h72018a1b5693219c
  18:      0x3ff7d123364 - rustc_middle::dep_graph::dep_node::<impl rustc_query_system::dep_graph::dep_node::DepNodeParams<rustc_middle::ty::context::TyCtxt> for rustc_span::def_id::LocalDefId>::recover::h49ebec74f5c931b2
  19:      0x3ff7ba9f000 - rustc_query_impl::query_callbacks::hir_owner::force_from_dep_node::h7b01c5e2191fdc81
  20:      0x3ff7bb5f6b2 - rustc_query_system::dep_graph::graph::DepGraph<K>::try_mark_previous_green::h14f8639d09d1b8f4
  21:      0x3ff7bae0bc0 - rustc_query_system::dep_graph::graph::DepGraph<K>::try_mark_green::h22b5f6a103787178
  22:      0x3ff7b815798 - rustc_query_system::query::plumbing::try_load_from_disk_and_cache_in_memory::h7d60ca4fa217d4f9
  23:      0x3ff7b7409fa - rustc_query_system::query::plumbing::try_execute_query::h99674b5ad35e0ca0
  24:      0x3ff7bab23a0 - <rustc_query_impl::Queries as rustc_middle::ty::query::QueryEngine>::hir_module_items::hcf559bf1b4b28e3d
  25:      0x3ff7acab3ba - rustc_passes::hir_id_validator::check_crate::h38c5f497e90721b1
  26:      0x3ff76cc6fda - rustc_interface::passes::analysis::h770f8448cc20b1aa
  27:      0x3ff7bb6c104 - rustc_query_system::dep_graph::graph::DepGraph<K>::with_task::h2daeb4e54f23e8be
  28:      0x3ff7b9d60d8 - rustc_data_structures::stack::ensure_sufficient_stack::h4839bdbfcd05d9e5
  29:      0x3ff7b7419b8 - rustc_query_system::query::plumbing::try_execute_query::h9a70781718d5d6b2
  30:      0x3ff7bab2c80 - <rustc_query_impl::Queries as rustc_middle::ty::query::QueryEngine>::analysis::h663b7da48ffc0a13
  31:      0x3ff76b8ad50 - rustc_interface::passes::QueryContext::enter::h25df7e6fa597ff02
  32:      0x3ff76b590a8 - rustc_interface::queries::<impl rustc_interface::interface::Compiler>::enter::h8b60f4540c29eef6
  33:      0x3ff76af92f6 - rustc_span::with_source_map::h185958c2f780fc29
  34:      0x3ff76b57280 - scoped_tls::ScopedKey<T>::set::hc0c42db9fb11713f
  35:      0x3ff76b00f60 - std::sys_common::backtrace::__rust_begin_short_backtrace::hab0924924599dc99
  36:      0x3ff76b93dd6 - core::ops::function::FnOnce::call_once{{vtable.shim}}::ha9e56b8c137932e3
  37:      0x3ff75bc598a - <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once::h5e11642d966ba0d8
                               at /rustc/d4647278cb2948e76b51e8cd7aa7d31ba6478a08/library/alloc/src/boxed.rs:1691:9
  38:      0x3ff75bc598a - <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once::h1b5a558aa983ed81
                               at /rustc/d4647278cb2948e76b51e8cd7aa7d31ba6478a08/library/alloc/src/boxed.rs:1691:9
  39:      0x3ff75bc598a - std::sys::unix::thread::Thread::new::thread_start::h32dbec0374245748
                               at /rustc/d4647278cb2948e76b51e8cd7aa7d31ba6478a08/library/std/src/sys/unix/thread.rs:106:17
  40:      0x3ff757880b6 - start_thread
  41:      0x3ff7597bd3e - <unknown>
  42:                0x0 - <unknown>

error: internal compiler error: unexpected panic

note: the compiler unexpectedly panicked. this is a bug.

note: we would appreciate a bug report: https://github.com/rust-lang/rust/issues/new?labels=C-bug%2C+I-ICE%2C+T-compiler&template=ice.md

note: rustc 1.57.0-beta.1 (d4647278c 2021-10-18) running on s390x-unknown-linux-gnu

note: compiler flags: -C embed-bitcode=no -C debuginfo=2 -C incremental --crate-type bin

note: some of the compiler flags provided by cargo are hidden

query stack during panic:
#0 [hir_module_items] HIR module items in ``
#1 [analysis] running analysis passes on this crate
end of query stack
error: could not compile `rustbug2`

[bjorn3]

The panic happens at the unwrap in

rust/compiler/rustc_hir/src/definitions.rs

Lines 446 to 453 in 6f53ddf

	pub fn local_def_path_hash_to_def_id(&self, hash: DefPathHash) -> LocalDefId {
	debug_assert!(hash.stable_crate_id() == self.stable_crate_id);
	self.table
	.def_path_hash_to_index
	.get(&hash)
	.map(|local_def_index| LocalDefId { local_def_index })
	.unwrap()
	}

I suspect the issue is in the odht crate. Could it be an endianness bug?

[Erk-]

I suspect the issue is in the odht crate. Could it be an endianness bug?

Had a look around odht and I think I found the culprit, made a pr to fix it rust-lang/odht#20.

[cuviper]

I have done a full toolchain build with rust-lang/odht#20 and confirmed that it fixes this issue.

[apiraino]

Assigning priority as discussed in the Zulip thread of the Prioritization Working Group.

@rustbot label -I-prioritize +P-high