Miri issues

[schungx]

Just out of curiosity, I ran my lib (which depends on smartstring) on miri, and immediately got the following:

   --> C:\Users\Stephen\.rustup\toolchains\nightly-x86_64-pc-windows-msvc\lib\rustlib\src\rust\library\alloc\src\alloc.rs:105:14
    |
105 |     unsafe { __rust_dealloc(ptr, layout.size(), layout.align()) }
    |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ incorrect layout on deallocation: alloc4396604 has size 51 and alignment 1, but gave size 51 and alignment 2
    |
    = help: this indicates a bug in the program: it performed an invalid operation, and caused Undefined Behavior
    = help: see https://doc.rust-lang.org/nightly/reference/behavior-considered-undefined.html for further information

    = note: inside `std::alloc::dealloc` at C:\Users\Stephen\.rustup\toolchains\nightly-x86_64-pc-windows-msvc\lib\rustlib\src\rust\library\alloc\src\alloc.rs:105:14
    = note: inside `<smartstring::boxed::BoxedString as std::ops::Drop>::drop` at C:\Users\Stephen\.cargo\registry\src\mygithub.libinneed.workers.dev-1ecc6299db9ec823\smartstring-1.0.0\src\boxed.rs:135:13
    = note: inside `std::ptr::drop_in_place::<smartstring::boxed::BoxedString> - shim(Some(smartstring::boxed::BoxedString))` at C:\Users\Stephen\.rustup\toolchains\nightly-x86_64-pc-windows-msvc\lib\rustlib\src\rust\library\core\src\ptr\mod.rs:188:1
    = note: inside `<smartstring::SmartString<smartstring::config::LazyCompact> as std::ops::Drop>::drop` at C:\Users\Stephen\.cargo\registry\src\mygithub.libinneed.workers.dev-1ecc6299db9ec823\smartstring-1.0.0\src\lib.rs:200:17 
    = note: inside `std::ptr::drop_in_place::<smartstring::SmartString<smartstring::config::LazyCompact>> - shim(Some(smartstring::SmartString<smartstring::config::LazyCompact>))` at C:\Users\Stephen\.rustup\toolchains\nightly-x86_64-pc-windows-msvc\lib\rustlib\src\rust\library\core\src\ptr\mod.rs:188:1
    = note: inside `std::ptr::drop_in_place::<[smartstring::SmartString<smartstring::config::LazyCompact>]> - shim(Some([smartstring::SmartString<smartstring::config::LazyCompact>]))` at C:\Users\Stephen\.rustup\toolchains\nightly-x86_64-pc-windows-msvc\lib\rustlib\src\rust\library\core\src\ptr\mod.rs:188:1       
    = note: inside `<smallvec::SmallVec<[smartstring::SmartString<smartstring::config::LazyCompact>; 3]> as std::ops::Drop>::drop` at C:\Users\Stephen\.cargo\registry\src\mygithub.libinneed.workers.dev-1ecc6299db9ec823\smallvec-1.8.0\src\lib.rs:1795:17
    = note: inside `std::ptr::drop_in_place::<smallvec::SmallVec<[smartstring::SmartString<smartstring::config::LazyCompact>; 3]>> - shim(Some(smallvec::SmallVec<[smartstring::SmartString<smartstring::config::LazyCompact>; 3]>))` at C:\Users\Stephen\.rustup\toolchains\nightly-x86_64-pc-windows-msvc\lib\rustlib\src\rust\library\core\src\ptr\mod.rs:188:1
    = note: inside `std::ptr::drop_in_place::<rhai::module::FnMetadata> - shim(Some(rhai::module::FnMetadata))` at C:\Users\Stephen\.rustup\toolchains\nightly-x86_64-pc-windows-msvc\lib\rustlib\src\rust\library\core\src\ptr\mod.rs:188:1
    = note: inside `std::ptr::drop_in_place::<rhai::module::FuncInfo> - shim(Some(rhai::module::FuncInfo))` at C:\Users\Stephen\.rustup\toolchains\nightly-x86_64-pc-windows-msvc\lib\rustlib\src\rust\library\core\src\ptr\mod.rs:188:1
    = note: inside `std::ptr::drop_in_place::<std::boxed::Box<rhai::module::FuncInfo>> - shim(Some(std::boxed::Box<rhai::module::FuncInfo>))` at C:\Users\Stephen\.rustup\toolchains\nightly-x86_64-pc-windows-msvc\lib\rustlib\src\rust\library\core\src\ptr\mod.rs:188:1
    = note: inside `std::mem::MaybeUninit::<std::boxed::Box<rhai::module::FuncInfo>>::assume_init_drop` at C:\Users\Stephen\.rustup\toolchains\nightly-x86_64-pc-windows-msvc\lib\rustlib\src\rust\library\core\src\mem\maybe_uninit.rs:735:18
    = note: inside `alloc::collections::btree::node::Handle::<alloc::collections::btree::node::NodeRef<alloc::collections::btree::node::marker::Dying, u64, std::boxed::Box<rhai::module::FuncInfo>, alloc::collections::btree::node::marker::LeafOrInternal>, alloc::collections::btree::node::marker::KV>::drop_key_val` at C:\Users\Stephen\.rustup\toolchains\nightly-x86_64-pc-windows-msvc\lib\rustlib\src\rust\library\alloc\src\collections\btree\node.rs:1074:13
    = note: inside `<std::collections::btree_map::IntoIter<u64, std::boxed::Box<rhai::module::FuncInfo>> as std::ops::Drop>::drop` at C:\Users\Stephen\.rustup\toolchains\nightly-x86_64-pc-windows-msvc\lib\rustlib\src\rust\library\alloc\src\collections\btree\map.rs:1497:22
    = note: inside `std::ptr::drop_in_place::<std::collections::btree_map::IntoIter<u64, std::boxed::Box<rhai::module::FuncInfo>>> - shim(Some(std::collections::btree_map::IntoIter<u64, std::boxed::Box<rhai::module::FuncInfo>>))` at C:\Users\Stephen\.rustup\toolchains\nightly-x86_64-pc-windows-msvc\lib\rustlib\src\rust\library\core\src\ptr\mod.rs:188:1
    = note: inside `std::mem::drop::<std::collections::btree_map::IntoIter<u64, std::boxed::Box<rhai::module::FuncInfo>>>` at C:\Users\Stephen\.rustup\toolchains\nightly-x86_64-pc-windows-msvc\lib\rustlib\src\rust\library\core\src\mem\mod.rs:965:24
    = note: inside `<std::collections::BTreeMap<u64, std::boxed::Box<rhai::module::FuncInfo>> as std::ops::Drop>::drop` at C:\Users\Stephen\.rustup\toolchains\nightly-x86_64-pc-windows-msvc\lib\rustlib\src\rust\library\alloc\src\collections\btree\map.rs:174:9
    = note: inside `std::ptr::drop_in_place::<std::collections::BTreeMap<u64, std::boxed::Box<rhai::module::FuncInfo>>> - shim(Some(std::collections::BTreeMap<u64, std::boxed::Box<rhai::module::FuncInfo>>))` at C:\Users\Stephen\.rustup\toolchains\nightly-x86_64-pc-windows-msvc\lib\rustlib\src\rust\library\core\src\ptr\mod.rs:188:1
    = note: inside `std::ptr::drop_in_place::<rhai::Module> - shim(Some(rhai::Module))` at C:\Users\Stephen\.rustup\toolchains\nightly-x86_64-pc-windows-msvc\lib\rustlib\src\rust\library\core\src\ptr\mod.rs:188:1
    = note: inside `<rhai::Shared<rhai::Module> as std::ops::Drop>::drop` at C:\Users\Stephen\.rustup\toolchains\nightly-x86_64-pc-windows-msvc\lib\rustlib\src\rust\library\alloc\src\rc.rs:1511:17
    = note: inside `std::ptr::drop_in_place::<rhai::Shared<rhai::Module>> - shim(Some(rhai::Shared<rhai::Module>))` at C:\Users\Stephen\.rustup\toolchains\nightly-x86_64-pc-windows-msvc\lib\rustlib\src\rust\library\core\src\ptr\mod.rs:188:1
    = note: inside `std::ptr::drop_in_place::<[rhai::Shared<rhai::Module>]> - shim(Some([rhai::Shared<rhai::Module>]))` at C:\Users\Stephen\.rustup\toolchains\nightly-x86_64-pc-windows-msvc\lib\rustlib\src\rust\library\core\src\ptr\mod.rs:188:1
    = note: inside `<smallvec::SmallVec<[rhai::Shared<rhai::Module>; 3]> as std::ops::Drop>::drop` at C:\Users\Stephen\.cargo\registry\src\mygithub.libinneed.workers.dev-1ecc6299db9ec823\smallvec-1.8.0\src\lib.rs:1795:17
    = note: inside `std::ptr::drop_in_place::<smallvec::SmallVec<[rhai::Shared<rhai::Module>; 3]>> - shim(Some(smallvec::SmallVec<[rhai::Shared<rhai::Module>; 3]>))` at C:\Users\Stephen\.rustup\toolchains\nightly-x86_64-pc-windows-msvc\lib\rustlib\src\rust\library\core\src\ptr\mod.rs:188:1
    = note: inside `std::ptr::drop_in_place::<rhai::Engine> - shim(Some(rhai::Engine))` at C:\Users\Stephen\.rustup\toolchains\nightly-x86_64-pc-windows-msvc\lib\rustlib\src\rust\library\core\src\ptr\mod.rs:188:1

That could be due to casting a byte reference (which is byte-aligned) to a struct reference?

[bodil]

Could this be a case of a std::string::String converted into a SmartString with From/Into? The internal allocator should always be dealing with an alignment of 2, but when converting String it only checks for alignment before deciding whether to co-opt the allocated buffer, and it would be deallocated as if it had an alignment of 2 (which it has, but only unofficially). This is probably a bad thing that should be fixed either way, but I'd like to know if this is the case here or if there's another problem hiding elsewhere.

[schungx]

Hhhmmm... not sure about that. Obviously it trips Miri off during cleanup destruction of a SmartString.

That SmartString happens to reside in a smallvec::SmallVec if that helps track it down.

I've used From/Into for String into SmartString quite liberally so that can definitely be a potential source.

BTW it is failing here:

impl Drop for BoxedString {
    fn drop(&mut self) {
        #[allow(unsafe_code)]
        unsafe {
            alloc::alloc::dealloc(self.ptr.as_ptr(), Self::layout_for(self.cap))
        }
    }
}

[bodil]

I've written a test which just does SmartString::from(String) which has Miri giving this exact error, so I'm going to assume this is the problem. Which is a bit annoying, because to fix it I have to get rid of the no-alloc From::from() implementations entirely.

[bodil]

Solved this by using the unstable Allocator API to re-align memory properly. The drawback is that unless you're on nightly or the aforementioned API has stabilised, From::from() now has to do alloc/copy/dealloc instead of reusing the String's heap memory, so if you're using it a lot you might see a slight performance hit.

I've tagged a 1.0.1 release with the fix.

[schungx]

HHhhmmmm.... since it would only trip up Miri, I don't think it is worth paying the performance penalty for an additional copy.

So I think: 1) Keep existing for the stable channel, 2) use Allocator API for nightly, 3) detect via a feature gate.