-
Notifications
You must be signed in to change notification settings - Fork 996
Closed
Labels
Description
Describe the bug
Noticed whilst working on #1225, calling MutableArrayData::extend_nulls
doesn't update the null bitmask. This means the final array may have a shifted or completely invalid bitmask.
#1225 changed it so that if MutableArrayData
is created with nulls disabled, it panics if you try to append nulls. Unfortunately it would appear this is being exploited by ArrowArrayReader
to avoid computing a bitmask.
To Reproduce
#[test]
fn test_nulls() {
let ints: ArrayRef = Arc::new(Int32Array::from(vec![1]));
let mut data = MutableArrayData::new(vec![ints.data()], true, 3);
data.extend_nulls(9);
let data = data.freeze();
assert_eq!(data.len(), 9);
assert_eq!(data.null_buffer().unwrap().len(), 2);
}
Expected behavior
Appending nulls should update the null bitmask, and it should panic if MutableArrayData
is created without null support. Fixing this is likely blocked on #1197 as it relies on this behaviour