Skip to content

MutableArrayData::extend_nulls doesn't update null bitmask #1230

@tustvold

Description

@tustvold

Describe the bug

Noticed whilst working on #1225, calling MutableArrayData::extend_nulls doesn't update the null bitmask. This means the final array may have a shifted or completely invalid bitmask.

#1225 changed it so that if MutableArrayData is created with nulls disabled, it panics if you try to append nulls. Unfortunately it would appear this is being exploited by ArrowArrayReader to avoid computing a bitmask.

To Reproduce

#[test]
    fn test_nulls() {
        let ints: ArrayRef = Arc::new(Int32Array::from(vec![1]));
        let mut data = MutableArrayData::new(vec![ints.data()], true, 3);
        data.extend_nulls(9);
        let data = data.freeze();
        
        assert_eq!(data.len(), 9);
        assert_eq!(data.null_buffer().unwrap().len(), 2);
    }

Expected behavior

Appending nulls should update the null bitmask, and it should panic if MutableArrayData is created without null support. Fixing this is likely blocked on #1197 as it relies on this behaviour

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions