Implicit binds in virtual calls cause borrow errors

[lilizoey]

Edit bromeon -- title was "Cannot add_child in a &mut self function if Self overrides on_notification"

#[derive(GodotClass)]
#[class(init, base = Node)]
pub struct Bar {
    #[base]
    base: Base<Node>,
}

#[godot_api]
impl NodeVirtual for Bar {
    fn ready(&mut self) {
        if Engine::singleton().is_editor_hint() {
            return;
        }

        let foo = Node::new_alloc();
        self.add_child(foo.share().upcast());
    }

    fn on_notification(&mut self, what: NodeNotification) {}
}

add_child will trigger a notification (https://docs.godotengine.org/en/stable/classes/class_node.html#class-node-constant-notification-child-order-changed), and that will in turn trigger a call to on_notification, which will try to get a mutable borrow of self. This causes a crash.

It seems like this is a big issue tbh because it basically makes self.add_child almost unusable whenever on_notification is overridden.

I'm not sure what a good solution to this is, but it's related to the common issue of calling a signal that will trigger on self.

[Bromeon]

This causes a crash.

I guess you mean panic?

I'm not sure what a good solution to this is, but it's related to the common issue of calling a signal that will trigger on self.

Yep, exactly. It's very hard to know if the on_notification call comes "from Rust". I was thinking about marking certain re-entrant calls as safe, e.g. set a flag during an outbound Rust -> Godot call and allow reborrows.

But the big problem is that we lose information about local borrows. How would we prevent scenarios like this?

#[derive(GodotClass)]
#[class(init, base = Node)]
pub struct Bar {
    #[base]
    base: Base<Node>,

    vec: Vec<i32>,
}

#[godot_api]
impl NodeVirtual for Bar {
    fn ready(&mut self) {
        self.vec = vec![1, 2, 3];

        for _ in self.vec.iter() {
            self.base.add_child(Node::new_alloc());
        }
    }

    fn on_notification(&mut self, what: NodeNotification) {
        self.vec.clear(); // unsound
    }
}

Maybe it needs a completely different design, where such methods don't implicitly borrow self, but only Gd<Self>. They become much less useful, because one would then need a 3rd object (ouside of self) to track state changes...

The same problem applies to signals. I don't see how we can allow safe reborrows. And requiring unsafe for simple things like this is also quite ugly.

[Bromeon]

One spontaneous idea: explicit partial borrows.

#[derive(GodotClass)]
#[class(init, base = Node)]
pub struct Bar {
    #[base]
    base: Base<Node>,

    // Let's say we have 2 data fields:
    vec: Vec<i32>,
    notify_count: usize,
}

And now, the user would need to explicitly request access to certain fields:

#[godot_api]
impl NodeVirtual for Bar {
    fn ready(&mut self) {
        self.vec = vec![1, 2, 3];

        // Tell gdext that we're only accessing these two fields
        // (it should limit the scope if possible, e.g. through a local function).
        // Closure syntax is just an example for familiarity; up to bikeshedding and technical limits.
        partial_borrow!(self => |vec, mut base| {
            for _ in vec.iter() {
                base.add_child(Node::new_alloc());
            }
        });
    }

    fn on_notification(this: Gd<Self>, what: NodeNotification) {
        // This would panic:
        partial_borrow!(this => |mut vec| {
            vec.clear();
        });

        // However this would work:
        partial_borrow!(this => |mut notify_count| {
            *notify_count += 1;
        });
    }
}

It's of course clumsy, but Rust itself is very pedantic about not calling &mut methods while there are any borrows. I'd argue that's one of the trade-offs that someone accepts when using Rust, in return for explicit control and avoiding "clear while iterate" style bugs.

[kulkalkul]

This is not related to the issue title, but commenting here as it is relevant to the rest of the comments.

The same problem applies to signals. I don't see how we can allow safe reborrows.

Same is also true for rpcs. Using rpcs right now infest the rust with "string method" calls. I tried limiting rpc calls like so:

    pub fn set_tool(&mut self, mut item: Item) {
        self.rpc("r_set_tool".into(), &[item.to_variant()]);
    }

But it doesn't work for the same reason. Only workaround that user can do is either using self.rpc("r_set_tool".into(), &[item.to_variant()]); directly or making it deferred. But I don't think either are good ways to solve this problem. I have no suggestion on the solution tho.

[gg-yb]

Also one important thing to note is that deferring is not really a workaround here: It explicitly changes behavior by introducing delays. For some applications that is unacceptable

[gg-yb]

One spontaneous idea: explicit partial borrows.

#[derive(GodotClass)]
#[class(init, base = Node)]
pub struct Bar {
    #[base]
    base: Base<Node>,

    // Let's say we have 2 data fields:
    vec: Vec<i32>,
    notify_count: usize,
}

And now, the user would need to explicitly request access to certain fields:

#[godot_api]
impl NodeVirtual for Bar {
    fn ready(&mut self) {
        self.vec = vec![1, 2, 3];

        // Tell gdext that we're only accessing these two fields
        // (it should limit the scope if possible, e.g. through a local function).
        // Closure syntax is just an example for familiarity; up to bikeshedding and technical limits.
        partial_borrow!(self => |vec, mut base| {
            for _ in vec.iter() {
                base.add_child(Node::new_alloc());
            }
        });
    }

    fn on_notification(this: Gd<Self>, what: NodeNotification) {
        // This would panic:
        partial_borrow!(this => |mut vec| {
            vec.clear();
        });

        // However this would work:
        partial_borrow!(this => |mut notify_count| {
            *notify_count += 1;
        });
    }
}

It's of course clumsy, but Rust itself is very pedantic about not calling &mut methods while there are any borrows. I'd argue that's one of the trade-offs that someone accepts when using Rust, in return for explicit control and avoiding "clear while iterate" style bugs.

Another option could be to only ever use a &self receiver, and force the user to do all state keeping using interior mutability e.g. through (Ref)Cell or Mutex. Not sure how that would play together with exports though.

[Bromeon]

@Soremwar mentioned a closely related issue in #383, which is a more general case of this: calling Rust -> GDScript -> Rust, which causes double borrows. That one is probably even harder to solve than on_notification, since we'd need a way to trace the borrow across FFI and back, with arbitrary GDScript code in the middle.

[lilizoey]

Allowing Gd<Self> #359 as the receiver could be used to do a workaround for #383, though it'd be kinda awkward code imo:

#[func]
fn foo(_self: Gd<Self>) {
  {
    let _self = _self.bind_mut();
    // do stuff with self
  }
  // some function call that would potentially grab a mutable reference to self
  _self.call("some_func", ...);
  {
    let _self = _self.bind_mut();
    // do other stuff with self
  }
}

[bluenote10]

This is an interesting challenge, and probably far more general than the specific example in the issue title. Working with notifications or signals in general has the potential to "backfire", leading to a panic.

Adding another data point: I encountered this issue while implementing a custom item list, which basically started as a port of item_list.cpp of Godot. The pattern used in that implementation doesn't work out-of-the box in Rust, because it involves backfiring signals, even in a relatively subtle way:

• The item list uses a scroll back, and its "value_changed" signal gets connected to a callback that simply does queue_redraw. That makes sense, in general the item list wants to react to scroll events.
• The main layouting actually happens inside NOTIFICATION_DRAW.
• Under certain conditions this drawing logic actually interacts with the scroll bar, calling its set_max and set_page functions. What is not immediately obvious: Under certain conditions these setter actually internally delegate to the set_value setter, which in turn triggers the signal. In Rust this results in a panic, because the callback cannot be called while the drawing is happening. Interestingly there is a set_value_no_signal setter, that allows to set the scrollbar value without generating a signal. However, there are no set_max_no_signal or set_page_no_signal variations, which makes it tricky to realize that the code can backfire a signal. In fact it took me a while to encounter the bug in practice, because it requires the special condition of the scroll bar being scrolled all the way to the bottom, followed by resizing the window in a way that the height increases.
• The work-around I'm using in this particular example is to temporarily disconnect the signal during the drawing logic that messes with the scrollbar and re-connect the signal after the interaction is done. It actually turns out that doing a queue_redraw is pointless in this case when the scrollbar changes its value triggered by the drawing logic itself, because the drawing is already happening. Triggering yet another redraw is actually a waste of resources. So while the bug was not so easy to find, it has actually revealed a code smell of the C++ version, and the resolution doesn't even feel so bad.

Some thoughts about more general solutions:

The idea in #359 looks quite helpful. For instance, instead of disconnecting and reconnecting the signal, the drawing logic could have set self.is_within_redraw = true (and at the end back to false)

#[func]
fn on_scroll_changed(_self: Gd<Self>) {
    // Only try to mutable bind & trigger a redraw if we are not already drawing...
    if self.is_within_redraw {
        let _self = self.bind_mut();
        _self.base.queue_redraw()
    }
}

But the manual maintenance of self.is_within_redraw feels a bit clumsy. This makes me wonder if it would be possible to offer a kind of try_bind_mut() which basically knows whether we are already bound or not? This would allow to write:

#[func]
fn on_scroll_changed(_self: Gd<Self>) {
    if let Some(_self) = _self.try_bind_mut() {
        // No other mut binding exists, so we know we have been triggered from "externally".
        // In this case we want to trigger a redraw.
        _self.base.queue_redraw()
    } else {
        // Apparently we cannot get a mut binding. This indicates that we have triggered ourselves!
        // And assuming this can only come from the drawing/layouting logic itself, we actually don't
        // have to do anything here in this particular example...
    }
}

The interesting question is what to do in the else branch in the general case. One possibility would be to use interior mutability to encode "what I would have liked to do, but couldn't". At the end of the other methods we could check whether such actions exist and execute them there. I'm not sure if it is possible/sensible to generalize this, but a crazy idea was to have a kind of DeferredQueue that uses interior mutability like this:

#[func]
fn on_scroll_changed(&self) { // due to interior mutability we don't need &mut
    // The idea of `run_or_enqueue` would be to either execute the code block synchronously
    // if a mutable binding is possible, or to push the FnOnce into a queue with interior mutability.
    // If desirable, a plain `.enqueue()` that is consistently lazy could be offered a well.
    self.deferred_queue.run_or_enqueue(|&mut this| {
        // here we have a &mut binding
    });
}

#[func]
fn process(&mut self, delta: f64) {
    // This executes all the accumulated actions. Since the `process` function runs once per frame, the amount
    // of delay can be okay-ish. To reduce the delay the user could also place the `run_all()` strategically in
    // certain other methods.
    self.deferred_queue.run_all();
}

Not sure if this makes sense, but I feel that offering a simple way to "defer" signals/notifications could help a lot to mitigate this issue, because it breaks the cyclic nature of the problem. In many use cases the slight delay may be a reasonable price to pay for being guaranteed not to run into "signal cycle panics".

[lilizoey]

But the manual maintenance of self.is_within_redraw feels a bit clumsy. This makes me wonder if it would be possible to offer a kind of try_bind_mut() which basically knows whether we are already bound or not? This would allow to write:

#[func]
fn on_scroll_changed(_self: Gd<Self>) {
    if let Some(_self) = _self.try_bind_mut() {
        // No other mut binding exists, so we know we have been triggered from "externally".
        // In this case we want to trigger a redraw.
        _self.base.queue_redraw()
    } else {
        // Apparently we cannot get a mut binding. This indicates that we have triggered ourselves!
        // And assuming this can only come from the drawing/layouting logic itself, we actually don't
        // have to do anything here in this particular example...
    }
}

The interesting question is what to do in the else branch in the general case. One possibility would be to use interior mutability to encode "what I would have liked to do, but couldn't". At the end of the other methods we could check whether such actions exist and execute them there. I'm not sure if it is possible/sensible to generalize this, but a crazy idea was to have a kind of DeferredQueue that uses interior mutability like this:

In this particular case, you could just do

#[func]
fn on_scroll_changed(_self: Gd<Self>) {
    _self.upcast().queue_redraw()
}

and in general i think the solution is gonna have to be that people need to be a bit more fine-grained with when they do bind_mut(). like do you really need to lock the entire struct for this? can you just lock the particular field you need instead? Like using shared references more with interior mutability. Or just upcasting the Gd if you only need to call something on the godot code.

[Bromeon]

upcast() is no longer needed; since #370 you can call Godot functions on the Gd<T> directly.