Skip to content

Pass a reference to C++ container to the Python callback #1200

Open
@uentity

Description

@uentity

Hi!

I faced this issue while implementing a Python's os.walk-like function for my tree structure. I hoped to mimic the ability of callback to edit passed list of directories in-place and guide further search.

Suppose we have the following in C++:

#include <pybind11/stl_bind.h>

PYBIND11_MAKE_OPAQUE(std::vector<double>);
namespace py = pybind11;
...

py::bind_vector<std::vector<double>>(m, "d_list", py::module_local(false));
// this will work as expected because I can explicitly point that return value is a reference
m.def("test_static_d", []() -> std::vector<double>& {
	static std::vector<double> v{0, 0};
	return v;
}, py::return_value_policy::reference);

// callback will fail to edit passed vector, because a copy is involved somewhere?
m.def("test_callback", [](std::function<void(std::vector<double>&)> f) {
	std::vector<double> v{0, 0};
	f(v);
	for(const auto& l : v) {
		std::cout << l << ' ';
	}
	std::cout << std::endl;
});

And now the Python code:

In [2]: a = example.test_static_d()

In [3]: a.append(1)

In [4]: a
Out[4]: d_list[0, 0, 1]

In [5]: b = example.test_static_d()

In [6]: b
Out[6]: d_list[0, 0, 1]
# so far so good
In [7]: b is a
Out[7]: True

# callback seems to modify a copy of vector
def f(v) :
    v.append(1.)
    print(v)

# C++ function don't get updated vector after callback invoke
In [1]: example.test_callback(f)
d_list[0, 0, 1]
0 0 

Is there a way to specify that callback should take an opaque container by reference (in order to modify it in-place on Python side)?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions