Skip to content

Add memoryview_scoped_release to manage short-lived memoryviews #2307

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions docs/advanced/pycpp/numpy.rst
Original file line number Diff line number Diff line change
Expand Up @@ -419,6 +419,23 @@ managed by Python. The user is responsible for managing the lifetime of the
buffer. Using a ``memoryview`` created in this way after deleting the buffer in
C++ side results in undefined behavior.

To prevent undefined behavior, you can call the ``release`` function on a
``memoryview``. After ``release`` is called, any further operation on the view
will raise a ``ValueError``. For short lived buffers, consider using
``memoryview_scoped_release`` to release the memoryview:

.. code-block:: cpp

{
auto view = py::memoryview::from_memory(buffer, size);
py::memoryview_scoped_release release(view);

some_function(view);
}

// operations on the memoryview after this scope exits will raise a
// ValueError exception

We can also use ``memoryview::from_memory`` for a simple 1D contiguous buffer:

.. code-block:: cpp
Expand Down
15 changes: 15 additions & 0 deletions include/pybind11/pybind11.h
Original file line number Diff line number Diff line change
Expand Up @@ -2181,6 +2181,21 @@ void print(Args &&...args) {
detail::print(c.args(), c.kwargs());
}

#if PY_VERSION_HEX >= 0x03020000
/// Release the underlying buffer exposed by the memoryview object when this
/// object goes out of scope. Any further operation on the view raises a
/// ValueError.
///
/// Only available in Python 3.2+
class memoryview_scoped_release {
public:
explicit memoryview_scoped_release(memoryview view) : m_view(std::move(view)) {}
~memoryview_scoped_release() { m_view.attr("release")(); }
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, CPython implementation is in 256 different files. To save everyone the trouble, attr("release") is defined here: https://github.com/python/cpython/blob/63298930fb531ba2bb4f23bc3b915dbf1e17e9e1/Objects/memoryobject.c#L1068

private:
memoryview m_view;
};
#endif

error_already_set::~error_already_set() {
if (m_type) {
gil_scoped_acquire gil;
Expand Down
5 changes: 4 additions & 1 deletion include/pybind11/pytypes.h
Original file line number Diff line number Diff line change
Expand Up @@ -1514,7 +1514,8 @@ class memoryview : public object {
This method is meant for providing a ``memoryview`` for C/C++ buffer not
managed by Python. The caller is responsible for managing the lifetime
of ``ptr`` and ``format``, which MUST outlive the memoryview constructed
here.
here. Consider using ``memoryview_scoped_release`` to manage the lifetime
for short-lived memoryview objects.

See also: Python C API documentation for `PyMemoryView_FromBuffer`_.

Expand Down Expand Up @@ -1568,6 +1569,8 @@ class memoryview : public object {
This method is meant for providing a ``memoryview`` for C/C++ buffer not
managed by Python. The caller is responsible for managing the lifetime
of ``mem``, which MUST outlive the memoryview constructed here.
Consider using ``memoryview_scoped_release`` to manage the lifetime
for short-lived memoryview objects.

This method is not available in Python 2.

Expand Down
10 changes: 10 additions & 0 deletions tests/test_pytypes.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -434,4 +434,14 @@ TEST_SUBMODULE(pytypes, m) {
m.def("weakref_from_object", [](const py::object &o) { return py::weakref(o); });
m.def("weakref_from_object_and_function",
[](py::object o, py::function f) { return py::weakref(std::move(o), std::move(f)); });

#if PY_VERSION_HEX >= 0x03020000
m.def("test_memoryview_scoped_release", [](const py::function &f) {
const char* buf = "\x42";
auto view = py::memoryview::from_memory(buf, 1);
py::memoryview_scoped_release release(view);
f(view);
});
#endif

}
14 changes: 14 additions & 0 deletions tests/test_pytypes.py
Original file line number Diff line number Diff line change
Expand Up @@ -589,3 +589,17 @@ def callback(wr):
del obj
pytest.gc_collect()
assert callback.called


@pytest.mark.skipif(sys.version_info < (3, 2), reason="API not available")
def test_memoryview_scoped_release():
class C:
def fn(self, view):
self.view = view
assert bytes(view) == b"\x42"

c = C()
m.test_memoryview_scoped_release(c.fn)
assert hasattr(c, "view")
with pytest.raises(ValueError):
bytes(c.view)