-
-
Notifications
You must be signed in to change notification settings - Fork 32.6k
bpo-33234 Improve list() pre-sizing for inputs with known lengths (no __length_hint__) #9846
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 5 commits
7602879
39a8757
ab450b3
717ca67
b678442
b40bd62
3837486
b6ff7ce
e7f8c46
0a6c8ba
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,2 @@ | ||
The list constructor will pre-size and not over-allocate when | ||
the input lenght is known. |
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -76,6 +76,31 @@ list_resize(PyListObject *self, Py_ssize_t newsize) | |
return 0; | ||
} | ||
|
||
static int | ||
list_preallocate_exact(PyListObject *self, Py_ssize_t size) | ||
{ | ||
PyObject **items; | ||
size_t allocated; | ||
|
||
allocated = (size_t)size; | ||
if (allocated > (size_t)PY_SSIZE_T_MAX / sizeof(PyObject *)) { | ||
PyErr_NoMemory(); | ||
return -1; | ||
} | ||
|
||
if (size == 0) { | ||
allocated = 0; | ||
} | ||
items = (PyObject **)PyMem_New(PyObject*, allocated); | ||
if (items == NULL) { | ||
PyErr_NoMemory(); | ||
return -1; | ||
} | ||
self->ob_item = items; | ||
self->allocated = allocated; | ||
return 0; | ||
} | ||
|
||
/* Debug statistic to compare allocations with reuse through the free list */ | ||
#undef SHOW_ALLOC_COUNT | ||
#ifdef SHOW_ALLOC_COUNT | ||
|
@@ -2649,6 +2674,19 @@ list___init___impl(PyListObject *self, PyObject *iterable) | |
(void)_list_clear(self); | ||
} | ||
if (iterable != NULL) { | ||
if (_PyObject_HasLen(iterable) && self->ob_item == NULL) { | ||
Py_ssize_t iter_len = PyObject_Size(iterable); | ||
if (iter_len == -1) { | ||
if (PyErr_ExceptionMatches(PyExc_Exception)) { | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. What tests will fail if not silence any exceptions at all? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. At least the following:
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. How can There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Ah, it is not true generator, it is a weakref. What if silence just TypeError? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I am a bit afraid if someone has similar tests/usage like these two cases and then they will receive new errors as raising all exceptions would be technically a backwards incompatible change. But if you think we should raise (maybe ignoring There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. See There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Done in 0a6c8ba |
||
PyErr_Clear(); | ||
} else { | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. nitpick: PEP 7 requires
|
||
return -1; | ||
} | ||
} | ||
if (iter_len > 0 && list_preallocate_exact(self, iter_len)) { | ||
return -1; | ||
} | ||
} | ||
PyObject *rv = list_extend(self, iterable); | ||
if (rv == NULL) | ||
return -1; | ||
|
Uh oh!
There was an error while loading. Please reload this page.