resizing a bytearray memory penalty

[kjm1102]

If I shrink a byte array I incur a ram loss equal to the new size

import gc
Q=bytearray(20000)
gc.collect(); b=gc.mem_free(); print(b)
Q=Q[:18000]
gc.collect(); a=gc.mem_free(); print(a, '  diff', b-a)

167536
149536   diff 18000

this happens even if I move the resizing into a function so that it is 'out of scope' on return. I'd expect this result if I'd used R=Q[:18000] but it seems that, even with Q=Q[:18000], I'm gonna loose 18k rather than gain 2k. Since I'm a bit short of ram I'm wondering if there is a way around this?

[jimmo]

@kjm1102 I actually get -1904 when I run this on a Pico and on a Pyboard (the expected result)

MicroPython's GC is "imprecise" which means that it can have temporary false positives (but not false negatives) in it's "is this allocation still referenced" check. i.e. it can occasionally think an allocation is in-use for some additional period of time. This can be due to e.g. a pointer still being held in a CPU register.

Compare to e.g. a reference counted GC, or one that uses a lot of additional memory and overhead to perfectly track references.

As a result can make it difficult to make inferences from tests like this. But I promise you that resizing this way does work, and the blocks will eventually be reclaimed.

FWIW, following on from https://github.com/orgs/micropython/discussions/14088 the "correct" way to resize a bytearray without copying is with slice assignment:

Q[-2000:] = b''   # trim the last 2000 elements from Q

However while this does technically resize the bytearray it does not free the unused blocks back. There's a TODO in the code for this: https://github.com/micropython/micropython/blob/master/py/objarray.c#L517 I think it would be relatively easy to fix (perhaps this code was written before we implemented support for this in the GC).

[kjm1102]

Oh yes! I might not get the 2k back but I don't loose the 18k that Q=Q[:1800] was causing, very nice.