Revisit maximum .gopclntab section size

[christos68k]

We ran into this issue in production:

time="2025-02-12T17:02:31.825021378Z" level=error msg="Failed to handle mapping for PID 353399,
 file /usr/local/bin/provider: failed to extract interval data: failed to extract stack deltas
 from /usr/local/bin/provider: failure to parse golang stack deltas: failed to load .gopclntab
 section: section size 144395608 is too large"

where /usr/local/bin/provider is a Go binary from xpkg.upbound.io/upbound/provider-aws-autoscaling:v0.47.4 with .gopclntab exceeding maxBytesGoPclntab

$ size -Ad provider
provider  :
section                 size        addr
<SNIP>
.gopclntab         144395608   320681888

We can increase the limit further (e.g. 256MB?) to support binaries such as these.

[fabled]

failure to parse golang stack deltas: failed to load .gopclntab section: section size 144395608 is too large

Wow. That's huge. But inevitable with Go static linking approach.

We can increase the limit further (e.g. 256MB?) to support binaries such as these.

This will obviously temporarily bloat the profilers memory usage, and potentially cause issues will memory restrictions. It might make sense to determine if using mmap would be feasible for the large binaries. This could also made be tunable or autotunable based on the available memory.

The other problem is that on aarch64 the stack deltas are used, and probably would not with to kernel memory (see also #342). The x86_64 side probably works better due to coalescing of stack deltas to large "use frame pointer" chunks.

[christos68k]

We can increase the limit further (e.g. 256MB?) to support binaries such as these.

This will obviously temporarily bloat the profilers memory usage, and potentially cause issues will memory restrictions. It might make sense to determine if using mmap would be feasible for the large binaries. This could also made be tunable or autotunable based on the available memory.

Hmm right, I didn't read far enough down the file to see that we're loading the entire section into a slice. If we can adapt our processing to work on top of mmap, it should be better.

The other problem is that on aarch64 the stack deltas are used, and probably would not with to kernel memory (see also #342). The x86_64 side probably works better due to coalescing of stack deltas to large "use frame pointer" chunks.

On amd64, agent RSS with an increased max limit to allow for processing the section is ~240MB, so based on what you wrote and #342, I'm assuming the memory hit on arm64 would be worse. So we should try to get FP unwinding working.

[florianl]

I'd like to propose the following changes, that would allow to handle large .gopclntab sections:

• add fd int as field to pfelf.File that holds the Unix file descriptor referencing the open file (if available).
• add MmappedData()([]byte, error) to pfelf.Section. If the file descriptor is available, this functions should return a mmapped slice of data using unix.Mmap().
• either add Munmap([]byte) error to pfelf.Section or provide a dedicated helper function that calls unix.Munmap() on the mmapped slice returned by MmappedData().

[AlonMenczerLema]

@florianl is this solved by #455 ?

[fabled]

@florianl is this solved by #455 ?

For all practical purposes yes. Closing this issue. Florian please reopen if you think something was not taken into account.