Hugepages

[victorstewart]

is HSE's usage of hugepages compatible with a setting of madvise? (aka echo madvise > /sys/kernel/mm/transparent_hugepage/enabled)

i can't enable always on my machines because i allocate vast amount of hypersparse virtual memory in my network server layer so each connection has a near infinite private sequential memory space for optimal / simple memory housekeeping. without hugepages, we can have a 0 byte residence when all the buffers are empty.

[tristan957]

@nabeelmmd would be best one to comment on this, I think.

[victorstewart]

in the nearish future once i scale up my infrastructure i won't colocate the database instances on the same machines as the application servers, so this conflict will go away and i can enable always. so not a big deal if the answer is no today.

[nabeelmmd]

HSE doesn't explicitly use THP for its allocations. Also, we don't test with or recommend THP.

If there are hugepages available in the reserved pool then HSE does take advantage of it for its large allocations via anonymous memory-maps.

So setting /sys/kernel/mm/transparent_hugepage/enabled to madvise is compatible with HSE.

[smoyerx]

Also as a reminder, we documented preferred virtual memory settings here

[victorstewart]

a follow up..

i assume 1GB pages are preferred to 2MB pages... and HSE will consume as many 1GB pages as available?

[smoyerx]

We have not conducted any studies on 2MB vs 1 GB hugepages, so I don't want to speculate on the results.

[victorstewart]

i was just browsing the code in slab.c and saw KMC_HUGE_MAX and how the code is designed around 2MB pages. so i guess the topic of "are 1GB pages better" might be somewhat null unless they were designed to be used? and also per the recommended vm.nr_hugepages=256, HSE won't exploit more hugepages than that?

at large my motivation is tuning the memory model and resources provided to the application to maximize the performance of 10s-100s of GBs of hot data in memory backed by terabytes on flash. and curious whether those memory recommendations are still optimal under those conditions.

[beckerg]

Hi Victor,

Currently HSE only uses huge pages from the persistent hugetlb page pool in its "uniform memory allocator" for the purpose of improving the locality of reference of small, fixed-sized objects (which does not include user data). It breaks these 2M allocations up into 256K chunks and distributes them across CPUs affined to a given NUMA node. HSE then allocates from these chunks as needed. Given typical use cases, a 2M huge page is a good size, whereas if we used 1G huge pages I suspect we'd end up using very little of it.

I do see that we only use the MAP_HUGETLB flag in slab.c:354, so we should probably add the MAP_HUGE_2MB flag to that to ensure we don't accidentally get a 1G huge page (I don't know if that's possible, I suspect if we ask for 2M and the default huge page size is 1G then the mmap would fail, need to investigate...)

Does that help clarify our use of huge pages? FWIW, if persistent huge pages are not configured/available we just fall back to normal mmap semantics, which in some use cases the application will experience many more TLB misses that it would with huge pages.

Greg

[victorstewart]

thanks Greg! i totally get it now. yes the confusion was i assumed huge pages were being used for user data.

i've never gotten around yet to reading the code so the program model is completely unknown to me— but i probably should soon.

so then user data is stored in anonymous memory backed by 4K pages?

and will HSE allocate more and more memory to store recently accessed user data until it comes under memory pressure?

as i said i was curious how 1G hugepages might potentially be useful for reducing TLB misses when there's a truly massive amount of user data in memory (of course would require testing). so curious if you all have ever considered that path.

the sunny cove cores have 8 1G hugepage slots in the L1D and 1024 (shared with 4K) in the unified L2, so this could be could be a terabyte of TLB hot memory.

i'll be running each database shard with at least 100GB of RAM (supplied by a cgroupv2), so want to make sure it'll be fully exploited... and find out if there's anything i can do to make that so.

[beckerg]

Exactly - User data resides entirely within file-backed 4K read-only pages whose residency is primarily managed by the OS. This is the design point of HSE: It will use all available free memory in this manner (we are big fans of the "free memory is wasted memory" mantra).
Note, however, that because all these pages are read-only the kernel can easily and quickly evict cold pages to make room for new pages that are not currently resident.

The exception to this is the user data in c0 (the staging buffer for new data) which resides entirely within dirty anonymous pages and will eventually be written to disk. To avoid churn, most of HSE's critical data is allocated from our uniform memory allocator and hence resides within huge pages which cannot be paged out (if available, of course!).
We haven't thought much about using huge pages for user data, but given how data is organized and distributed throughout cN I suspect the number of TLB misses to find a key/value would be about the same regardless of page size (primarily due to the way cN is not necessarily one big chunk of data, but instead is a list of accretions of user data over time that is eventually compacted into 32M disparate chunks).

Regarding Sunny Cove's 8 1G TLB entries, HSE is thread based, so any given thread would only be able to map 8G at a time... Am I missing something? I suspect using huge pages for our c0 buffers could be a big win in large memory systems. We are also trying to work well in small memory systems (e.g., 16G RAM), so we'd need to be able to work well in both environments.
Regarding performance, obviously your best performance will be obtained if your KVDB fits entirely within RAM. If not, then the next best improvement is to use fast SSDs (e.g., Micron 9300 and 9400).

Curious, can you tell me what database you are using, number of shards, number of machines and CPU counts? I would also be interested in know what your average key and value sizes are...

Thanks!
Greg

[victorstewart]

these bleeding edge cores have a unified L2 TLB which store either x number of 4K or 1G pages or 4K + 2M pages interchangeably. so could be a mix of 4K and 1G, or all 1G.

the Sunny Cove L1D TLB can store 8 1G pages, but the core also has 1024 slots in its L2 TLB.

then Golden Cove which was just released today doubled the L1D number to 16 1G pages and it seems like the L2 is also doubled now to 2048 pages (but can't find the exact numbers).

maybe there could be some way to exploit this to extract more performance by reducing TLB misses to 0, was my thinking. but just a wild guess.

it would be simple to test it's just a matter of adding the MAP_HUGETLB to the user data file-backed mmap calls. and then it's operator configurable: simply suggest the operator to configure 1G pages when running a massive KVDB.

i'll get around to testing it myself in the future.

i wrote my own Redis Enterprise like clone on top on HSE. it's an io_uring based network layer beneath an application server running maybe 30-something base operations.

then i run custom flavors on top which add specialized operations.

i have "Hot" database with lots of specialized operations which obviate then need for transactions or multi round trips with a client to complete a meta operation. this holds the entire working set of data to run our product— something like 2TB per 1 million users with the current feature set. that's the one i want all in memory if possible.

then a "Cold" database which basically only takes writes— that data will never be accessed unless we're running some historical analysis work.

also had a "Truth" version that provided a single source of truth, but i erased that and injected such unique seeds into the applications via the orchestrator when i decided to write my own to eliminate any source of sequentialism.

i built sub maps into it, but they just map ontop the main map by appending to the base key.

i also have a list object which overflows into new keys when the values exceed 1MB in total, but they're only used in 1 place.

i built key / subkey expirations in as well.

our key sizes are > 16 bytes and probably at most than 48 bytes. the values are wide ranging, but none more than 1MB, a batch in the 64K and 10K range, then the vast majority probably 1-1K.

at the moment all the databases run single threaded. new shards trigger automatically based on to-be-decided latency statistics.

i haven't decided yet if threads are necessary at all... but i guess since HSE is designed to be thread safe i'm paying the cost for locks anyway? and it would be simple enough for me to implement lock and synchronization free because writers only write to their wholly owned subset of the data... besides one cross writer sub-map— but that's also multiple writer safe because the writers identity is embedded in the key. (part of the beauty of writing everything you use yourself is you can hand-wave away all unnecessary performance costly protections lol).