Skip to content

[memprof] Use CallStackRadixTreeBuilder in the V3 format #94708

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Jun 7, 2024
Merged

[memprof] Use CallStackRadixTreeBuilder in the V3 format #94708

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
2d3fab0
[memprof] Use CallStackRadixTreeBuilder in the V3 format
kazutakahirata May 29, 2024
06bc0ee
Migrate existing users of getCallStackPos to takeCallStackPos.
kazutakahirata Jun 7, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 14 additions & 2 deletions llvm/include/llvm/ProfileData/MemProf.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -900,9 +900,17 @@ struct LinearCallStackIdConverter {
Frames.reserve(NumFrames);
for (; NumFrames; --NumFrames) {
LinearFrameId Elem =
support::endian::readNext<LinearFrameId, llvm::endianness::little>(
Ptr);
support::endian::read<LinearFrameId, llvm::endianness::little>(Ptr);
// Follow a pointer to the parent, if any.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe point to the comment block further down that describes the radix tree format.

if (static_cast<std::make_signed_t<LinearFrameId>>(Elem) < 0) {
Ptr += (-Elem) * sizeof(LinearFrameId);
Elem =
support::endian::read<LinearFrameId, llvm::endianness::little>(Ptr);
}
// We shouldn't encounter another pointer.
assert(static_cast<std::make_signed_t<LinearFrameId>>(Elem) >= 0);
Frames.push_back(FrameIdToFrame(Elem));
Ptr += sizeof(LinearFrameId);
}

return Frames;
Expand Down Expand Up @@ -1028,6 +1036,10 @@ class CallStackRadixTreeBuilder {
getCallStackPos() const {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we remove getCallStackPos and replace its users with takeCallStackPos? Looks like existing callers are all unit tests where we could do this.

return CallStackPos;
}

llvm::DenseMap<CallStackId, LinearCallStackId> takeCallStackPos() {
return std::move(CallStackPos);
}
};

// Verify that each CallStackId is computed with hashCallStack. This function
Expand Down
22 changes: 7 additions & 15 deletions llvm/lib/ProfileData/InstrProfWriter.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -547,19 +547,11 @@ writeMemProfCallStackArray(
llvm::DenseMap<memprof::CallStackId, memprof::LinearCallStackId>
MemProfCallStackIndexes;

MemProfCallStackIndexes.reserve(MemProfCallStackData.size());
uint64_t CallStackBase = OS.tell();
for (const auto &[CSId, CallStack] : MemProfCallStackData) {
memprof::LinearCallStackId CallStackIndex =
(OS.tell() - CallStackBase) / sizeof(memprof::LinearCallStackId);
MemProfCallStackIndexes.insert({CSId, CallStackIndex});
const llvm::SmallVector<memprof::FrameId> CS = CallStack;
OS.write32(CS.size());
for (const auto F : CS) {
assert(MemProfFrameIndexes.contains(F));
OS.write32(MemProfFrameIndexes[F]);
}
}
memprof::CallStackRadixTreeBuilder Builder;
Builder.build(std::move(MemProfCallStackData), MemProfFrameIndexes);
for (auto I : Builder.getRadixArray())
OS.write32(I);
MemProfCallStackIndexes = Builder.takeCallStackPos();

// Release the memory of this vector as it is no longer needed.
MemProfCallStackData.clear();
Expand Down Expand Up @@ -695,8 +687,8 @@ static Error writeMemProfV2(ProfOStream &OS,
// uint64_t Schema entry 1
// ....
// uint64_t Schema entry N - 1
// OnDiskChainedHashTable MemProfFrameData
// OnDiskChainedHashTable MemProfCallStackData
// Frames serialized one after another
// Call stacks encoded as a radix tree
// OnDiskChainedHashTable MemProfRecordData
static Error writeMemProfV3(ProfOStream &OS,
memprof::IndexedMemProfData &MemProfData,
Expand Down