[lldb] Implement diagnostics detecting when a variable is not captured #10772

felipepiovezan · 2025-05-30T21:38:30Z

This series of commits implement diagnostics for when a variable could have been available for v or expr commads, had it been captured inside a closure.

The first commit is just an NFC patch from upstream and should be ignored, it fixes some move semantics / const correctness of data structures.
The second commit implements the v diagnostics in a language agnostic way. It is what should be upstreamed to LLVM once we can support this in C++. I punted on doing this because C++ mangling is a bit <redacted> to work with.
The third commit implements Swift support for v by doing the demangling work inside the SwiftLanguage plugin.
The fourth commit builds upon the third and adds support for expr.

…m#142052) This PR adds the missing move operators for VariableList: this class is just a wrapper around a vector, so it can use the default move operations. Subsequent patches will want to return VariableLists from functions, so the move operation is required (the copy constructors are deleted). It also fixes constness for a method in DiagnosticManager returning its list of diagnostics. Previously, the method always returned a const list, even though the method was not const itself. Subsequent patches will make use of the ability to mutate the diagnostics. (cherry picked from commit 03d1f3d)

felipepiovezan · 2025-05-30T21:39:42Z

@swift-ci test

lldb/source/Target/StackFrame.cpp

adrian-prantl

This is going to be very helpful!

adrian-prantl · 2025-05-30T21:59:07Z

lldb/source/Plugins/ExpressionParser/Swift/SwiftUserExpression.cpp

+                       "was not captured.\nHint: the variable may be available "
+                       "in a parent frame.",
+                       var_name, parent_func_name);
+}


why is this a separate function?

The thinking is that:

I expect us to one day unify this with the v case

To separate the logic of "when to emit the diagnostics" from the logic of "how to write the diagnostics". Anyone debugging this in the future will likely want to debug one or the other.

adrian-prantl · 2025-05-30T22:00:53Z

lldb/source/Plugins/LanguageRuntime/Swift/SwiftLanguageRuntimeNames.cpp

+using Node = swift::Demangle::Node;
+
+/// Returns the first child of `node` whose kind is in `kinds`.
+NodePointer getFirstChildOfKind(NodePointer node,


Move to SwiftDemangle.h?

I wasn't aware of that file!
I've moved them there, but beware that that header has no corresponding cpp file, so we should not let it grow too wildly.

adrian-prantl · 2025-05-30T22:05:19Z

lldb/source/Target/StackFrame.cpp

+      lang_plugin->FindParentOfClosureWithVariable(missing_var_name, sc);
+  if (!parent_func)
+    return "";
+  return llvm::formatv("Current frame is a closure.\nA variable named '{0}' "


Why is the text duplicated here?

The second commit implements the v diagnostics in a language agnostic way. It is what should be upstreamed to LLVM once we can support this in C++. I punted on doing this because C++ mangling is a bit to work with.

I see.

felipepiovezan · 2025-06-02T19:43:30Z

@swift-ci test

felipepiovezan · 2025-06-02T19:44:25Z

Addressed review comments, fixed linux test issue by specifying an extra regex to match against, fixed windows testing issue by adding the @swiftTest annotation

felipepiovezan · 2025-06-03T21:11:40Z

I've just realized why Linux was failing: I had added a test looking for a variable in an async function, which is not supposed to be working on linux right now.
Updated the test.

felipepiovezan · 2025-06-03T21:11:46Z

@swift-ci test

felipepiovezan · 2025-06-04T05:09:15Z

@swift-ci test macos platform

felipepiovezan · 2025-06-04T14:10:16Z

@swift-ci test macos platform

felipepiovezan · 2025-06-04T14:10:44Z

CI is being troublesome right now, the macOS job is timing out

adrian-prantl · 2025-06-04T15:49:43Z

lldb/source/Target/Language.cpp

+
+  llvm::SmallVector<Function *> parents;
+  Function *root = sc.function;
+  if (root == nullptr)


Suggested change

if (root == nullptr)

if (!root)

Out of curiosity, why do you prefer this version? Let's say that that you encounter the two versions of this line out in the wild. One of them tells you root is implicitly convertible to bool, the other one tells you root is a pointer. Do you believe the former is better?

felipepiovezan · 2025-06-04T16:05:30Z

@swift-ci test macos platform

kastiglione · 2025-06-04T17:14:21Z

lldb/test/API/lang/swift/closures_var_not_captured/TestSwiftClosureVarNotCaptured.py

+    test.assertIn(expected_error, error)
+
+    test.expect(f"frame variable {var_name}", substrs=[expected_error], error=True)
+    test.expect(f"frame variable {var_name} + 1", substrs=[expected_error], error=True)


frame variable doesn't support expressions. The expression would be interpreted as a request to print 3 variables (var_name, +, and 1).

Good catch! Updated

kastiglione · 2025-06-04T17:21:05Z

lldb/source/Plugins/ExpressionParser/Swift/SwiftUserExpression.cpp

+  return llvm::formatv("Current frame is a closure.\nA variable named '{0}' "
+                       "exists in function '{1}', but it "
+                       "was not captured.\nHint: the variable may be available "
+                       "in a parent frame.",


Is the hint contradictory (specifically the phrase "may be available")? The message says "A variable named <v> exists in function <f>" – doesn't that mean a variable is guaranteed to be available in a parent frame?

No, the function where a closure is declared is not necessarily a parent frame of the frame where the closure was called

oh I see my misunderstanding, the diagnostic doesn't attempt check the stack for the function which has the found variable, which I assumed it would. I had pictured this as a runtime check, but it's more of a static check?

This ties back to my other question #10772 (comment): In cases where the found function is on the stack (ex: call chains into a non-escaping closures in Swift), then should lldb print the variable using the ancestor frame where it exists?

I think it is impossible to do this:

func make_closure(arg: Int) { if (arg < 0} return { /*empty closure */} closure = make_closure(arg - 1); closure(); return { print ("hello from closure") } }

Let's say we are stopped inside the print line and we try to do a expr arg. There might be N frames, all with the same name make_closure. Which one has the correct arg variable?

Even if we somehow found out the correct frame, this would require a lot of changes in the debugger, as the current code assumes expressions are running with variables all in the same frame.

Since it's known that a variable exists in an ancestor frame

This is a false premise: it is not known that the variable exists in an ancestor frame. Only that it exists in an ancestor scope

The misunderstanding I've had might not be unique to me. What do you think of:

A variable named '{0}' existed where the current closure was defined (function '{1}'), but it was not captured. If '{1}' is in the call stack, select that frame and try again.

I personally think either version works, but IIRC @adrian-prantl had some thoughts on being more verbose?

kastiglione · 2025-06-04T17:24:22Z

Since it's known that a variable exists in an ancestor frame, was it discussed to use that variable? In other words, should lldb print the variable for the user, and if not, why not?

felipepiovezan · 2025-06-04T20:49:13Z

@swift-ci test

jimingham · 2025-06-04T22:22:57Z

Do we have a way of determining which block the closure was captured from? You might see:

func foo() {
  // LOTS OF CODE
  {
      takes_closure() {
          // Set a breakpoint here and print bar by accident
      }
   }
   // LOTS MORE CODE
    if whatever {
      let bar = 20
  }
  // STILL MORE CODE
}

Then we'll find bar and say "bar was in the parent frame but wasn't captured." would be confusing... If you knew the defining block, you could crawl outward from there so you only say it is in the parent frame if it really was capturable.

felipepiovezan · 2025-06-04T22:46:27Z

Do we have a way of determining which block the closure was captured from? You might see:
func foo() {
  // LOTS OF CODE
  {
      takes_closure() {
          // Set a breakpoint here and print bar by accident
      }
   }
   // LOTS MORE CODE
    if whatever {
      let bar = 20
  }
  // STILL MORE CODE
}
Then we'll find bar and say "bar was in the parent frame but wasn't captured." would be confusing... If you knew the defining block, you could crawl outward from there so you only say it is in the parent frame if it really was capturable.

I thought about that, but I'm not sure we have a way of doing this. We really only have the mangled names to go by...

We could potentially find the line where the variable was defined and compare this to the line where the closure is defined?

jimingham · 2025-06-04T22:54:10Z

If you have the defining source line for the closure, then you should be able to find the block that contains that line in the parent function. That would also be more efficient since you wouldn't be running through all the blocks in the function, just the ones visible to the defining block.

felipepiovezan · 2025-06-05T15:18:38Z

If you have the defining source line for the closure

We have a Function/SC object defining the closure and a Function/SC object defining the parent.
We can call SymbolContext::GetFunctionStartLineEntry on the closure and find its first line, but I don't think that guarantees there will be a block in the parent function containing that line, right?

Blocks only have a range of addresses, and the range of addresses of the closure and the defining function are non-overlapping (which makes sense, they are different functions entirely). Even if we were to take the first address of the closure, map that to a line number, there is no guarantee we would find that line number in the parent function...

Here, it would have been useful if the SymbolContextScope hierarchy mirrored the dwarf hierarchy, but AFAICT that is lost in translation. We can see that here: Function *Function::CalculateSymbolContextFunction() { return this; }.

Did you have something else in mind?

felipepiovezan · 2025-06-05T15:27:59Z

Would a line number comparison suffice / approximate this well enough?

felipepiovezan · 2025-06-05T15:42:31Z

Replaced std::map with llvm::MapVector to fix non-determinism and incorrect iteration order over map elements. Added test for that using variables inside closure shadowing other outer variables.

jimingham · 2025-06-05T17:47:09Z

If you have the defining source line for the closure

We have a Function/SC object defining the closure and a Function/SC object defining the parent. We can call SymbolContext::GetFunctionStartLineEntry on the closure and find its first line, but I don't think that guarantees there will be a block in the parent function containing that line, right?

Blocks only have a range of addresses, and the range of addresses of the closure and the defining function are non-overlapping (which makes sense, they are different functions entirely). Even if we were to take the first address of the closure, map that to a line number, there is no guarantee we would find that line number in the parent function...

Here, it would have been useful if the SymbolContextScope hierarchy mirrored the dwarf hierarchy, but AFAICT that is lost in translation. We can see that here: Function *Function::CalculateSymbolContextFunction() { return this; }.

Did you have something else in mind?

If we have the Function object for the closure, we should have the line where it is defined. That line most likely also has line table entry in the parent function. That should be the block that determines the variable scope of the closure.

If this is really hard, this is probably over-kill, but then we need to make it clear that we are just guessing that it's a problem with capture, and not just a misspelling.

This is useful for checking whether a variable is in scope inside a specific block.

…vars in "frame var" This commit adds the language-agnostic logic to support diagnosing when a `frame var` command failed because a variable is not visible in the current frame, but the current frame is a "closure-like"(*) function and it could have access to the variable, had it been captured by the closure. (*) A lambda in C++, a block in C or Objective C, a closure in Swift. The logic relies on language plugins being able to produce the name of the function where the current closure is defined, given the mangled name of the closure. Since this commit does not implement support for this in any languages, it is NFCI. It contains all the necessary pieces to upstream the code, once we implement Language support for at least one upstream language.

This commit implements SwiftLanguage::GetParentNameIfClosure, effectively adding support for diagnosing when a `frame var` command inside a closure targets a variable that was not captured.

This commit implements the same diagnostic that was previously implemented for frame var. The strategy here is slightly different due to how expression evaluation works. We have to pattern match this diagnostic from the compiler: ``` error: <EXPR>:6:1: cannot find 'NAME' in scope 1 + NAME ^~~~ ``` If we find it, we trigger the usual search for a variable named `NAME`. If it is found, the diagnostic is replaced with: ``` error: <EXPR>:6:5: Current frame is a closure. A variable named 'NAME' exists in function 'foo(_:)', but it was not captured. Hint: the variable may be available in a parent frame. 1 + NAME ^~~~ ``` Implementation note: the pattern matching could be a _lot_ simpler if we don't insist on replacing the original diagnostic and, instead, append to it.

felipepiovezan · 2025-06-10T18:28:35Z

If you have the defining source line for the closure

We have a Function/SC object defining the closure and a Function/SC object defining the parent. We can call SymbolContext::GetFunctionStartLineEntry on the closure and find its first line, but I don't think that guarantees there will be a block in the parent function containing that line, right?
Blocks only have a range of addresses, and the range of addresses of the closure and the defining function are non-overlapping (which makes sense, they are different functions entirely). Even if we were to take the first address of the closure, map that to a line number, there is no guarantee we would find that line number in the parent function...
Here, it would have been useful if the SymbolContextScope hierarchy mirrored the dwarf hierarchy, but AFAICT that is lost in translation. We can see that here: Function *Function::CalculateSymbolContextFunction() { return this; }.
Did you have something else in mind?

If we have the Function object for the closure, we should have the line where it is defined. That line most likely also has line table entry in the parent function. That should be the block that determines the variable scope of the closure.

If this is really hard, this is probably over-kill, but then we need to make it clear that we are just guessing that it's a problem with capture, and not just a misspelling.

It was quite hard tbh, but I think I did it :)
Line Table scanning like this might have some pitfalls, but I don't think we have any false positives (i.e. we said a variable existed when it didn't). I also added a fallback where we don't mention variables defined at a line greater than the line where the closures are defined; this is needed because swift async functions are split into multiple functions / line tables, and things are not very pretty in that case.

felipepiovezan · 2025-06-10T18:29:53Z

@swift-ci test

felipepiovezan · 2025-06-10T18:31:34Z

lldb/include/lldb/Symbol/Variable.h

@@ -91,6 +91,9 @@ class Variable : public UserID, public std::enable_shared_from_this<Variable> {

  bool IsInScope(StackFrame *frame);

+  /// Returns true if this variable is in scope at `addr` inside `block`.
+  bool IsInScope(Block &block, Address addr);
+


Upstreaming this here llvm#143572

felipepiovezan requested review from adrian-prantl and Michael137 May 30, 2025 21:38

felipepiovezan requested a review from a team as a code owner May 30, 2025 21:38

felipepiovezan commented May 30, 2025

View reviewed changes

lldb/source/Target/StackFrame.cpp Show resolved Hide resolved

adrian-prantl reviewed May 30, 2025

View reviewed changes

felipepiovezan force-pushed the felipe/enhance_not_found_diagnostic branch from d57be96 to ed8ae16 Compare June 2, 2025 19:42

felipepiovezan force-pushed the felipe/enhance_not_found_diagnostic branch from ed8ae16 to d864c20 Compare June 3, 2025 21:11

adrian-prantl approved these changes Jun 4, 2025

View reviewed changes

kastiglione reviewed Jun 4, 2025

View reviewed changes

felipepiovezan force-pushed the felipe/enhance_not_found_diagnostic branch from d864c20 to dcd9331 Compare June 4, 2025 20:38

felipepiovezan force-pushed the felipe/enhance_not_found_diagnostic branch from dcd9331 to 632fab8 Compare June 5, 2025 15:41

[lldb][nfc] Factor out code checking if Variable is in scope

66081ea

This is useful for checking whether a variable is in scope inside a specific block.

felipepiovezan force-pushed the felipe/enhance_not_found_diagnostic branch from 632fab8 to ec07e03 Compare June 10, 2025 18:14

felipepiovezan added 3 commits June 10, 2025 11:25

[lldb] Implement swift diagnostic for non-captured vars in "frame var"

a2ddd87

This commit implements SwiftLanguage::GetParentNameIfClosure, effectively adding support for diagnosing when a `frame var` command inside a closure targets a variable that was not captured.

felipepiovezan force-pushed the felipe/enhance_not_found_diagnostic branch from ec07e03 to 0cefdfc Compare June 10, 2025 18:25

felipepiovezan commented Jun 10, 2025

View reviewed changes

[lldb] Implement diagnostics detecting when a variable is not captured #10772

Are you sure you want to change the base?

[lldb] Implement diagnostics detecting when a variable is not captured #10772

Conversation

felipepiovezan commented May 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

felipepiovezan commented May 30, 2025

Uh oh!

Uh oh!

adrian-prantl left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

felipepiovezan commented Jun 2, 2025

Uh oh!

felipepiovezan commented Jun 2, 2025

Uh oh!

felipepiovezan commented Jun 3, 2025

Uh oh!

felipepiovezan commented Jun 3, 2025

Uh oh!

felipepiovezan commented Jun 4, 2025

Uh oh!

felipepiovezan commented Jun 4, 2025

Uh oh!

felipepiovezan commented Jun 4, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

felipepiovezan Jun 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

felipepiovezan commented Jun 4, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kastiglione Jun 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kastiglione Jun 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

felipepiovezan Jun 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kastiglione commented Jun 4, 2025

Uh oh!

felipepiovezan commented Jun 4, 2025

Uh oh!

jimingham commented Jun 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

felipepiovezan commented Jun 4, 2025

Uh oh!

felipepiovezan commented May 30, 2025 •

edited

Loading

felipepiovezan Jun 4, 2025 •

edited

Loading

kastiglione Jun 4, 2025 •

edited

Loading

kastiglione Jun 4, 2025 •

edited

Loading

felipepiovezan Jun 4, 2025 •

edited

Loading

jimingham commented Jun 4, 2025 •

edited

Loading

felipepiovezan commented Jun 5, 2025 •

edited

Loading