Skip to content
Closed
Show file tree
Hide file tree
Changes from 11 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 25 additions & 0 deletions .chloggen/nil-empty-slice.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
# Use this changelog template to create an entry for release notes.

# One of 'breaking', 'deprecation', 'new_component', 'enhancement', 'bug_fix'
change_type: 'breaking'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was okay merging this with the expectation that we do not consider this a breaking change, but rather a change in an unspecified part of confmap. If we think this is a breaking change, then I am against merging this, since confmap is 1.0

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm a bit on the fence here.

  • On one hand this does change the current output data type of confmap so it will break users who rely on the type or format.
  • On the other hand, the current behavior is subtle - it is not documented or specified anywhere. And from the PR description it sounds like the current behavior should not have been there in the first place. I'm OK with changing this back to bug_fix if treating it as a breaking change will block it.

Copy link
Contributor

@evan-bradley evan-bradley Dec 13, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've finally spent some time digging into this, and I think we should consider merging this after we define what we want to be the correct behavior. In my opinion, the current behavior is undefined and that's a bug in confmap, even if changing the existing behavior will break users. Personally, I'd like to keep a 1:1 mapping between YAML <-> map[string]any <-> confmap.Conf <-> Config struct as much as is reasonable.

I'm also not sure if we should keep zeroSliceHookFunc: removing it from our list of decoder hooks only breaks a single test in cmd/mdatagen, and that test passes with the hook removed if the changes in this PR are applied. In general, we should likely try to keep a distinction between []any(nil) and any{}, whereas the hook currently maps []any(nil) to []any{} (and ToStringMap, without the changes in this PR, maps []any{} to []any(nil)). Additionally, if I understand the changes in #11734, I'm not 100% sure the hook does much anymore.

Edit: I see now that the hook will clear out a slice during unmarshaling. I will look into how we can confirm this with a test case.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I created a PR to illustrate a little of what I have in mind here: #11882.

I ran the k8s attributes processor tests with this change and didn't see any failing tests from removing zeroSliceHookFunc.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@evan-bradley thanks for the great explanation and the illustration in the PR. Let me know how you think we should proceed here. Do we want to replace my PR with the one you have open?


# The name of the component, or a single word describing the area of concern, (e.g. otlpreceiver)
component: confmap

# A brief description of the change. Surround your text with quotes ("") if it needs to start with a backtick (`).
note: "Correctly differentiate between a nil and empty slice in `ToStringMap`"

# One or more tracking issues or pull requests related to the change
issues: [11749]

# (Optional) One or more lines of additional information to render under the primary note.
# These lines will be padded with 2 spaces and then inserted directly into the document.
# Use pipe (|) for multiline entries.
subtext:

# Optional: The change log or logs in which this entry should be included.
# e.g. '[user]' or '[user, api]'
# Include 'user' if the change is relevant to end users.
# Include 'api' if there is a change to a library API.
# Default: '[user]'
change_logs: [api]
4 changes: 4 additions & 0 deletions confmap/confmap.go
Original file line number Diff line number Diff line change
Expand Up @@ -133,6 +133,10 @@
return c
case []any:
var newSlice []any
if m == nil {
return newSlice
}

Check warning on line 138 in confmap/confmap.go

View check run for this annotation

Codecov / codecov/patch

confmap/confmap.go#L137-L138

Added lines #L137 - L138 were not covered by tests
Comment on lines +136 to +138
Copy link
Contributor Author

@mahadzaryab1 mahadzaryab1 Nov 26, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mx-psi This block is currently not covered by tests because we're not decoding nil values in this PR (that will happen in #11734). We could (1) remove this if block here and add it in the following PR; (2) leave it in here but merge with missing code coverage; (3) add a unit test for this function directly. Let me know what you think.

newSlice = []any{}
for _, e := range m {
newSlice = append(newSlice, sanitizeExpanded(e, useOriginal))
}
Expand Down
11 changes: 8 additions & 3 deletions confmap/confmaptest/configtest_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -30,11 +30,16 @@ func TestLoadConf(t *testing.T) {
assert.Equal(t, map[string]any{"floating": 3.14}, cfg.ToStringMap())
}

func TestToStringMapSanitizeEmptySlice(t *testing.T) {
func TestToStringMapSanitizeNil(t *testing.T) {
cfg, err := LoadConf(filepath.Join("testdata", "nil.yaml"))
require.NoError(t, err)
assert.Equal(t, map[string]any{"slice": nil}, cfg.ToStringMap())
}

func TestToStringMapEmptySlice(t *testing.T) {
cfg, err := LoadConf(filepath.Join("testdata", "empty-slice.yaml"))
require.NoError(t, err)
var nilSlice []interface{}
assert.Equal(t, map[string]any{"slice": nilSlice}, cfg.ToStringMap())
assert.Equal(t, map[string]any{"slice": []interface{}{}}, cfg.ToStringMap())
}

func TestValidateProviderScheme(t *testing.T) {
Expand Down
2 changes: 1 addition & 1 deletion confmap/confmaptest/testdata/empty-slice.yaml
Original file line number Diff line number Diff line change
@@ -1 +1 @@
slice: [] # empty slices are sanitized to nil in ToStringMap
slice: []
1 change: 1 addition & 0 deletions confmap/confmaptest/testdata/nil.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
slice:
Loading