API: Use lowest precision floating type found in reindex with columns #51444
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
phofl
reviewed
Feb 17, 2023
| expected = DataFrame( | ||
| { | ||
| "a": np.array([1, 2, 3], dtype="float32"), | ||
| "d": np.array([np.nan, np.nan, np.nan], dtype="float32"), |
There was a problem hiding this comment.
Not sure if this is a good idea. This might lead to precision issues if you don't expect this.
Is there a good argument for making this float32 (internally speaking)
There was a problem hiding this comment.
Didn't mean to PR so early. This should only apply to all nan cols. We'd use the highest precision float in the frame found if any, otherwise it's float64
There was a problem hiding this comment.
Yep I understand that. But if you set values into the all nan columns later on you are stuck with float32
simonjayhawkins
added
Bug
Reshaping
|
I think I overcomplicated this. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
doc/source/whatsnew/vX.X.X.rstfile if fixing a bug or adding a new feature.The problem there was that the
.valuescall herepandas/pandas/core/reshape/reshape.py
Line 746 in a131266
upcasts the values, so the float32 will become float64.
I had to modify reindex in order to make this possible, since I don't think you can easily tell whether an all-nan col was there before or was created because of re-indexing.