-
-
Notifications
You must be signed in to change notification settings - Fork 466
Fix az.compare stacking weights not sum up to one #2492
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #2492 +/- ##
==========================================
- Coverage 86.84% 86.83% -0.02%
==========================================
Files 124 124
Lines 13031 13020 -11
==========================================
- Hits 11317 11306 -11
Misses 1714 1714 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
OriolAbril
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good, thanks! I will try playing with it locally and make sure we include it in the upcoming release
|
also requesting more reviews to also see how to port that to arviz-stats too. |
aloctavodia
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM! I will add it to arviz-stats. Thanks @littleTT0704
jordandeklerk
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM. Thanks for catching this!
Description
While using
az.compare(models, method="stacking"), I encountered the same problem as mentioned in #2359, where theweightscolumn in the returned dataframe sums up to greater than 1.My assumption is that
scipy.optimize.minimizeconverts our dict-type constraints to a list ofNonLinearConstraint, which over-complicates the problem inaz.compare, since the sum of weights is a linear combination.Moreover, by removing the
np.concatenateoperation inw_fullerand using vectorized operations, we can increase the efficiency of the optimizer.In my extreme use case of comparing 54 models, I was able to reduce computation time by 14.3% (543s -> 465s) while getting the correct weight sum (3.636 -> 1.000).
However, I was not able to synthesize some test data to capture this behavior. Although we already have some test cases that asserts sum of weights should be 1, the models are too simple to produce a failure.
Checklist
📚 Documentation preview 📚: https://arviz--2492.org.readthedocs.build/en/2492/