Goodness of fit for density visualizations

[aloctavodia]

This shows the basic computations described in https://arxiv.org/abs/2503.01509 for assessing the quality of a density representation (histogram, KDE, or quantile dot plot). The method used the PIT evaluated using the computed density.

EDIT

This implements the functions plot_dgof and plot_dgof_dist. The first one only includes the diagnostics and the second also includes the density in the left column.

Not happy with the dgof name.

depends on arviz-devs/arviz-stats#273

For the visualization part, I guess we would like to have a two-column plot with the dist on the left and the diagnostic on the right. Something like plot_trace_dist and plot_rank_dist. To avoid duplicating the computation, we should compute the density once and use that information to both plot it on the left and calculate the visualization on the right.

For the computation part, I am not sure if we want the PIT to be included with the output of histogram/kde/qds, or do we want a separate function that uses that output.

While working on this, I realized we currently don't support bins="auto" (for more than one variable), and hence this diagnostic is usually very bad for the default histogram we compute, arviz-devs/arviz-stats#262

dt = azb.from_dict({"posterior": {"mu":pz.Normal(0, 1).rvs((4, 200), random_state=42)}})

azp.plot_dgof(dt, kind="hist")

azp.plot_dgof(dt, kind="hist", stats={"dist": {"bins":"auto"}})

azp.plot_dgof(dt, kind="kde")

azp.plot_dgoft(dt, kind="qds")

[codecov-commenter]

Codecov Report

❌ Patch coverage is 90.32258% with 9 lines in your changes missing coverage. Please review.
✅ Project coverage is 85.35%. Comparing base (381dde9) to head (c4d3c87).
⚠️ Report is 2 commits behind head on main.

Files with missing lines	Patch %	Lines
src/arviz_plots/plots/dgof_dist_plot.py	88.13%	7 Missing ⚠️
src/arviz_plots/plots/dgof_plot.py	93.75%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #383      +/-   ##
==========================================
+ Coverage   85.04%   85.35%   +0.30%     
==========================================
  Files          56       58       +2     
  Lines        6661     6827     +166     
==========================================
+ Hits         5665     5827     +162     
- Misses        996     1000       +4     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[read-the-docs-community]

Documentation build overview

📚 arviz-plots | 🛠️ Build #30783644 | 📁 Comparing c4d3c87 against latest (ac226b0)

🔍 Preview build

Show files changed (67 files in total): 📝 65 modified | ➕ 2 added | ➖ 0 deleted

File	Status
gallery/add_reference_bands.html	📝 modified
gallery/add_reference_lines.html	📝 modified
gallery/combine_plots.html	📝 modified
gallery/index.html	📝 modified
gallery/plot_autocorr.html	📝 modified
gallery/plot_bf.html	📝 modified
gallery/plot_compare.html	📝 modified
gallery/plot_convergence_dist.html	📝 modified
gallery/plot_dgof.html	➕ added
gallery/plot_dgof_dist.html	➕ added
gallery/plot_dist_ecdf.html	📝 modified
gallery/plot_dist_hist.html	📝 modified
gallery/plot_dist_kde.html	📝 modified
gallery/plot_dist_models.html	📝 modified
gallery/plot_dist_qds.html	📝 modified
gallery/plot_ecdf_coverage.html	📝 modified
gallery/plot_ecdf_pit.html	📝 modified
gallery/plot_energy.html	📝 modified
gallery/plot_ess_evolution.html	📝 modified
gallery/plot_ess_local.html	📝 modified
gallery/plot_ess_models.html	📝 modified
gallery/plot_ess_quantile.html	📝 modified
gallery/plot_forest.html	📝 modified
gallery/plot_forest_ess.html	📝 modified
gallery/plot_forest_models.html	📝 modified
gallery/plot_forest_pp_obs.html	📝 modified
gallery/plot_forest_shade.html	📝 modified
gallery/plot_khat.html	📝 modified
gallery/plot_khat_aesthetics.html	📝 modified
gallery/plot_khat_facet_cols.html	📝 modified
gallery/plot_khat_facet_grid.html	📝 modified
gallery/plot_loo_pit.html	📝 modified
gallery/plot_mcse.html	📝 modified
gallery/plot_pair.html	📝 modified
gallery/plot_pair_distribution.html	📝 modified
gallery/plot_pair_focus.html	📝 modified
gallery/plot_pair_focus_distribution.html	📝 modified
gallery/plot_parallel.html	📝 modified
gallery/plot_pava_calibration.html	📝 modified
gallery/plot_ppc_censored.html	📝 modified
gallery/plot_ppc_coverage.html	📝 modified
gallery/plot_ppc_dist.html	📝 modified
gallery/plot_ppc_interval.html	📝 modified
gallery/plot_ppc_pava_residuals.html	📝 modified
gallery/plot_ppc_pit.html	📝 modified
gallery/plot_ppc_rootogram.html	📝 modified
gallery/plot_ppc_tstat.html	📝 modified
gallery/plot_prior_posterior.html	📝 modified
gallery/plot_psense.html	📝 modified
gallery/plot_psense_quantities.html	📝 modified
gallery/plot_rank.html	📝 modified
gallery/plot_rank_dist.html	📝 modified
gallery/plot_ridge.html	📝 modified
gallery/plot_ridge_multiple.html	📝 modified
gallery/plot_trace.html	📝 modified
gallery/plot_trace_dist.html	📝 modified
api/generated/arviz_plots.PlotCollection.generate_aes_dt.html	📝 modified
api/generated/arviz_plots.plot_ecdf_pit.html	📝 modified
api/generated/arviz_plots.plot_loo_pit.html	📝 modified
api/generated/arviz_plots.plot_ppc_pit.html	📝 modified
api/generated/arviz_plots.plot_rank.html	📝 modified
api/generated/arviz_plots.plot_rank_dist.html	📝 modified
_modules/arviz_plots/plots/ecdf_plot.html	📝 modified
_modules/arviz_plots/plots/loo_pit_plot.html	📝 modified
_modules/arviz_plots/plots/ppc_pit_plot.html	📝 modified
_modules/arviz_plots/plots/rank_dist_plot.html	📝 modified
_modules/arviz_plots/plots/rank_plot.html	📝 modified

[tomicapretto]

I'm not sure I can add many additional comments about the implementation itself, as I'm not very familiar with the details.

As for the layout of the plot, I agree the two-column plot would be useful in most cases.
If possible, it can be helpful to also be able to only generate the plot you show above. I imagine it can be used with many distributions when the goal is to see the diagnostic.

[aloctavodia]

Thanks for the comment and discussion.

The easy thing to do is to have some utility function arviz-stats to compute the diagnostics, and then here we should have plot_x and then a plot_x_dist (like we have, for instance, with plot_rank and plot_rank_dist). This will give users the main features (the diagnostic plot) and then, in the future, we can think about how to improve the internals and/or expose the computation of the diagnostic to the user if we decide to do so.

[tomicapretto]

Thanks for the comment and discussion.

The easy thing to do is to have some utility function arviz-stats to compute the diagnostics, and then here we should have plot_x and then a plot_x_dist (like we have, for instance, with plot_rank and plot_rank_dist). This will give users the main features (the diagnostic plot) and then, in the future, we can think about how to improve the internals and/or expose the computation of the diagnostic to the user if we decide to do so.

Sounds good to me!

[aloctavodia]

@OriolAbril do you have suggestions for the name of these functions?