Request to add discriminator based guidance

[anubhav1997-sony]

Would it be possible to get discriminator based guidance from the paper "Refining Generative Process with Discriminator Guidance in Score-based Diffusion Models" that was published at ICML'23.

Paper link: https://proceedings.mlr.press/v202/kim23i/kim23i.pdf

Thanks,
Anubhav

[sayakpaul]

Thanks for expressing this interest.

However, your issue description is not really definitive enough for the maintainers to even consider if it's worth the integration. And in case we could suggest any alternatives to achieve similar results. Just the paper name doesn't really cut it (going through the entire paper isn't efficient, either). It would have been better to have a small description of what the technique is about and how it helps with performance improvements, etc.

Regardless, I welcome you to contribute it as a community pipeline.

[anubhav1997-sony]

Hello, the paper proposes using a discriminator for guidance when using a score based sampling approach. I believe the only difference in the current codebase would be in the diffusers.KarrasVeScheduler.

Here is an implementation from the original authors where they have updated the EDM sampling code to incorporate a discriminator - https://github.com/alsdudrla10/DG/blob/main/generate.py

Additionally the discriminator is trained for real/fake classification. The authors had used the same U-Net based structure without the decoder part with a classification layer. Here is the training code from the original authors - https://github.com/alsdudrla10/DG/blob/main/train.py

The authors have shown improved image quality performance on multiple datasets.

[sayakpaul]

If it's on toy datasets, I am not yet sure about the true value-add from the perspective of a feature integration for the library.

[anubhav1997-sony]

What do you mean by "toy datasets"??

It has been tested on multiple unconditional and conditional generation scenarios including and not limited to CIFAR-10, CelebA/FFHQ and ImageNet. I am also hoping integration into the diffusers library will allow us to use it for text-to-image diffusion models as well.

Additionally, from a value add perspective, this approach has been the foundation for various other approaches and has been used by various authors for various applications such as debiasing diffusion models ("Training Unbiased Diffusion Models From Biased Dataset (ICLR 2024), paper link - "https://openreview.net/forum?id=39cPKijBed ; code-https://github.com/alsdudrla10/TIW-DSM).

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.