GenerativeZoo: A Unified Repository of Generative Models for Computer Vision

Francisco Caetano1, Christiaan Viviers1, Peter H.N. de With1, Fons van der Sommen1
1Eindhoven University of Technology

Technical Report Code
description of what image is showing.

The GenerativeZoo is a collection of generative algorithms and techniques implemented in Python using PyTorch. This work is part of the Xecs TASTI project, nr. 2022005.

Goals

Welcome to the Generative Zoo, a collection of (so far) 27 diverse models aggregated in one repository. Our main objectives are:

  • Aggregation: Bring together a variety of generative models in a single, unified platform.
  • Usability: Simplify the process of training, sampling, and evaluating out-of-distribution (OOD) detection across all models.
  • Standardization: Establish a standardized framework for working with these models, ensuring consistency and ease of use.

This repository is designed to grow, continually incorporating new models and advancements in generative modeling, with a big focus on Computer Vision tasks.

Models

To ensure consistency and usability across all models in the Generative Zoo, the following implementation rules were defined:

  • Self-contained Code: Each model's code should be provided, if possible, in a self-contained manner, ensuring it can be easily understood and utilized independently.
  • Model Methods: Training and sampling functionalities should be implemented as methods within the model class.
  • Command Line Control: Training should be controllable via command line arguments, allowing for flexible and repeatable experiments.
  • Full Documentation: Comprehensive documentation must be provided for all parameters, detailing their purpose and usage.
  • Example Script: An example script demonstrating the main functions of each model should be included, showing users how to effectively train and sample from the model.
  • References: Both the paper that proposed the model and the repositories that were used to create the Zoo's implementation should be explicitly mentioned.

Implemented Models

The GenerativeZoo framework currently aggregates a total of 27 models spanning 4 distinct generative model families and 9 different model types, reflecting the breadth of approaches in modern generative modeling. These include Latent Variable Models (VAEs, GANs, and Diffusion Models), Score-based Models (Score Matching and Flow Matching), Autoregressive Models, and Flow-based Models. The table below provides a summary of the models implemented in the Generative Zoo.

Model Family Model Type Model Source
Latent Variable VAE Vanilla VAE arXiv
Conditional VAE OpenReview
Hierarchical VAE arXiv
GAN Adversarial VAE arXiv
DC-GAN arXiv
Conditional GAN arXiv
CycleGAN arXiv
Prescribed GAN arXiv
Wasserstein GAN with Gradient Penalty arXiv
Diffusion Models DDPM arXiv
Conditional DDPM arXiv
Diffusion AE arXiv
Stable Diffusion Stable Diffusion + LoRA arXiv
ControlNet arXiv
InstructPix2Pix arXiv
Score-based Score Matching SGM arXiv
NCSNv2 arXiv
Flow Matching Flow Matching arXiv
Conditional Flow Matching arXiv
Rectified Flows arXiv
Autoregressive Transformer VQ-VAE + Transformer arXiv
VQ-GAN + Transformer arXiv
PixelCNN PixelCNN arXiv
Flow-based Normalizing Flows Vanilla Flow arXiv
RealNVP arXiv
Glow arXiv
Flow++ arXiv

Datasets

You can implement your own dataloaders, but the following ones are already provided in the zoo, as well as tools to automatically download the datasets or instructions on how to get them.

Grayscale Datasets RGB Datasets
MNIST CIFAR-10
FashionMNIST CIFAR-100
ChestMNIST++ SVHN
OctMNIST++ Places365
PneumoniaMNIST++ DTD
TissueMNIST++ TinyImageNet
Horse2Zebra
ImageNet-1K

BibTeX

@misc{GenerativeZoo,
  author    = {Francisco Caetano},
  title     = {GenerativeZoo: A Model Zoo for Generative Models},
  journal   = {GitHub repository},
  howpublished = {\url{https://github.com/caetas/GenerativeZoo}},
  year      = {2024},
}