Chapter 18: Generative Models — GANs, VAEs, and Diffusion

Bloom's Taxonomy Map for This Chapter

Learning Objectives

By the end of this chapter, you will be able to:

1. Distinguish discriminative models P(y|x) from generative models P(x), and explain why learning to generate data is harder than learning to classify it
2. Derive the GAN minimax objective from first principles, prove the optimal discriminator D*(x) = p_data(x) / (p_data(x) + p_g(x)), and show the connection to Jensen-Shannon divergence
3. Implement a vanilla GAN, DCGAN, and WGAN from scratch in NumPy and PyTorch on MNIST
4. Diagnose GAN training pathologies — mode collapse, vanishing gradients, oscillation — and apply fixes (label smoothing, spectral normalization, progressive growing)
5. Explain the VAE ELBO decomposition: log P(x) ≥ 𝔼[log P(x|z)] − KL(q(z|x) ∥ p(z)), and implement the reparameterization trick
6. Derive the forward and reverse processes of DDPM, explain the noise schedule, and implement a simple diffusion model
7. Compare GANs, VAEs, and Diffusion models on five axes: sample quality, diversity, training stability, latent space structure, and compute requirements
8. Analyze real-world systems: Meesho's diffusion-based product photography and Midjourney/DALL-E text-to-image generation
9. Evaluate ethical implications of generative AI, including deepfakes, misinformation, and IP concerns
10. Solve GATE-style problems on GAN objectives, VAE loss components, and diffusion mathematics

Opening Hook

The Intuition First — Three Roads to Creation

The Art Forgery Analogy (GANs) 🎨

Imagine a forger (Generator) trying to create fake Picasso paintings, and an art critic (Discriminator) trying to spot the fakes. Initially, the forger is terrible — scribbling stick figures — and the critic catches every fake easily. But as the forger gets feedback ("your brushstrokes are too uniform, your color palette is wrong"), they improve. Meanwhile, the critic must also improve, because the fakes are getting better.

This arms race continues until the forger creates paintings so perfect that even expert critics flip a coin: 50% chance any painting is real or fake. At that point, the Generator has learned the true distribution of Picasso paintings.

"Aha" question: What if the forger only learns to copy one perfect painting and shows it every time? The critic can't tell it from a real Picasso, but you've lost all diversity. This is mode collapse, and it's the central challenge of GAN training.

The Postal Code Analogy (VAEs) 📮

Think of a VAE like a postal system for images. The encoder takes an image and compresses it into a "postal code" — a small vector in latent space. The decoder takes any postal code and reconstructs the image. The key insight: VAEs don't just learn one postal code per image — they learn a region (a Gaussian cloud). Nearby postal codes decode to similar images. You can generate new images by sampling random postal codes and decoding them.

The Thermodynamics Analogy (Diffusion) 🌡️

Drop a single drop of ink into a glass of water. Over time, the ink molecules spread out until they're uniformly distributed — this is the forward process (adding noise). Now imagine you could reverse time and watch the uniform ink reconcentrate into a single drop. Diffusion models learn exactly this: they learn to reverse the noise process, turning pure static back into a coherent image, step by step.

18.1 — Discriminative vs. Generative Models

Before diving into specific architectures, you need to understand a fundamental split in machine learning.

Mathematical Formulation

By Bayes' theorem: P(y|x) = P(x|y)P(y) / P(x). A generative model learns P(x|y) and P(y), which also lets you compute P(y|x). So generative models are strictly more powerful — but that power comes at a cost.

18.2 — GANs: The Minimax Game

The GAN Framework

A GAN consists of two neural networks trained simultaneously:

• Generator G(z; θ_g): Takes random noise z ~ P_z(z) (usually N(0, I)) and maps it to a fake sample G(z)
• Discriminator D(x; θ_d): Takes a sample x (real or fake) and outputs D(x) ∈ [0, 1] — the probability that x is real

The Minimax Objective — Derivation from First Principles

This is beautiful! The optimal discriminator is simply the ratio of real data density to total density. When p_g = p_data (Generator perfectly matches data), D*(x) = 1/2 everywhere — the discriminator is reduced to random guessing.

Connection to Jensen-Shannon Divergence

Practical Training: Alternating Gradient Descent

In practice, you don't solve the minimax analytically. Instead, you alternate:

1. Train D for k steps: Sample minibatch of real data x, sample noise z, compute loss = −[log D(x) + log(1−D(G(z)))], update θ_d via gradient ascent
2. Train G for 1 step: Sample noise z, compute loss = log(1−D(G(z))), update θ_g via gradient descent

18.3 — GAN Variants: DCGAN, WGAN, and StyleGAN

DCGAN — Deep Convolutional GAN (Radford et al., 2015)

The original GAN used fully connected layers. DCGAN established the architectural guidelines that made convolutional GANs work:

WGAN — Wasserstein GAN (Arjovsky et al., 2017)

The key insight of WGAN: Jensen-Shannon Divergence is a terrible training signal when the generator and data distributions don't overlap (which is almost always true early in training, since images live on a low-dimensional manifold in pixel space).

Why JSD Fails

When p_data and p_g have disjoint supports (don't overlap), JSD = log(2) regardless of how "close" the distributions are. The discriminator achieves perfect accuracy instantly, and gradients for G vanish. This is the vanishing gradient problem in GANs.

The Wasserstein Distance (Earth Mover's Distance)

The beauty of W: it's continuous and differentiable even when distributions don't overlap. Think of it as: "how much earth do you need to move, and how far?" — even when two piles of dirt don't touch, you can always measure the distance.

WGAN Training Changes

StyleGAN — Style-Based Generator Architecture (Karras et al., 2019)

StyleGAN revolutionized GANs by separating what is generated from how it's styled:

• Mapping Network: z → w (8-layer MLP, maps noise to "style" space W)
• Synthesis Network: Generates image progressively (4×4 → 8×8 → ... → 1024×1024)
• AdaIN (Adaptive Instance Normalization): Style vector w controls normalization at each layer
• Noise injection: Stochastic details (hair strands, pores) via per-pixel noise

18.4 — Mode Collapse and GAN Training Tricks

What is Mode Collapse?

Imagine p_data is a mixture of 10 Gaussians (like MNIST's 10 digit classes). Mode collapse occurs when the Generator learns to produce only 1-2 of these modes, ignoring the rest. The Discriminator catches on — "you're only generating 7s!" — but the Generator responds by switching to another mode: "fine, now I'll only generate 3s."

Causes and Fixes

Practical GAN Training Checklist

for epoch in range(100):
    # Train D
    real = next(dataloader)
    fake = G(torch.randn(64, 100))
    d_loss = -torch.mean(torch.log(D(real)) + torch.log(1 - D(fake)))
    d_optimizer.step()
    
    # Train G (same batch!)
    g_loss = torch.mean(torch.log(1 - D(G(torch.randn(64, 100)))))
    g_optimizer.step()

18.5 — Variational Autoencoders (VAEs) and β-VAE

From Autoencoders to VAEs

You already know autoencoders (Ch 12): encoder compresses x → z, decoder reconstructs z → x̂. But regular autoencoders learn a deterministic mapping to a messy, disconnected latent space. You can't sample from it meaningfully.

VAEs fix this by making the encoding probabilistic: instead of z = f(x), the encoder outputs parameters of a distribution: μ(x), σ(x). Then z is sampled from N(μ, σ²). A KL divergence term pushes this distribution toward the standard normal N(0, 1), ensuring the latent space is smooth and continuous.

Deriving the ELBO from First Principles

The Reparameterization Trick

Problem: z ~ N(μ, σ²) is a stochastic node. You can't backpropagate through random sampling!

Solution: Rewrite z = μ + ε · σ, where ε ~ N(0, 1). Now the randomness (ε) is external to the computation graph, and gradients flow through μ and σ normally.

KL Divergence — Closed Form

When q(z|x) = N(μ, σ²) and P(z) = N(0, 1), the KL divergence has a beautiful closed form:

β-VAE: Controlling Disentanglement

Higgins et al. (2017) introduced β-VAE by simply scaling the KL term:

ℒ_β-VAE = Reconstruction Loss + β · KL(q(z|x) ∥ P(z))

• β = 1: Standard VAE
• β > 1: Stronger regularization → more disentangled latent factors (each dimension captures one independent feature: rotation, scale, color), but blurrier reconstructions
• β < 1: Better reconstruction, but messier latent space

18.6 — Diffusion Models: DDPM and the Denoising Revolution

The Core Idea

Diffusion models draw inspiration from non-equilibrium thermodynamics. The idea is stunningly simple:

1. Forward process (fixed, no learning): Gradually add Gaussian noise to an image over T steps until it becomes pure noise
2. Reverse process (learned): Train a neural network to reverse each step — to denoise slightly at each step — until you recover a clean image from pure noise

Forward Process (Adding Noise)

Reverse Process (Learning to Denoise)

The reverse process aims to undo each noise step:

p_θ(x_{t-1} | x_t) = N(x_{t-1}; μ_θ(x_t, t), σ_t² · I)

We train a neural network ε_θ(x_t, t) to predict the noise ε that was added. Once we know the noise, we can compute the denoised image.

DDPM Training Objective

Sampling (Generating Images)

To generate an image from scratch:

1. Start with pure noise: x_T ~ N(0, I)
2. For t = T, T−1, ..., 1: use the trained ε_θ to predict the noise, subtract it, get x_{t-1}
3. The final x₀ is your generated image!

18.7 — Stable Diffusion and Text-to-Image

Latent Diffusion Models (LDM)

Running diffusion directly on 512×512×3 images is computationally expensive. Latent Diffusion (Rombach et al., 2022) solves this by running the diffusion process in a compressed latent space:

Three Components of Stable Diffusion

1. VAE (Autoencoder): Compresses images from pixel space (512×512×3) to latent space (64×64×4) and back. Trained separately.
2. U-Net Denoiser: The diffusion model itself — predicts noise in latent space. Conditioned on timestep t and text embedding via cross-attention layers.
3. Text Encoder (CLIP): Converts text prompts to embeddings that guide the U-Net's denoising process.

Classifier-Free Guidance

To make outputs follow text prompts more closely, Stable Diffusion uses classifier-free guidance:

ε̃ = ε_θ(x_t, ∅) + w · (ε_θ(x_t, c) − ε_θ(x_t, ∅))

where c is the text condition, ∅ is the unconditional embedding, and w is the guidance scale (typically 7.5). Higher w = more adherent to prompt but less diverse.

18.8 — GANs vs. VAEs vs. Diffusion: The Complete Comparison

Worked Examples

Worked Example 1: Computing Optimal Discriminator (By Hand) ✍️

Worked Example 2: VAE KL Divergence (Indian Industry Context) 🇮🇳

Worked Example 3: DDPM Noise Scheduling (US/Global Context) 🇺🇸

From-Scratch Code: NumPy Implementations

1. Vanilla GAN from Scratch (NumPy)

Python / NumPyimport numpy as np

# ═══ Utility Functions ═══
def sigmoid(x):
    return 1 / (1 + np.exp(-np.clip(x, -500, 500)))

def sigmoid_deriv(x):
    s = sigmoid(x)
    return s * (1 - s)

def relu(x):
    return np.maximum(0, x)

def relu_deriv(x):
    return (x > 0).astype(np.float64)

def leaky_relu(x, alpha=0.2):
    return np.where(x > 0, x, alpha * x)

def leaky_relu_deriv(x, alpha=0.2):
    return np.where(x > 0, 1, alpha)

# ═══ Generator: z(100) → hidden(256) → output(784) ═══
np.random.seed(42)
z_dim = 100
h_dim = 256
img_dim = 784  # 28×28 for MNIST
lr = 0.0002

# Generator weights (Xavier init)
W_g1 = np.random.randn(z_dim, h_dim) * np.sqrt(2/z_dim)
b_g1 = np.zeros((1, h_dim))
W_g2 = np.random.randn(h_dim, img_dim) * np.sqrt(2/h_dim)
b_g2 = np.zeros((1, img_dim))

# Discriminator weights
W_d1 = np.random.randn(img_dim, h_dim) * np.sqrt(2/img_dim)
b_d1 = np.zeros((1, h_dim))
W_d2 = np.random.randn(h_dim, 1) * np.sqrt(2/h_dim)
b_d2 = np.zeros((1, 1))

def generator_forward(z):
    """z → ReLU(zW1+b1) → Sigmoid(hW2+b2) → fake image"""
    h = z @ W_g1 + b_g1         # (batch, 256)
    h_act = relu(h)              # ReLU activation
    out = h_act @ W_g2 + b_g2   # (batch, 784)
    img = sigmoid(out)           # Sigmoid → [0,1] pixel values
    return z, h, h_act, out, img

def discriminator_forward(x):
    """x → LeakyReLU(xW1+b1) → Sigmoid(hW2+b2) → probability"""
    h = x @ W_d1 + b_d1         # (batch, 256)
    h_act = leaky_relu(h)       # LeakyReLU
    out = h_act @ W_d2 + b_d2   # (batch, 1)
    prob = sigmoid(out)          # probability real/fake
    return x, h, h_act, out, prob

def train_step(real_batch, batch_size=64):
    global W_g1, b_g1, W_g2, b_g2, W_d1, b_d1, W_d2, b_d2
    
    # ── Step 1: Train Discriminator ──
    z = np.random.randn(batch_size, z_dim)
    _, g_h, g_h_act, g_out, fake = generator_forward(z)
    
    # D on real data (want D(x) → 1)
    _, d_h_r, d_ha_r, d_out_r, d_prob_r = discriminator_forward(real_batch)
    # D on fake data (want D(G(z)) → 0)
    _, d_h_f, d_ha_f, d_out_f, d_prob_f = discriminator_forward(fake)
    
    # Binary cross-entropy gradients for D
    # Loss_D = -[log(D(real)) + log(1-D(fake))]
    d_loss = -np.mean(np.log(d_prob_r + 1e-8) + np.log(1 - d_prob_f + 1e-8))
    
    # Backprop through D (real path)
    dL_dout_r = -(1 / (d_prob_r + 1e-8)) * sigmoid_deriv(d_out_r) / batch_size
    dW_d2_r = d_ha_r.T @ dL_dout_r
    db_d2_r = np.sum(dL_dout_r, axis=0, keepdims=True)
    dL_dha_r = dL_dout_r @ W_d2.T
    dL_dh_r = dL_dha_r * leaky_relu_deriv(d_h_r)
    dW_d1_r = real_batch.T @ dL_dh_r
    db_d1_r = np.sum(dL_dh_r, axis=0, keepdims=True)
    
    # Backprop through D (fake path)
    dL_dout_f = (1 / (1 - d_prob_f + 1e-8)) * sigmoid_deriv(d_out_f) / batch_size
    dW_d2_f = d_ha_f.T @ dL_dout_f
    db_d2_f = np.sum(dL_dout_f, axis=0, keepdims=True)
    dL_dha_f = dL_dout_f @ W_d2.T
    dL_dh_f = dL_dha_f * leaky_relu_deriv(d_h_f)
    dW_d1_f = fake.T @ dL_dh_f
    db_d1_f = np.sum(dL_dh_f, axis=0, keepdims=True)
    
    # Update D
    W_d2 -= lr * (dW_d2_r + dW_d2_f)
    b_d2 -= lr * (db_d2_r + db_d2_f)
    W_d1 -= lr * (dW_d1_r + dW_d1_f)
    b_d1 -= lr * (db_d1_r + db_d1_f)
    
    # ── Step 2: Train Generator ──
    # Non-saturating loss: G maximizes log(D(G(z)))
    z = np.random.randn(batch_size, z_dim)
    _, g_h, g_h_act, g_out, fake = generator_forward(z)
    _, d_h_f, d_ha_f, d_out_f, d_prob_f = discriminator_forward(fake)
    
    g_loss = -np.mean(np.log(d_prob_f + 1e-8))
    
    # Backprop through D (frozen) then through G
    dL_dout = -(1 / (d_prob_f + 1e-8)) * sigmoid_deriv(d_out_f) / batch_size
    dL_dha = dL_dout @ W_d2.T
    dL_dh = dL_dha * leaky_relu_deriv(d_h_f)
    dL_dfake = dL_dh @ W_d1.T  # gradient at fake image
    
    # Continue through G
    dL_gout = dL_dfake * sigmoid_deriv(g_out)
    dW_g2 = g_h_act.T @ dL_gout
    db_g2 = np.sum(dL_gout, axis=0, keepdims=True)
    dL_ghact = dL_gout @ W_g2.T
    dL_gh = dL_ghact * relu_deriv(g_h)
    dW_g1 = z.T @ dL_gh
    db_g1 = np.sum(dL_gh, axis=0, keepdims=True)
    
    # Update G
    W_g2 -= lr * dW_g2
    b_g2 -= lr * db_g2
    W_g1 -= lr * dW_g1
    b_g1 -= lr * db_g1
    
    return d_loss, g_loss

# Training loop (with simulated MNIST data)
print("Training Vanilla GAN from scratch...")
for epoch in range(100):
    # Simulate a batch of "real" data (replace with actual MNIST)
    real = np.random.rand(64, img_dim) * 0.5 + 0.25
    d_loss, g_loss = train_step(real)
    if epoch % 20 == 0:
        print(f"Epoch {epoch}: D_loss={d_loss:.4f}, G_loss={g_loss:.4f}")

2. Simple Diffusion Model from Scratch (NumPy)

Python / NumPyimport numpy as np

# ═══ DDPM from Scratch — Simplified for 1D data ═══
# We'll implement the core math, then show PyTorch version for images

T = 100  # number of diffusion steps (1000 in practice)
beta_start = 1e-4
beta_end = 0.02

# Linear noise schedule
betas = np.linspace(beta_start, beta_end, T)
alphas = 1.0 - betas
alpha_bars = np.cumprod(alphas)  # ᾱ_t = cumulative product

print("Noise schedule check:")
print(f"  ᾱ_1   = {alpha_bars[0]:.6f}  (almost clean)")
print(f"  ᾱ_50  = {alpha_bars[49]:.6f}  (partially noisy)")
print(f"  ᾱ_100 = {alpha_bars[99]:.6f}  (mostly noise)")

def forward_diffusion(x0, t, noise=None):
    """Add noise to x0 at timestep t: x_t = √ᾱ_t·x₀ + √(1−ᾱ_t)·ε"""
    if noise is None:
        noise = np.random.randn(*x0.shape)
    sqrt_ab = np.sqrt(alpha_bars[t])
    sqrt_1_ab = np.sqrt(1 - alpha_bars[t])
    return sqrt_ab * x0 + sqrt_1_ab * noise, noise

# Simple denoiser: 2-layer MLP that predicts noise
# Input: [x_t, t_embedding], Output: predicted noise
h_dim = 64
input_dim = 2  # 1D data + timestep encoding

W1 = np.random.randn(input_dim, h_dim) * 0.1
b1 = np.zeros(h_dim)
W2 = np.random.randn(h_dim, 1) * 0.1
b2 = np.zeros(1)

def predict_noise(x_t, t):
    """Simple MLP to predict noise ε from (x_t, t)"""
    t_norm = t / T  # normalize timestep to [0, 1]
    inp = np.column_stack([x_t.reshape(-1, 1), 
                           np.full((len(x_t), 1), t_norm)])
    h = np.tanh(inp @ W1 + b1)  # hidden layer
    return h @ W2 + b2             # predicted noise

def train_diffusion(data, epochs=1000, lr=0.001):
    """Train denoiser to predict noise at random timesteps"""
    global W1, b1, W2, b2
    for epoch in range(epochs):
        # 1. Sample random data point
        x0 = data[np.random.randint(len(data))]
        # 2. Sample random timestep
        t = np.random.randint(0, T)
        # 3. Add noise (forward process)
        x_t, true_noise = forward_diffusion(np.array([x0]), t)
        # 4. Predict noise
        pred_noise = predict_noise(x_t, t)
        # 5. Loss = MSE(true_noise, pred_noise)
        loss = np.mean((true_noise - pred_noise.flatten()) ** 2)
        
        # Manual backprop (simplified for 1D)
        # ... gradient computation omitted for brevity ...
        
        if epoch % 200 == 0:
            print(f"Epoch {epoch}: loss = {loss:.4f}")

# Generate 1D data: mixture of two Gaussians
data = np.concatenate([
    np.random.randn(500) * 0.5 + 3.0,  # mode 1
    np.random.randn(500) * 0.5 - 3.0,  # mode 2
])
print("Data shape:", data.shape)
print("Training simplified diffusion model...")
train_diffusion(data, epochs=500)

PyTorch Implementations

1. DCGAN on MNIST (PyTorch)

Python / PyTorchimport torch
import torch.nn as nn
import torch.optim as optim
from torchvision import datasets, transforms
from torch.utils.data import DataLoader

# ═══ Hyperparameters ═══
z_dim = 100
img_channels = 1
features_g = 64
features_d = 64
lr = 0.0002
batch_size = 128
epochs = 50

# ═══ Generator ═══
class Generator(nn.Module):
    def __init__(self):
        super().__init__()
        self.net = nn.Sequential(
            # z → 7×7×256
            nn.ConvTranspose2d(z_dim, features_g*4, 7, 1, 0),
            nn.BatchNorm2d(features_g*4),
            nn.ReLU(True),
            # 7×7 → 14×14
            nn.ConvTranspose2d(features_g*4, features_g*2, 4, 2, 1),
            nn.BatchNorm2d(features_g*2),
            nn.ReLU(True),
            # 14×14 → 28×28
            nn.ConvTranspose2d(features_g*2, img_channels, 4, 2, 1),
            nn.Tanh(),  # output in [-1, 1]
        )
    
    def forward(self, z):
        return self.net(z.view(-1, z_dim, 1, 1))

# ═══ Discriminator ═══
class Discriminator(nn.Module):
    def __init__(self):
        super().__init__()
        self.net = nn.Sequential(
            # 28×28 → 14×14
            nn.Conv2d(img_channels, features_d, 4, 2, 1),
            nn.LeakyReLU(0.2, inplace=True),
            # 14×14 → 7×7
            nn.Conv2d(features_d, features_d*2, 4, 2, 1),
            nn.BatchNorm2d(features_d*2),
            nn.LeakyReLU(0.2, inplace=True),
            # 7×7 → 1×1
            nn.Conv2d(features_d*2, 1, 7, 1, 0),
            nn.Sigmoid(),
        )
    
    def forward(self, x):
        return self.net(x).view(-1, 1)

# ═══ Training Loop ═══
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
G = Generator().to(device)
D = Discriminator().to(device)
criterion = nn.BCELoss()
opt_g = optim.Adam(G.parameters(), lr=lr, betas=(0.5, 0.999))
opt_d = optim.Adam(D.parameters(), lr=lr, betas=(0.5, 0.999))

transform = transforms.Compose([
    transforms.ToTensor(),
    transforms.Normalize((0.5,), (0.5,)),  # → [-1, 1]
])
dataset = datasets.MNIST(root="./data", train=True, transform=transform, download=True)
loader = DataLoader(dataset, batch_size=batch_size, shuffle=True)

for epoch in range(epochs):
    for real, _ in loader:
        real = real.to(device)
        bs = real.size(0)
        
        # ── Train Discriminator ──
        z = torch.randn(bs, z_dim).to(device)
        fake = G(z).detach()
        d_real = D(real)
        d_fake = D(fake)
        loss_d = criterion(d_real, torch.ones_like(d_real) * 0.9) + \
                 criterion(d_fake, torch.zeros_like(d_fake))
        opt_d.zero_grad()
        loss_d.backward()
        opt_d.step()
        
        # ── Train Generator (non-saturating) ──
        z = torch.randn(bs, z_dim).to(device)
        fake = G(z)
        d_fake = D(fake)
        loss_g = criterion(d_fake, torch.ones_like(d_fake))  # fool D
        opt_g.zero_grad()
        loss_g.backward()
        opt_g.step()
    
    print(f"Epoch [{epoch+1}/{epochs}] D_loss: {loss_d:.4f} G_loss: {loss_g:.4f}")

2. Simple DDPM on MNIST (PyTorch)

Python / PyTorchimport torch
import torch.nn as nn
import torch.nn.functional as F

# ═══ Noise Schedule ═══
T = 1000
betas = torch.linspace(1e-4, 0.02, T)
alphas = 1.0 - betas
alpha_bars = torch.cumprod(alphas, dim=0)

def forward_diffusion(x0, t, device):
    """q(x_t | x_0) — add noise at timestep t"""
    noise = torch.randn_like(x0)
    sqrt_ab = alpha_bars[t].sqrt().view(-1, 1, 1, 1).to(device)
    sqrt_1_ab = (1 - alpha_bars[t]).sqrt().view(-1, 1, 1, 1).to(device)
    return sqrt_ab * x0 + sqrt_1_ab * noise, noise

# ═══ U-Net (simplified) ═══
class SimpleUNet(nn.Module):
    """Minimal U-Net for noise prediction on 28×28 images."""
    def __init__(self):
        super().__init__()
        # Time embedding
        self.time_mlp = nn.Sequential(
            nn.Linear(1, 64), nn.ReLU(), nn.Linear(64, 64)
        )
        # Encoder
        self.enc1 = nn.Sequential(nn.Conv2d(1, 32, 3, padding=1), nn.ReLU(),
                                  nn.Conv2d(32, 32, 3, padding=1), nn.ReLU())
        self.enc2 = nn.Sequential(nn.Conv2d(32, 64, 3, stride=2, padding=1), nn.ReLU(),
                                  nn.Conv2d(64, 64, 3, padding=1), nn.ReLU())
        # Bottleneck
        self.bottleneck = nn.Sequential(nn.Conv2d(64, 128, 3, stride=2, padding=1), nn.ReLU())
        # Decoder
        self.dec2 = nn.Sequential(nn.ConvTranspose2d(128, 64, 4, stride=2, padding=1), nn.ReLU())
        self.dec1 = nn.Sequential(nn.ConvTranspose2d(128, 32, 4, stride=2, padding=1), nn.ReLU())
        self.final = nn.Conv2d(64, 1, 1)  # predict noise
    
    def forward(self, x, t):
        # Time conditioning
        t_emb = self.time_mlp(t.float().unsqueeze(1) / T)  # (B, 64)
        
        # Encoder
        e1 = self.enc1(x)                     # (B, 32, 28, 28)
        e2 = self.enc2(e1)                    # (B, 64, 14, 14)
        
        # Bottleneck + time embedding
        b = self.bottleneck(e2)               # (B, 128, 7, 7)
        b = b + t_emb.view(-1, 64, 1, 1).expand_as(b[:, :64]).repeat(1, 2, 1, 1)
        
        # Decoder with skip connections
        d2 = self.dec2(b)                     # (B, 64, 14, 14)
        d2 = torch.cat([d2, e2], dim=1)      # skip: (B, 128, 14, 14)
        d1 = self.dec1(d2)                    # (B, 32, 28, 28)
        d1 = torch.cat([d1, e1], dim=1)      # skip: (B, 64, 28, 28)
        return self.final(d1)                 # (B, 1, 28, 28)

# ═══ Training ═══
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model = SimpleUNet().to(device)
optimizer = torch.optim.Adam(model.parameters(), lr=2e-4)

# Training loop (using same MNIST loader from above)
for epoch in range(20):
    total_loss = 0
    for images, _ in loader:
        images = images.to(device)
        t = torch.randint(0, T, (images.size(0),)).to(device)
        
        x_t, noise = forward_diffusion(images, t, device)
        pred_noise = model(x_t, t)
        
        loss = F.mse_loss(pred_noise, noise)
        
        optimizer.zero_grad()
        loss.backward()
        optimizer.step()
        total_loss += loss.item()
    
    print(f"Epoch {epoch+1}: Loss = {total_loss/len(loader):.4f}")

# ═══ Sampling (Generate from noise) ═══
@torch.no_grad()
def sample(model, n_samples=16):
    """Generate images via reverse diffusion"""
    model.eval()
    x = torch.randn(n_samples, 1, 28, 28).to(device)
    
    for t in reversed(range(T)):
        t_batch = torch.full((n_samples,), t, device=device)
        pred_noise = model(x, t_batch)
        
        alpha = alphas[t]
        alpha_bar = alpha_bars[t]
        beta = betas[t]
        
        # Reverse step: x_{t-1} from x_t
        x = (1/alpha.sqrt()) * (x - (beta / (1-alpha_bar).sqrt()) * pred_noise)
        
        if t > 0:
            noise = torch.randn_like(x)
            x += beta.sqrt() * noise  # add stochasticity
    
    return x.clamp(-1, 1)

samples = sample(model)
print(f"Generated {samples.shape[0]} images of shape {samples.shape[1:]}")

Visual Diagrams

Diagram 1: The Three Generative Paradigms

Diagram 2: VAE Latent Space

Diagram 3: Diffusion Forward/Reverse Process

Industry Case Study: Meesho AI Product Photography 🇮🇳

Industry Case Study: Midjourney & DALL-E 🇺🇸

Common Misconceptions

GATE / Exam Corner

GATE-Style MCQs

GATE Prediction Table (2025-2027)

Interview Prep

Conceptual Questions

Coding Challenge

def vae_kl_loss(mu, log_var):
    """
    Compute KL divergence KL(N(μ, σ²) ∥ N(0, I))
    Args:
        mu: (batch, latent_dim) — encoder mean
        log_var: (batch, latent_dim) — log variance
    Returns:
        KL divergence (scalar, averaged over batch)
    """
    # KL = -0.5 * Σ(1 + log(σ²) - μ² - σ²)
    kl = -0.5 * torch.sum(1 + log_var - mu.pow(2) - log_var.exp(), dim=1)
    return kl.mean()

# Test
mu = torch.tensor([[0.5, -1.0], [0.0, 0.0]])
log_var = torch.tensor([[0.0, 0.5], [0.0, 0.0]])
print(f"KL = {vae_kl_loss(mu, log_var):.4f}")
# Row 2 (μ=0, σ²=1) should contribute 0 KL

Case Study Interview (India Focus)

Hands-On Lab: Build a Conditional DCGAN

Exercises (22 Questions)

Section A: Conceptual (5 Questions)

Section B: Mathematical (8 Questions)

Section C: Coding (4 Questions)

Section D: Critical Thinking (3 Questions)

★ Starred Research Questions (2 Questions)

Deepfakes and the Ethics of Generative AI

Connections

Chapter Summary

Further Reading

🇮🇳 Indian Resources

• NPTEL: "Deep Learning" by Prof. Mitesh Khapra (IIT Madras) — Lectures 35-38 on generative models
• NPTEL: "Advanced Deep Learning" by Prof. Prabir Kumar Biswas (IIT KGP) — VAE and GAN modules
• GATE CS syllabus: Generative models under "Machine Learning" (added in GATE 2024 pattern)
• AI4Bharat Wiki: Indian-language generative AI resources and datasets

🌍 Global Resources

• Papers:
  • Goodfellow et al., "Generative Adversarial Nets" (NeurIPS 2014) — arxiv.org/abs/1406.2661
  • Kingma & Welling, "Auto-Encoding Variational Bayes" (ICLR 2014) — arxiv.org/abs/1312.6114
  • Ho et al., "Denoising Diffusion Probabilistic Models" (NeurIPS 2020) — arxiv.org/abs/2006.11239
  • Rombach et al., "High-Resolution Image Synthesis with Latent Diffusion Models" (CVPR 2022) — arxiv.org/abs/2112.10752
  • Arjovsky et al., "Wasserstein GAN" (ICML 2017) — arxiv.org/abs/1701.07875
  • Karras et al., "A Style-Based Generator Architecture for GANs" (CVPR 2019) — StyleGAN
• Visual Explainers:
  • Lil'Log: "What are Diffusion Models?" — lilianweng.github.io
  • 3Blue1Brown: "But what is a neural network?" (foundation for all chapters)
  • Jay Alammar: "The Illustrated Stable Diffusion" — jalammar.github.io
• Books:
  • Goodfellow et al., "Deep Learning" (MIT Press) — Chapter 20: Deep Generative Models
  • Prince, "Understanding Deep Learning" (MIT Press, 2023) — Chapters 14-18
  • Foster, "Generative Deep Learning" (O'Reilly, 2nd ed.) — Hands-on with TensorFlow/Keras
• Code:
  • HuggingFace Diffusers: github.com/huggingface/diffusers
  • PyTorch GAN Zoo: github.com/facebookresearch/pytorch_GAN_zoo