I am very happy to finally share something I have been working on and off for the past year:

"The Information Dynamics of Generative Diffusion"

This paper connects entropy production, divergence of vector fields and spontaneous symmetry breaking

link: arxiv.org/abs/2508.19897

Generative decisions in diffusion models can be detected locally as symmetry breaking in the energy and globally as peaks in the conditional entropy rate.

The both corresponds to a (local or global) suppression of the quadratic potential (Hessian trace).

In continuous generative diffusion, the conditional entropy rate is the constant term that separates the score matching and the denoising score matching loss

This can be directly interpreted as the information transfer (bit rate) from the state x_t and the final generation x_0.

Decisions during generative diffusion are analogous to phase transitions in physics. They can be identified as peaks in the conditional entropy rate curve!

I am very happy to share our latest work on the information theory of generative diffusion:

"Entropic Time Schedulers for Generative Diffusion Models"

We find that the conditional entropy offers a natural data-dependent notion of time during generation

Link: arxiv.org/abs/2504.13612

Flow Matching in a nutshell.

Generative decisions in diffusion models are made at special critical time points.

Missing these points with a fast sampler results in loss of diversity. It's like missing an exit in the highway!

Paper: openreview.net/forum?id=lxG...

I know that you guys do not care much about GPs, but I am very proud of this solo paper where I defined smooth non-mean reverting kernels!

openreview.net/forum?id=6CV...

I am happy to share here our paper: "Spontaneous symmetry breaking in generative diffusion models", published at Neurips 2023.

We found that the generative capabilities of diffusion models are the result of a phase transition!

Preprint: arxiv.org/abs/2305.19693

Code: github.com/gabrielraya/...

Nuggets of generative diffusion

Modern Hopfield networks are related to transformers, but did you know that they are mathematically equivalent to generative diffusion models?

Happy to share: "In search of dispersed memories: Generative diffusion models are associative memory networks":

arxiv.org/abs/2309.17290