CenteredRBMs Julia package

Train and sample centered Restricted Boltzmann machines in Julia. See [Melchior et al] for the definition of centered. Consider an RBM with binary units. Then the centered variant has energy defined by:

$$ E(v,h) = -\sum_i a_i v_i - \sum_\mu b_\mu h_\mu - \sum_{i\mu} w_{i\mu} (v_i - c_i) (h_\mu - d_\mu) $$

with offset parameters $c_i,d_\mu$. Typically $c_i,d_\mu$ are set to approximate the average activities of $v_i$ and $h_\mu$, respectively, as this seems to help training (see [Montavon et al]).

Installation

This package is registered. Install with:

import Pkg
Pkg.add("CenteredRBMs")

This package does not export any symbols.

Related

RestrictedBoltzmannMachines, which defines RBM and layer types.

References

• Montavon, Grégoire, and Klaus-Robert Müller. "Deep Boltzmann machines and the centering trick." Neural networks: tricks of the trade. Springer, Berlin, Heidelberg, 2012. 621-637.

• Melchior, Jan, Asja Fischer, and Laurenz Wiskott. "How to center deep Boltzmann machines." The Journal of Machine Learning Research 17.1 (2016): 3387-3447.

Citation

If you use this package in a publication, please cite:

• Jorge Fernandez-de-Cossio-Diaz, Simona Cocco, and Remi Monasson. "Disentangling representations in Restricted Boltzmann Machines without adversaries." Physical Review X 13, 021003 (2023).

Or you can use the included CITATION.bib.