MATHEMATICS FOR ARTIFICIAL INTELLIGENCE

Introduction to neural networks: genesis and future perspectives.

Basics of statistical mechanics and statistical inference.

Neural networks for associative memory and pattern recognition.

Hopfield model with low-load and solution via log-constrained entropy.

Hopfield model with high-load and solution via stochastic stability.

Rosenblatt and Minsky&Papert perceptrons.

Neural networks for statistical learning and feature discovery.

Supervised Boltzmann machines.

Unsupervised Boltzmann machines.

Bayesian equivalence between Hopfield retrieval and Boltzmann learning.

Multilayered Boltzmann machines and deep learning.

Advanced topics: Numerical tools for machine learning; Non-mean-field neural networks; (Bio-)Logic gates; Maximum entropy approach, Hamilton-Jacobi techniques for mean-field models.

Reference:

[CKS] A.C.C. Coolen, R. Kuhn, P. Sollich, "Theory of Neural Information Processing Systems", Oxford Press.

Further references:

[A] D. J. Amit, "Modeling Brain Function: The World of Attractor Neural Networks", Cambridge University Press

[Ti] B. Tirozzi, "Modelli matematici di reti neurali", CEDAM

[J] E. T. Jaynes, "Probability Theory: The Logic of Science", Cambridge University Press

[Th] C. J. Thompson, "Mathematical Statistical Mechanics", Princeton University Press

Detailed program:

Introduction

Basic concepts in statistical mechanics (statistical ensemble, free energy, equilibrium states)

Maximum Entropy approach

Exact solution of the Curie-Weiss model (existance of the thermodynamic limit, evaluation of the thermodynamic limit, self-consistency equation)

Phase transition and spontaneous symmetry breakdown

Slides available here; see also [CKS] (chapt. 20), [A] (chapt. 3), [Ti] (chapt. 2), [Th].

Neurophysiological background

McCulloch-Pitts neurons

Local field alignment, parallel and sequential dynamics

Noiseless networks, synaptic symmetry and Lyapunov functions

Noisy networks, ergodicity, detailed balance and equilibrium distribution

Hopfield model (Hebbian kernel, orthogonal patterns and retrieval as pure state configurations)

Solution of the Hopfiel model at low-load via log-constrained entropy

Slides available here; see also [CKS] (chapts. 1, 3, 16, 20, 21), [A] (chapts. 2, 3, 4), [Ti] (chapts. 3, 4).

Self-averaging

Signal-to-noise for checking the stability of pure states and of symmetric mixtures of n patterns

Summary of the low-storage Hopfield model

Introduction to complexity

The Sherrington-Kirkpatrick model for spin-glasses

Replicas and overlaps

Solution of the high-storage Hopfield model via interpolation techniques at the RS level

Slides available here; see also [CKS] (chapts. 20, 21), [A] (chapts. 4, 6), [Ti] (chapts. 2, 3).

Introduction to machine learning: examples, ideas and tools

A review on statistical inference methods

Rosenblatt's perceptron

Minsky&Papert's perceptron

Binary classifier and regression

Bayesian learning

A link between Bayesian learning and "traditional learning"

Regularization, cross-validation and their meaning

Slides available here; see also [CKS] (chapts. 2, 6, 13), [J].

Restricted Boltzmann Machines

Supervised learning with RBM

Monte Carlo Markov chain sampling

Constrastive divergence algorithm

Simulated annealing

Mean-field methods

Examples of RBM

Slides available here; see also [CKS] (chapt. 14).

Integrate-and-fire model

Stein's model

Different approaches to information processing

Connection between Boltzmann Machines and Hopfield model

Bayesian equivalence between Hopfield retrieval and Boltzmann learning.

Beyond the Hebbian kernel: correlation and dilution

Getting closer to biology: topology matters

Multilayered Boltzmann machines and deep learning.

Outlooks and comments.

Slides available here; see also references therein.

Further discussions and seminars:

Francesco Allegra: "The Backpropagation Algorithm"

Elishan Braun: "Neural networks for solving differential equations"

Rania Giannopoulou: "The contrastive divergence method"

Introduction to neural networks: genesis and future perspectives.

Basics of statistical mechanics and statistical inference.

Neural networks for associative memory and pattern recognition.

Hopfield model with low-load and solution via log-constrained entropy.

Hopfield model with high-load and solution via stochastic stability.

Rosenblatt and Minsky&Papert perceptrons.

Neural networks for statistical learning and feature discovery.

Supervised Boltzmann machines.

Unsupervised Boltzmann machines.

Bayesian equivalence between Hopfield retrieval and Boltzmann learning.

Multilayered Boltzmann machines and deep learning.

Advanced topics: Numerical tools for machine learning; Non-mean-field neural networks; (Bio-)Logic gates; Maximum entropy approach, Hamilton-Jacobi techniques for mean-field models.

Reference:

[CKS] A.C.C. Coolen, R. Kuhn, P. Sollich, "Theory of Neural Information Processing Systems", Oxford Press.

Further references:

[A] D. J. Amit, "Modeling Brain Function: The World of Attractor Neural Networks", Cambridge University Press

[Ti] B. Tirozzi, "Modelli matematici di reti neurali", CEDAM

[J] E. T. Jaynes, "Probability Theory: The Logic of Science", Cambridge University Press

[Th] C. J. Thompson, "Mathematical Statistical Mechanics", Princeton University Press

Detailed program:

__15/11__- Room B - 14.30-16.30Introduction

Basic concepts in statistical mechanics (statistical ensemble, free energy, equilibrium states)

Maximum Entropy approach

Exact solution of the Curie-Weiss model (existance of the thermodynamic limit, evaluation of the thermodynamic limit, self-consistency equation)

Phase transition and spontaneous symmetry breakdown

Slides available here; see also [CKS] (chapt. 20), [A] (chapt. 3), [Ti] (chapt. 2), [Th].

__22/11__- Room B - 14.30-16.30Neurophysiological background

McCulloch-Pitts neurons

Local field alignment, parallel and sequential dynamics

Noiseless networks, synaptic symmetry and Lyapunov functions

Noisy networks, ergodicity, detailed balance and equilibrium distribution

Hopfield model (Hebbian kernel, orthogonal patterns and retrieval as pure state configurations)

Solution of the Hopfiel model at low-load via log-constrained entropy

Slides available here; see also [CKS] (chapts. 1, 3, 16, 20, 21), [A] (chapts. 2, 3, 4), [Ti] (chapts. 3, 4).

__29/11__- Room G - 14.30-16.30Self-averaging

Signal-to-noise for checking the stability of pure states and of symmetric mixtures of n patterns

Summary of the low-storage Hopfield model

Introduction to complexity

The Sherrington-Kirkpatrick model for spin-glasses

Replicas and overlaps

Solution of the high-storage Hopfield model via interpolation techniques at the RS level

Slides available here; see also [CKS] (chapts. 20, 21), [A] (chapts. 4, 6), [Ti] (chapts. 2, 3).

__06/12__- Room B - 14.30-16.30Introduction to machine learning: examples, ideas and tools

A review on statistical inference methods

Rosenblatt's perceptron

Minsky&Papert's perceptron

Binary classifier and regression

Bayesian learning

A link between Bayesian learning and "traditional learning"

Regularization, cross-validation and their meaning

Slides available here; see also [CKS] (chapts. 2, 6, 13), [J].

__13/12__- Room B - 14.30-16.30Restricted Boltzmann Machines

Supervised learning with RBM

Monte Carlo Markov chain sampling

Constrastive divergence algorithm

Simulated annealing

Mean-field methods

Examples of RBM

Slides available here; see also [CKS] (chapt. 14).

__20/12__- Room B - 14.30-16.30Integrate-and-fire model

Stein's model

Different approaches to information processing

Connection between Boltzmann Machines and Hopfield model

Bayesian equivalence between Hopfield retrieval and Boltzmann learning.

Beyond the Hebbian kernel: correlation and dilution

Getting closer to biology: topology matters

Multilayered Boltzmann machines and deep learning.

Outlooks and comments.

Slides available here; see also references therein.

Further discussions and seminars:

__10/07__- Room B - 10.00-13.00Francesco Allegra: "The Backpropagation Algorithm"

Elishan Braun: "Neural networks for solving differential equations"

Rania Giannopoulou: "The contrastive divergence method"