MATHEMATICS FOR ARTIFICIAL INTELLIGENCE
Introduction to neural networks: genesis and future perspectives.
Basics of statistical mechanics and statistical inference.
Neural networks for associative memory and pattern recognition.
Hopfield model with low-load and solution via log-constrained entropy.
Hopfield model with high-load and solution via stochastic stability.
Rosenblatt and Minsky&Papert perceptrons.
Neural networks for statistical learning and feature discovery.
Supervised Boltzmann machines.
Unsupervised Boltzmann machines.
Bayesian equivalence between Hopfield retrieval and Boltzmann learning.
Multilayered Boltzmann machines and deep learning.
Advanced topics: Numerical tools for machine learning; Non-mean-field neural networks; (Bio-)Logic gates; Maximum entropy approach, Hamilton-Jacobi techniques for mean-field models.
Reference:
[CKS] A.C.C. Coolen, R. Kuhn, P. Sollich, "Theory of Neural Information Processing Systems", Oxford Press.
Further references:
[A] D. J. Amit, "Modeling Brain Function: The World of Attractor Neural Networks", Cambridge University Press
[Ti] B. Tirozzi, "Modelli matematici di reti neurali", CEDAM
[J] E. T. Jaynes, "Probability Theory: The Logic of Science", Cambridge University Press
[Th] C. J. Thompson, "Mathematical Statistical Mechanics", Princeton University Press
Detailed program:
15/11 - Room B - 14.30-16.30
Introduction
Basic concepts in statistical mechanics (statistical ensemble, free energy, equilibrium states)
Maximum Entropy approach
Exact solution of the Curie-Weiss model (existance of the thermodynamic limit, evaluation of the thermodynamic limit, self-consistency equation)
Phase transition and spontaneous symmetry breakdown
Slides available here; see also [CKS] (chapt. 20), [A] (chapt. 3), [Ti] (chapt. 2), [Th].
22/11 - Room B - 14.30-16.30
Neurophysiological background
McCulloch-Pitts neurons
Local field alignment, parallel and sequential dynamics
Noiseless networks, synaptic symmetry and Lyapunov functions
Noisy networks, ergodicity, detailed balance and equilibrium distribution
Hopfield model (Hebbian kernel, orthogonal patterns and retrieval as pure state configurations)
Solution of the Hopfiel model at low-load via log-constrained entropy
Slides available here; see also [CKS] (chapts. 1, 3, 16, 20, 21), [A] (chapts. 2, 3, 4), [Ti] (chapts. 3, 4).
29/11 - Room G - 14.30-16.30
Self-averaging
Signal-to-noise for checking the stability of pure states and of symmetric mixtures of n patterns
Summary of the low-storage Hopfield model
Introduction to complexity
The Sherrington-Kirkpatrick model for spin-glasses
Replicas and overlaps
Solution of the high-storage Hopfield model via interpolation techniques at the RS level
Slides available here; see also [CKS] (chapts. 20, 21), [A] (chapts. 4, 6), [Ti] (chapts. 2, 3).
06/12 - Room B - 14.30-16.30
Introduction to machine learning: examples, ideas and tools
A review on statistical inference methods
Rosenblatt's perceptron
Minsky&Papert's perceptron
Binary classifier and regression
Bayesian learning
A link between Bayesian learning and "traditional learning"
Regularization, cross-validation and their meaning
Slides available here; see also [CKS] (chapts. 2, 6, 13), [J].
13/12 - Room B - 14.30-16.30
Restricted Boltzmann Machines
Supervised learning with RBM
Monte Carlo Markov chain sampling
Constrastive divergence algorithm
Simulated annealing
Mean-field methods
Examples of RBM
Slides available here; see also [CKS] (chapt. 14).
20/12 - Room B - 14.30-16.30
Integrate-and-fire model
Stein's model
Different approaches to information processing
Connection between Boltzmann Machines and Hopfield model
Bayesian equivalence between Hopfield retrieval and Boltzmann learning.
Beyond the Hebbian kernel: correlation and dilution
Getting closer to biology: topology matters
Multilayered Boltzmann machines and deep learning.
Outlooks and comments.
Slides available here; see also references therein.
Further discussions and seminars:
10/07 - Room B - 10.00-13.00
Francesco Allegra: "The Backpropagation Algorithm"
Elishan Braun: "Neural networks for solving differential equations"
Rania Giannopoulou: "The contrastive divergence method"
Introduction to neural networks: genesis and future perspectives.
Basics of statistical mechanics and statistical inference.
Neural networks for associative memory and pattern recognition.
Hopfield model with low-load and solution via log-constrained entropy.
Hopfield model with high-load and solution via stochastic stability.
Rosenblatt and Minsky&Papert perceptrons.
Neural networks for statistical learning and feature discovery.
Supervised Boltzmann machines.
Unsupervised Boltzmann machines.
Bayesian equivalence between Hopfield retrieval and Boltzmann learning.
Multilayered Boltzmann machines and deep learning.
Advanced topics: Numerical tools for machine learning; Non-mean-field neural networks; (Bio-)Logic gates; Maximum entropy approach, Hamilton-Jacobi techniques for mean-field models.
Reference:
[CKS] A.C.C. Coolen, R. Kuhn, P. Sollich, "Theory of Neural Information Processing Systems", Oxford Press.
Further references:
[A] D. J. Amit, "Modeling Brain Function: The World of Attractor Neural Networks", Cambridge University Press
[Ti] B. Tirozzi, "Modelli matematici di reti neurali", CEDAM
[J] E. T. Jaynes, "Probability Theory: The Logic of Science", Cambridge University Press
[Th] C. J. Thompson, "Mathematical Statistical Mechanics", Princeton University Press
Detailed program:
15/11 - Room B - 14.30-16.30
Introduction
Basic concepts in statistical mechanics (statistical ensemble, free energy, equilibrium states)
Maximum Entropy approach
Exact solution of the Curie-Weiss model (existance of the thermodynamic limit, evaluation of the thermodynamic limit, self-consistency equation)
Phase transition and spontaneous symmetry breakdown
Slides available here; see also [CKS] (chapt. 20), [A] (chapt. 3), [Ti] (chapt. 2), [Th].
22/11 - Room B - 14.30-16.30
Neurophysiological background
McCulloch-Pitts neurons
Local field alignment, parallel and sequential dynamics
Noiseless networks, synaptic symmetry and Lyapunov functions
Noisy networks, ergodicity, detailed balance and equilibrium distribution
Hopfield model (Hebbian kernel, orthogonal patterns and retrieval as pure state configurations)
Solution of the Hopfiel model at low-load via log-constrained entropy
Slides available here; see also [CKS] (chapts. 1, 3, 16, 20, 21), [A] (chapts. 2, 3, 4), [Ti] (chapts. 3, 4).
29/11 - Room G - 14.30-16.30
Self-averaging
Signal-to-noise for checking the stability of pure states and of symmetric mixtures of n patterns
Summary of the low-storage Hopfield model
Introduction to complexity
The Sherrington-Kirkpatrick model for spin-glasses
Replicas and overlaps
Solution of the high-storage Hopfield model via interpolation techniques at the RS level
Slides available here; see also [CKS] (chapts. 20, 21), [A] (chapts. 4, 6), [Ti] (chapts. 2, 3).
06/12 - Room B - 14.30-16.30
Introduction to machine learning: examples, ideas and tools
A review on statistical inference methods
Rosenblatt's perceptron
Minsky&Papert's perceptron
Binary classifier and regression
Bayesian learning
A link between Bayesian learning and "traditional learning"
Regularization, cross-validation and their meaning
Slides available here; see also [CKS] (chapts. 2, 6, 13), [J].
13/12 - Room B - 14.30-16.30
Restricted Boltzmann Machines
Supervised learning with RBM
Monte Carlo Markov chain sampling
Constrastive divergence algorithm
Simulated annealing
Mean-field methods
Examples of RBM
Slides available here; see also [CKS] (chapt. 14).
20/12 - Room B - 14.30-16.30
Integrate-and-fire model
Stein's model
Different approaches to information processing
Connection between Boltzmann Machines and Hopfield model
Bayesian equivalence between Hopfield retrieval and Boltzmann learning.
Beyond the Hebbian kernel: correlation and dilution
Getting closer to biology: topology matters
Multilayered Boltzmann machines and deep learning.
Outlooks and comments.
Slides available here; see also references therein.
Further discussions and seminars:
10/07 - Room B - 10.00-13.00
Francesco Allegra: "The Backpropagation Algorithm"
Elishan Braun: "Neural networks for solving differential equations"
Rania Giannopoulou: "The contrastive divergence method"
