"Deep Learning I"
M2 Data-Science
Geo roy Peeters, Alasdair Newson (Telecom Paris)
2022-2023
Teachers: Geo roy Peeters, Alasdair Newson (Telecom Paris, IP-Paris)
Deep Learning (machine learning based on deep artificial neural networks) has become extremely popular over the last years due to the very good results it allows for tasks such as regression, classification or genera- tion. The objective of this course is to provide a theoretical understanding and a practical usage of the three main types of networks (Multi-Layer- Perceptron, Recurrent-Neural-Network and Convolutional Neural Network). The content of this course ranges from the perceptron to the generation of adversarial images. Each theoretical lecture is followed by a practical lab on the corresponding content where student learn to implement these networks using the currently three popular frameworks: pytorch, tensorflow and keras.
Format : 6 sessions of 3.5 hours + Exam
Lectures content:
- Multi-Layer-Perceptron (MLP): Perceptron, Logistic Regression, Chain rule, Back-propagation, Deep Neural Activation functions, Vanishing gradient, Ini- tialization, Regularization (L1,L2,DropOut), Alternative Gradient Descent, Batch-
normalization
- Recurrent Neural Network (RNN) : Simple RNN, Forward Propagation, Backward Propagation Through Time, Vanishing/ Exploding gradients, Gated Units (LSTM, GRU), Various architectures, Sequence-to-sequence, Attention model
- Convolutional Neural Network (CNN) : CNNs use sparse connectivity and weight sharing to reduce parameters and create more powerful networks , connections are organized in a convolution operation, CNNs now provide the state- of-the-art in a vast array of problems, we will see how CNNs work and we will implement them for classification problems
Labs content :
- text recognition, sentiment classification
- music generation
- image recognition
- image generation
Programming language
- Python (numpy, scikit-learn, matplotlib)
- DL frameworks: pytorch, tensorflow, keras
- Use Télécom computers, your own labtop or colab.research.google.com → needs a Google account → open one before the first Lab !
- a Graphics Processing Unit (GPU) will not be required, however if you have one, this will speed up the learning process
Grading : 30% labs/project + 70% written exam
Abstract
Deep Learning (machine learning based on deep arti cial neural
networks) has become extremely popular over the last years due to the
very good results it allows for regression, classi cation or generation.
The objective of this course is to cover the three main types of
networks (multi-layer-perceptron, recurrent-neural-network and con-
volutional neural network). This course range from the perceptron to
the generation of adversarial images.
Each lesson is followed by a corresponding lab where student learn
to implement these networks using the currently most popular frame-
works (tensor
ow, pytorch and keras).
1 Format
6 sessions of 3.5 hours + Exam
2 Teachers
Geo roy Peeters, Alasdair Newson (Telecom Paris)
3 Grading
30% labs/project, 70% written exam
Lectures content
Multi-Layer-Perceptron (MLP)
Perceptron, Logistic Regression, Chain rule, Back-propagation, Deep Neural Acti-
vation functions, Vanishing gradient, Initialization, Regularization (L1,L2,DropOut),
Alternative Gradient Descent, Batch-normalization
Recurrent Neural Network (RNN)
Simple RNN, Forward Propagation, Backward Propagation Through Time, Van-
ishing/ Exploding gradients, Gated Units (LSTM, GRU), Various architectures,
Sequence-to-sequence, Attention model
Convolutional Neural Network (CNN)
CNNs use sparse connectivity and weight sharing to reduce parameters and create
more powerful networks , connections are organized in a convolution operation,
CNNs now provide the state-of-the-art in a vast array of problems, we will see how
CNNs work and we will implement them for classi cation problems
Labs
Content
text recognition, sentiment classi cation
generating music
image recognition
image generation
Programming language
Python (numpy, scikit-learn, matplotlib)
DL frameworks: tensor
ow, pytorch, keras
Use Telecom computers, your own labtop or colab.research.google.com !
needs a Google account ! open one before the rst Lab !
a Graphics Processing Unit (GPU) will not be required, however if you have
one, this will speed up the learning process