index.md

layout	splash
header	overlay_color overlay_filter og_image overlay_image #000 0.7 /assets/images/intro-to-dl.png /assets/images/splash.png
excerpt	with flavor of Natural Language Processing (NLP)
description	Deep Neural Networks (FFNN, CNN, RNN, LSTM, GRU), Word Embeddings, Encoder-Decoder (Attention, Transformer, GPT, BERT).
intro	excerpt This site accompanies the latter half of the [ART.T458: Advanced Machine Learning](http://www.ocw.titech.ac.jp/index.php?module=General&action=T0300&GakubuCD=4&GakkaCD=342305&KeiCD=23&course=5&KougiCD=202004845&Nendo=2020&lang=EN&vid=03) course at [Tokyo Institute of Technology](https://www.titech.ac.jp/english/), which focuses on Deep Learning for Natural Language Processing (NLP).
lecture1	title speakerdeck Slides id ratio 932bbd2dad6b45eaac6c4ad0270740c4 1.44428772919605 url image_path alt title demo-slp.html /assets/images/demo-slp.png Interactive single-layer perceptron Interactive SLP model url image_path alt title demo-mlp.html /assets/images/demo-mlp.png Interactive multi-layer perceptron Interactive MLP model url image_path alt title excerpt https://github.com/chokkan/deeplearning/blob/master/notebook/binary.ipynb /assets/images/pytorch.png Implementations in Jupyter notebook Implementations Perceptron algorithm in numpy; automatic differentiation in autograd, pytorch, TensorFlow, and JAX; single and multi layer neural network in pytorch.
lecture2	title speakerdeck Slides id ratio a37a47297ebe447f980eeafe7cdcdd7f 1.44428772919605 url image_path alt title excerpt https://github.com/chokkan/deeplearning/blob/master/notebook/mnist.ipynb /assets/images/multi-impl.png Implementations in Jupyter notebook Implementations Preparing the MNIST dataset; perceptron algorithm in numpy; stochastic gradient descent in numpy; single and multi layer neural network in pytorch.
lecture3	title speakerdeck Slides id ratio 86260627cf1e41e8876e2a0e581c4670 1.44428772919605 url image_path alt title excerpt https://github.com/chokkan/deeplearning/blob/master/notebook/resnet.ipynb /assets/images/resnet.png Using ResNet-50 Object recognition Classify an image using ResNet-50. url image_path alt title excerpt https://github.com/chokkan/deeplearning/blob/master/notebook/convolution.ipynb /assets/images/convolution.png Convolutions as Image filters Image filters Various image filters by manually setting values of a weight matrix in torch.nn.Conv2d.
lecture4	title speakerdeck Slides id ratio bb5ae4e2e81a453699c453d1db158b28 1.44428772919605 url image_path alt excerpt title https://github.com/chokkan/deeplearning/blob/master/notebook/word2vec_en.ipynb /assets/images/word2vec_en.png Loading word vectors pre-trained on English newspapers; computing similarity; word analogy Loading word vectors pre-trained on English news; computing similarity; word analogy English Word Vector url image_path alt excerpt title https://github.com/chokkan/deeplearning/blob/master/notebook/word2vec_ja.ipynb /assets/images/word2vec_ja.png Loading word vectors trained on Japanese Wikipedia; computing similarity; word analogy Loading word vectors trained on Japanese Wikipedia; computing similarity; word analogy Japanese Word Vector
lecture5	title speakerdeck Slides id ratio 13c7414498d843b5ae7d539a14f66f50 1.44428772919605 url image_path alt title excerpt https://github.com/chokkan/deeplearning/blob/master/notebook/rnn.ipynb /assets/images/structure-impl.png Implementations in Jupyter notebook Implementations RNN; Mini-batch RNN
lecture6	title speakerdeck Slides id ratio e9af4583e098484881d8259647ffbb5d 1.44428772919605

{% include feature_row id="intro" type="center" %}

Lecture #1: Feedforward Neural Network (I)

Keywords: binary classification, Threshold Logic Units (TLUs), single-layer neural network, Perceptron algorithm, sigmoid function, stochastic gradient descent (SGD), multi-layer neural network, backpropagation, computation graph, automatic differentiation, universal approximation theorem.

{% include feature_row_custom id="lecture1" %}

Lecture #2: Feedforward Neural Network (II)

Keywords: multi-class classification, linear multi-class classifier, softmax function, stochastic gradient descent (SGD), mini-batch training, loss functions, activation functions, ReLU, dropout.

{% include feature_row_custom id="lecture2" %}

Lecture #3: Convolutional Neural Network

Keywords: Convolutional Neural Networks (CNNs), MNIST, 2D convolution, padding, stride, channels, image filter, max pooling, ILSVRC, ImageNet, AlexNet, VGGNet, ResNet.

{% include feature_row_custom id="lecture3" %}

Lecture #4: Word embeddings

Keywords: word embeddings, distributed representation, distributional hypothesis, pointwise mutual information, singular value decomposition, word2vec, word analogy, GloVe, fastText.

{% include feature_row_custom id="lecture4" %}

Lecture #5: DNN for structural data

Keywords: Recurrent Neural Networks (RNNs), Gradient vanishing and exploding, Long Short-Term Memory (LSTM), Gated Recurrent Units (GRUs), Recursive Neural Network, Tree-structured LSTM, Convolutional Neural Networks (CNNs).

{% include feature_row_custom id="lecture5" %}

Lecture #6: Encoder-decoder models

Keywords: language modeling, Recurrent Neural Network Language Model (RNNLM), encoder-decoder models, sequence-to-sequence models, attention mechanism, Convolutional Sequence to Sequence (ConvS2S), Transformer, GPT, BERT.

{% include feature_row_custom id="lecture6" %}

<script type="text/javascript" > window.onload = function () { $(window).trigger('resize'); } </script>