Skip to content
hackertubeai

Seattle Applied Deep Learning

LSTM is dead. Long Live Transformers!

About this video

Leo Dirac explains how transformer models have replaced LSTMs in NLP. He gives background from bag of words through RNNs and LSTMs, then a technical deep dive into multi-headed self-attention and positional encoding, with sample code.

Keep your curiosity going.

A few more ideas from the same corner of the archive.

The spelled-out intro to neural networks and backpropagation: building micrograd

Andrej Karpathy

But what is a neural network? | Deep learning chapter 1

3Blue1Brown

A Better Default Colormap for Matplotlib | SciPy 2015 | Nathaniel Smith and Stéfan van der Walt

Enthought

How Self-Driving Cars will Destroy Cities (and what to do about it)

Not Just Bikes