Skip to content
hackertubeai

Sasha Rush

Do we need Attention? - Linear RNNs and State Space Models (SSMs) for NLP

About this video

Sasha Rush surveys linear RNNs and state space models as replacements for attention in transformer-style models, in a talk for MLSys. It covers the foundational preliminaries and predates Mamba.

Keep your curiosity going.

A few more ideas from the same corner of the archive.

The spelled-out intro to neural networks and backpropagation: building micrograd

Andrej Karpathy

But what is a neural network? | Deep learning chapter 1

3Blue1Brown

A Better Default Colormap for Matplotlib | SciPy 2015 | Nathaniel Smith and Stéfan van der Walt

Enthought

How Self-Driving Cars will Destroy Cities (and what to do about it)

Not Just Bikes