Transformers didn't come out of nowhere.1958: Rosenblatt's perceptron, the first artificial neuron that could learn.1980...

Transformers didn't come out of nowhere.1958: Rosenblatt's perceptron, the first artificial neuron that could learn.1980s-90s: RNNs read text one word at a time, with memory that fades on long sequences.1997: LSMTs remember longer, but still word by word2014: attention lets a model look at every word in the input2017: "Attention Is All You Need" drops the word-by-word reading. Parallel processing, faster training, huge datasets.That's where LLMs come from.#AI #MachineLearning