Why Transformers Changed Everything

There’s a puzzle at the center of modern AI: why did transformers, unlike all the architectures before them, so suddenly and completely change what computers could do with language, images, and even code? For years, recurrent neural networks (RNNs) had been the standard, plodding through sequences one step at a time, like a careful accountant checking each number in a ledger. They worked, but never astounded. Then, almost overnight, transformers arrived and the difference was so dramatic it felt unfair—like someone replacing a shovel with a backhoe.

The trick was in how transformers pay attention. RNNs process data serially: word by word, pixel by pixel. Transformers look at everything at once. Instead of crawling through a sentence, they take in the whole thing, then decide, for each word, which other words matter most. This is called self-attention.

The mechanics are simple, but the effect is profound. For every token—say, the word “bank”—the model creates three vectors: a query, a key, and a value. The query asks, “What am I looking for?” The keys answer, “What do I have?” Each query is compared to every key, and the results become weights: how much should this word care about that one? It’s as if every word is in a room with every other word, and can listen as closely as it wants to any of them. This is why a transformer can tell if “bank” means money or rivers (or a bench)—it doesn’t have to wait for context to arrive one word at a time. It just sees the whole context and decides.

Stack enough of these self-attention layers and the model starts to build up complex relationships. Each layer refines the picture. Information from the original input is preserved along the way, so nothing important gets lost. Transformers also parallelize well, which means they can be trained on massive datasets in reasonable time. This is why models like GPT-4.1 exist at all.

But is this really understanding, or just a very clever way of matching patterns? Transformers are very good at association, but still struggle with things like commonsense reasoning and causality. They don’t really “know” what they’re talking about; they just know what tends to go together. Then again, maybe that’s how understanding works—at least at first. Layer enough associations, and meaning starts to emerge. Perhaps our own brains aren’t so different.

The implications go beyond AI. If you can process an entire sentence or image in one go, maybe intelligence isn’t about following steps, but about holding a whole network of relationships in your head and constantly updating what matters most. For programmers, understanding transformers means learning to frame problems not as sequences, but as webs of connections. For researchers, it opens up new possibilities—combining transformers with explicit reasoning, grounding them in real-world knowledge, or applying them to new kinds of data.

The deeper lesson here is philosophical: maybe understanding isn’t about following a path, but about seeing the whole map and deciding what matters. I consider my focus and attention to be my most cherished resources (from the conceptual metaphor: time AS money), so why shouldn’t the same be said of any intelligent system? The story of transformers is really a story about attention—about how meaning arises when you can see everything at once, and weigh it all, instead of just trudging from one step to the next. That’s why transformers work. And that might be why we do, too.