Transformer

Transformer is the deep-learning architecture that has served as the de facto foundation of modern AI since its introduction in 2017. Its core mechanism is attention, which lets a model selectively weight the importance of different positions in a sequence when producing an output—essentially deciding "which word to look at" when processing language . In the view of Moonshot AI founder Zhilin Yang, what made Transformer uniquely powerful was its near-limitless scalability: unlike recurrent or convolutional networks, which tend to plateau, Transformer keeps improving as parameters and compute are added, enabling the emergence of general-purpose learning .

The architecture has since become the target of fundamental critique and overhaul. Researchers note that standard Transformer lacks readable-writeable memory and recursive mechanisms, making it brittle on complex, long-horizon reasoning tasks where intermediate states must be tracked . Others have zeroed in on its "egalitarian" residual connections—every layer's output weighted equally—as an efficiency bottleneck in very deep models, since early, important signals get diluted by later noise . In response, the field is actively exploring alternatives and patches: state-space models like Mamba, linear attention variants, and stack-based mechanisms that promise better state management ; Yunqi Capital's reporting also flags Google's "Nested Learning" paradigm, which introduces multi-timescale updates inside Transformer-like blocks to mimic neural plasticity .

The name has taken on brand resonance beyond the technical layer. Yunqi Capital, for instance, explicitly named its youth-focused AI founder program "Y Transformers" as a dual nod to the architecture and to "transformation," signaling belief that the next generation of entrepreneurs will reshape the technology rather than merely adopt it .