Interdisciplinary Attention Mechanism Interview Series: Introduction

In 2017, a paper titled Attention Is All You Need introduced the Transformer architecture, establishing the structural foundation for generative AI. From language models to multimodal models, from BERT to GPT, and on to the rise of diffusion models, the attention mechanism has remained at the core of every technological leap. The widespread adoption of Stable Diffusion broke through the existing logic of image generation, pushing "denoising" to the forefront as a structural way of thinking: no longer trying to "construct" images, but operating on an entirely new premise:

The image was always there — it was simply obscured by noise.

This has been Oasis Capital's underlying methodology all along.

Looking back at the main thread of AI's technical trajectory over the past seven years, attention is the common substrate beneath nearly every key advance. It is not merely a model component, but a paradigm about structure, focus, and information distribution — and a migration of technical methodology.

From that moment on, we had already entered a new era.

Standing at today's inflection point, we are launching an in-depth interview series centered on attention, focusing on interdisciplinary research into the "attention mechanism."

This article is a Q&A ahead of that series, attempting to clarify one question: If this isn't a retrospective or tribute to a classic topic, then why — today — do we need to talk about attention again?

What gets obscured by noise isn't just images — it's also markets.

In the first half of 2022–2023, mainstream market discourse was caught in hesitation and debate over whether AI was a massive bubble, and whether this generation of AI was fundamentally different from the last. Amid that noise, Oasis Capital completed the core construction of our current AI and embodied intelligence portfolio in the first half of 2023 — nearly twenty projects including MiniMax, Vast, Boson, LimX Dynamics, Spirit AI, and Hypershell Tech.

Because we believed this was innovation on a scale beyond the Industrial Revolution — shorter in duration, greater in magnitude.

We then launched Oasis Capital's first in-depth interview series, themed "AI."

The motivation came from a realization we formed while building this AI portfolio: this was not a revolution driven by product or operational model changes, but a scientific exploration centered on the frontier of artificial intelligence. So we needed to step back and return to the most essential question — "What is AI, really?" — engaging with top professors and scholars globally to discuss what artificial intelligence is, what GPT is, and what technical and cognitive foundations the changes we were witnessing were actually built upon.

At that time, Oasis Capital interviewed dozens of professors worldwide. Through these conversations, a clear picture emerged: the large models we were seeing were essentially future infrastructure. In a binary world, intelligence would be standardized, managed, and distributed — like today's electrical grid — supplying model capabilities everywhere that "electricity" was needed, with the receiving endpoints being the "appliances" of the AI era.

This insight became the closing frame of Oasis Capital's first AI interview series, while also opening a new question: If we understand the form of the "power system," then what will the future's "home appliances" be?

Thus Oasis Capital launched its second in-depth interview series — Agent.

In July 2023, mainstream market opinion coalesced around two bets: one held that the future belonged to vertical domain-specific large models, the other to the evolution of general-purpose large models themselves. At that time, few looked beyond the model itself to examine the systemic form that would carry model capabilities — Agent.

Though we wrote repeatedly and emphasized in our dialogue for "The People Betting Hardest on AI": we do not believe the future belongs to vertical models; model generality will inevitably be the endgame. But focusing on generality alone isn't enough — what matters more is how model capabilities get packaged into interfaces, which is what Oasis Capital saw as Agent.

Agent is the future.

Today, Agent has become a prominent field in AI, but stepping back to mid-2023, it enjoyed neither mainstream market favor nor unified theoretical recognition.

So Oasis Capital launched its second in-depth interview series, themed Agent, once again seeking out top researchers and professors globally to deliberate on an essential question: When we talk about Agent, what exactly are we talking about?

This series lasted nearly a full year, until August 2024, and through the course of these conversations, the answer gradually emerged: Agent is not something fragmented or a wrapper of some kind. At the micro level, Agent is an activatable and adaptable behavioral unit, approaching something like a living organism; at the macro level, Agent resembles more a river.

Essentially, Agent is a demand-and-intelligence-integrated service driven by large models and manifested through specific scenarios. Its core is not tools, but a mode of being.

With that, the second interview series came to a close.

We are grateful to the many researchers who engaged deeply with Oasis Capital across our AI and Agent series — they collectively charted the critical path of this exploration. Now, we are launching our third in-depth interview series, themed Attention.

So returning to the question at the beginning of this article: What prompted our third topic selection?

As mentioned at the outset with that famous paper Attention Is All You Need, humanity has always been trying to teach machines one thing:

What is attention?

But why is humanity so fixated on teaching machines to understand attention?

Consider a simple example: when humans drive, we instinctively notice changes in road signs or a rabbit that suddenly darts out — but AI may not. This is certainly not because AI isn't smart enough. On the contrary, it's precisely because the human brain's computational capacity is extremely limited, far below the total information received by the retina, that we were forced to evolve a mechanism called attention. This mechanism allows humans to rapidly lock onto the most critical information at any given moment and filter out noise that is currently less important.

Unfortunately, AI does not inherently possess this mechanism. In AI's world, all pixels are equal; given infinite compute, it will attempt to process every input completely. Thus for a long time, humans have been seeking a methodology, building a new paradigm to give AI attention — and good scalability (scaling law) — believing this would enable AI to process information better.

As technical exploration has advanced, we have been pleased to see, for example, that MiniMax — an Oasis Capital portfolio company — recently released Flash Attention, optimizing the attention module within the Transformer architecture itself and significantly improving computational efficiency during both training and inference, achieving a breakthrough for attention at the algorithmic level. Yet the significance of the attention mechanism has long since transcended model structural optimization itself. Over the past few years, attention has not only driven breakthroughs in language models but has gradually permeated neuroscience, cognitive science, psychology, and other disciplines. We are beginning to realize that the process of AI learning attention is, in turn, helping us re-understand human perception and cognition itself.

So what is the conclusion?

The conclusion is that we see AI exhibiting a dual evolutionary path: on one hand, scholars globally are attempting ever-larger-scale training on the Transformer architecture; on the other hand, at the level of cognitive structure and algorithmic frameworks, there are attempts at further innovation to improve and push AI toward the one thing we have always wanted it to learn — what is attention.

If today we are to continue deeply understanding AI's future, the next step of exploration should point toward an even more essential question:

In a system composed of both humans and AI, what does attention actually mean?

Taking one step further, as we move from technical research to examining human society itself, as Agents become society's primary producers and grow ever more attuned to humanity, human attention mechanisms will face unprecedented challenges. Twenty years ago we read books, ten years ago we watched films, five years ago we watched short videos — now we will be lost in infinitely fragmented AI-generated information.

Any thought we have can point toward infinite information; the world will fragment further.

Thus, an even deeper question begins to surface: As humans help AI learn and improve attention, how do we protect our own?

The answer may not be optimistic.

Data shows that the average person picks up their phone over 500 times per day; sustained attention is being compressed to less than 100 seconds. From feature films to short videos, from deep reading to information fragments, the window of attention humans can maintain is in continuous decline. Meanwhile, AI is pushing the speed of information acquisition and response to unprecedented levels. If a super AI emerges that can precisely capture human preferences, predict needs, and generate all desired content, will human attention mechanisms fall further? Might attention itself be outsourced? Will humans ultimately hand over the "power to attend" entirely to machines?

Buddhism speaks of the "consciousness-mind" — where the human mind's consciousness rests, the world manifests. From a signal processing perspective, attention determines the frequency of consciousness; where our frequency lies, our time-domain lies. Translated into scientific language, the principle is similar: a person's ultimate self-management is ultimately attention management. In this era of inevitable human-AI symbiosis, understanding "attention" is not only necessary for clarifying AI's technical development, but also a necessary path for humanity's own development.

Help AI build attention, while also helping ourselves protect attention.

This is our answer to the question at the beginning of this article, and the starting point for our third in-depth interview series.

This series will run longer than our previous two; Oasis Capital will invest more time and resources to complete it. We believe that in doing this, we will meet like-minded friends, and we look forward to building new understanding together with all of you.

The first installment of this series will be published in August, with monthly updates to follow. We hope you enjoy it.

Celebrating vitality.