Microsoft Research Asia's Yuqing Yang: The Attention System of Agents | Attention

Long-context understanding is the critical path for large language models to tackle complex tasks and system orchestration.

And when it comes to solving the computational bottleneck in the Prefill phase, TriangleMix is one of the few bottom-up structural optimization methods that balances performance and accuracy.

This method was proposed by Dr. Yuqing Yang, Principal Research Manager at Microsoft Research Asia, and colleagues in the paper "TriangleMix: A Lossless and Efficient Attention Pattern for Long-Context Prefilling": a training-free, ultra-long-input-compatible attention pattern combination. Through a shallow-dense, deep-triangular-sparse structural design, it significantly reduces prefill latency while maintaining model output quality.

• TriangleMix is a structural layering scheme for attention that reduces Time to First Token (TTFT) by 12%–32% at 32K–128K sequence lengths, with attention kernel latency improvements of 3.7×–15.3×;
• The underlying logic: through gradient sensitivity analysis, it prunes useless Middle Q-K blocks, retaining only Streaming and tail aggregation regions;
• This optimization is training-free and can be stacked with dynamic sparsity methods (such as MInference, FlexPrefill), achieving end-to-end cost reduction without architectural changes;

For Dr. Yang's team, TriangleMix is not an isolated project, but part of a broader inquiry into attention mechanisms, information organization, context orchestration logic, and even agent-native systems thinking.

But what exactly has changed in the understanding of attention behind TriangleMix? How can it "almost losslessly" eliminate a massive chunk of computation? And can this method extend to memory, retrieval, and larger agent system architectures? It was these questions that led us into an in-depth conversation with Dr. Yang.

Before diving into the interview, here's a quick primer on TriangleMix's technical motivation and core structure.

When handling long-context tasks, large models' attention typically faces explosive computation growth during the prefill phase, with complexity scaling as O(N²) relative to input length. At 32K–128K input scales especially, this creates significant memory pressure and TTFT, becoming a major deployment bottleneck that hinders performance gains.

TriangleMix addresses this with a hierarchically sparse attention architecture: by analyzing gradient sensitivity of each layer's attention to final outputs, the authors discovered that deep layers show extremely low dependence on Middle Q-K regions. Therefore, they retain standard dense attention in shallow layers while switching to a triangle-shaped mask in deep layers — skipping the middle section and preserving only the front (Streaming region) and end (Last Q-K region), thereby dramatically reducing deep-layer attention computation cost and dropping complexity from O(N²) to O(N).

In practice, TriangleMix employs a layer-wise attention splitting strategy: the first 16 layers use standard full attention, while the latter 16 layers switch to triangle attention, activating only the lower-triangular region of the attention matrix (i.e., each query can only attend to keys before it).

This structure supports combination with existing dynamic sparsity methods (such as MInference, FlexPrefill) to build hybrid modes; it is also a training-free structural optimization that can be directly deployed on mainstream models like Llama-3.1 and Qwen2.5 without retraining.

Paper experiments demonstrate that on Llama-3.1-8B-Instruct and Llama-3-8B-262K, applying triangle attention to the latter 62.5% of layers (L_tri_start = 12) retains 99.7% of original performance.

This means: TriangleMix can employ O(N) attention structures across most deep layers without significant loss of expressiveness, achieving substantial inference acceleration.

The paper's empirical results also show that TriangleMix significantly reduces latency and memory consumption with near-zero accuracy loss.

Empirical results In the Llama-3.1-8B-Instruct model, triangle attention compresses per-layer kernel latency from 750ms (128K context) to 49ms, achieving a 15.3× speedup, with TTFT dropping 12%–32%.

On multiple benchmark tasks including RULER (Revisiting Long Context Benchmark) and LongBench, TriangleMix maintains nearly identical accuracy to dense attention, validating its "training-free + near-lossless" structural advantage.

We've organized our in-depth interview with Dr. Yang, centering on the research insights behind the paper and the broader system evolution path that TriangleMix connects to, covering discussions from structural design to deployment efficiency.

This article presents the edited interview, with an estimated reading time of 15 minutes.

Enjoy

"So I believe that rather than discussing attention in isolation, we should take a higher-level perspective — placing it within the larger issues of agent systems, training mechanisms, context representation, and task structure."

— Dr. Yang

Oasis Capital: Dr. Yang, you work at Microsoft Research Asia, a place where academia and industry intersect. From that intersection, how do you view and think about the TriangleMix research?

Dr. Yang: Let me start by introducing our overall research direction and framework.

We (the Shanghai Machine Learning Systems Group at Microsoft Research Asia) focus on co-innovation in systems and algorithms for large language models and agent systems. Our work concentrates on two main areas:

First, efficient computing for large models, particularly in long-context scenarios, with special emphasis on sparse attention mechanisms and their acceleration.

In this area, beyond TriangleMix, our team's key contributions include:

• MInference (NeurIPS 24) and MMInference (ICML 25) introduce sparse computation into attention, primarily reducing computation and latency (Time-to-First-Token, TTFT) during the prefilling phase;
• Retrieval Attention and its follow-up RetroInfer incorporate vector index retrieval techniques into attention computation and KV cache organization, enabling high-throughput inference with low GPU memory;
• SCBench (ICLR 25) provides systematic categorization and performance benchmarking of various sparsification methods from the perspective of KV cache sharing;
• LeanK (EMNLP 25) examines how mainstream position encoding techniques affect the frequency-domain distribution (dimension-wise) of KV caches, reducing storage and computation needs by eliminating redundancy in low-frequency domains;

The second area, which we call "Agent-Native Systems", involves systematic research on the development, optimization, and efficient deployment of agent systems. This work treats agents as first-class citizens of systems research, rather than focusing solely on the model component.

We recognized early on: When the system's service target is an agent rather than a single model invocation, agent-native systems offer far broader room for performance gains and efficiency optimization — enabling agents to become not only more efficient and cost-effective, but also to improve their work quality, better solve problems, and create real value.

Take Parrot (OSDI 24), which we proposed in 2024, as an example. Its starting point is that computation graphs in agent systems create additional optimization opportunities for inference systems. Traditional LLM inference systems optimize for "single-request" scenarios, yet no real-world agent completes tasks through single invocations. Systems that only consider single invocations are typically suboptimal at the agent level.

Oasis Capital: Could you elaborate on why agents cannot complete tasks through single invocations?

Dr. Yang: An agent is essentially a software program comprising multiple model calls with specific dependencies between them — for instance, one model's output becoming another's input — along with tool usage or database queries. This demands that system-level optimization consider the "entire task chain" rather than single inference steps.

In practical deployment, we've observed two notable patterns:

First, system-level optimization from a global agent perspective consistently yields additional performance gains. In some scenarios, we've seen improvements exceeding 10x over traditional approaches. This is because the optimization objective shifts — from "making one request fast" to "making an entire task set more synergistic."

Second, an interesting shift: An increasing share of LLM traffic originates not from humans, but from programs calling themselves. These programmatic invocation chains function more like a new kind of system than traditional "user input + model output" pipelines.

Oasis Capital: Does this shift also affect the architecture design of training services?

Dr. Yang: Yes. We've also launched a new project called AgentLightning focused on agent training and optimization. The question we address: facing the vast diversity of agent implementations, how do we build a standardized training service? Crucially, this service must be non-intrusive — many optimization methods assume you'll use a particular framework, but many real-world projects have no unified framework at all; some developers find frameworks themselves burdensome.

So we're building an "Agent optimization middleware" that opens new possibilities for enhancing base model capabilities. AgentLightning provides unified data interfaces, allowing continuously generated, meaningful agent interaction data to flow into base models in standardized form, thereby further improving their capabilities.

On another front, it sketches a blueprint for future AI application development. By seamlessly empowering all agents with model optimization capabilities as a service, AgentLightning significantly lowers the barrier to developing, iterating, and deploying high-performance adaptive agents.

We also explore training-free optimization approaches, such as prompt optimization and context optimization.

Oasis Capital: So the structural representation of context becomes a critical issue as well?

Dr. Yang: Exactly. In current agent applications, developers often need to place different data objects into context — tables, files, code repositories, and so on.

But these diverse objects require conversion (rendering) into tokens the model can process. Due to distribution differences in training data, these conversion requirements are often model-specific; inappropriate conversion easily degrades final performance.

Take a table: whether you flatten it as text or encode it as structured tokens yields completely different results. And these decisions shouldn't fall to agent developers — just as frontend engineers don't manually manage DOM trees.

So we built a framework called POML, somewhat like HTML and frontend frameworks for web development. Developers simply declare the object type, and the system automatically converts it to underlying structures, then maps to appropriate token representations. This isn't merely a developer convenience tool — it also yields insights about context: what granularity we should use to understand, manage, and optimize model context.

These insights have sparked interesting thinking for our Attention + Context architecture.

Oasis Capital: So your team's focus has shifted from optimizing individual modules to constructing entire system logic?

Dr. Yang: Precisely. Our team currently focuses on "Agent-Native Systems", including:

• Agent optimization middleware: Decoupling agent developers from system layers, supporting scheduling of diverse strategies including model training, prompt, and context optimization.
• Multimodal structural fusion: For instance, Video RAG and memory components, fusing semantic memory with knowledge graphs to serve different task requirements.
• Population-sensitive system design: Including code interaction optimization for visually impaired users, daily training agents for cognitively impaired individuals — using agents to assist special populations.

Underlying optimization techniques remain attention + memory + retrieval, but we now focus more on how to integrate these into service-oriented, deployable, interpretable system solutions.

Because our work itself takes a systems perspective, we don't view attention as a single module. I believe that rather than discussing attention in isolation, we should take a higher-level perspective — placing it within the larger issues of agent systems, training mechanisms, context representation, and task structure.

TriangleMix is merely one form of innovation among many.

But if we broaden our view, we find that attention mechanisms are becoming the most strategically important control center across AI systems — they determine not only "where to look," but also "where to retrieve information from," "what to retain," and "at what granularity and perspective to match." In essence, attention has transcended its definition as an NLP module to become a structural foundation for AI agent systems.

— Dr. Yang

Oasis Capital: How should we understand placing attention within these larger issues? And how does TriangleMix help advance this goal?

Dr. Yang: We know that large models are controllable generative models — their outputs are determined by their inputs. Drawing on the memory hierarchy concept from computer systems, we can understand this input-to-output process through an attention hierarchy:

• At the bottom is what we call the "infinite knowledge pool" — internet data, databases, documents, tables, which may be open-domain or domain-specific knowledge such as medical manuals, legal statutes, and so forth;
• The middle layer comprises model inputs in various contexts, often called prompts, context, or model memory;
• The top layer is the model's processed internal state space and its generated outputs;

And between these three layers:

• The transition from bottom to middle layer is primarily handled by the user (or developer), retrievers, and memory modules. Beyond manual intervention, this step relies heavily on retrieval operations — a critical underlying component in many systems, such as common retrieval-augmented generation (RAG) or agent memory systems;

• The transition from middle to top layer is mainly accomplished through internal model computation, where the relationships between Tokens are determined primarily by Attention.

From this perspective, Attention is not an independent module — it is part of the complete information filtering mechanism of large models. To use a simple analogy, this mechanism functions like an information scheduler: deciding what to look at, what to read, and what information to retain.

Starting from this viewpoint, we can actually redefine: in what scenarios should which type of Attention be used? The key lies in the trade-off between Efficiency and Effectiveness.

Oasis Capital: So what we're actually facing is an "information retrieval" problem?

Dr. Yang: Yes, at least we can understand the problem from this angle. And this retrieval mechanism has two possible paths: Retrieval (due to the widespread use of embedding-based vector retrieval, we sometimes use Embedding to refer to all retrieval technologies in discussion, though this isn't always entirely rigorous) and Attention.

For simple problems, like fact retrieval, where the question (the riddle) and answer (the solution) can be directly mapped one-to-one, Embedding retrieval is the most efficient approach. Sparse retrieval (such as various techniques based on TF-IDF) or vector matching can handle these very well.

But if the problem is more complex, requiring multi-step reasoning or combining multiple knowledge chunks, Embedding alone is insufficient — this is where Attention has the advantage.

Mathematically, Attention is a token-level inner-product retrieval: Query matches Key, weights are assigned via Softmax, and relevant information is extracted from Value. You can think of it as a "differentiable search engine," where each token performs query enrichment.

Oasis Capital: Can Attention constitute a more advantageous retrieval mechanism?

Dr. Yang: Yes, we can look at this from three angles:

First is Granularity: Embedding typically operates at the chunk or document level, while Attention operates at the token level, enabling finer-grained reasoning;

Second is Dimensionality: Embeddings are typically tens to hundreds of dimensions, while Attention uses multi-head mechanisms where each "head" corresponds to high-dimensional hidden states that add up to thousands of dimensions at minimum, enabling richer multi-perspective representation;

Third is Capacity: Embedding models are typically weakly trained with poor generalization, whereas Transformer models activate learned knowledge pathways through the Attention mechanism, with stronger representational and reasoning power.

There's also a more important point: Attention doesn't complete its work in a single step — it can construct a reasoning path, starting from the question and connecting intermediate information points step by step to piece together the complete answer. This is like continuous query enrichment, constantly transforming the query content to complete the final information generation.

Oasis Capital: Similar to an information scheduling system.

Dr. Yang: Exactly. To better achieve matching between user intent and existing information, modern search engines employ retrieval augmentation and rewriting techniques — this relies on "explicit augmentation." The internal computation in LLMs, however, performs "self-organizing augmentation": before each token is generated, it initiates a query, dynamically matching Keys in the context to find corresponding Values. What Attention does is implement retrieval augmentation inside the model.

Our series of Attention-related work can all be understood as improvements to the performance and efficiency of large models' internal information retrieval mechanisms — such as computation sparsification (MInference and MMInference), reorganization and sparsification of KV Cache (Retrieval Attention and subsequent work RetroInfer), and so on.

TriangleMix itself is a relatively new and interesting extension of this line of work: it structurally introduces Streaming zones, Middle Q-K zones, and Last Q-K zones, then uses Gradient distribution to judge which regions are worth preserving and which can be pruned. Attention is no longer a dense matrix, but a task-driven information selection system.

Moreover, TriangleMix is just one form of innovation. But if we broaden our view, we'll find that the attention mechanism is becoming the most strategically important control center across the entire AI system. It not only determines "where to look," but also "from where to retrieve information," "which parts to retain," and "at what granularity and perspective to match." Essentially, it has transcended its definition as an NLP module to become a structural foundation for AI agent systems.

Oasis Capital: In the "attention hierarchy" you just mentioned, is the bottom-layer "Infinite Knowledge" the global memory? Many papers also mention Global/Local/Context — how do these concepts map?

Dr. Yang: These terms are essentially analogies to memory hierarchy, not strict definitions.

The bottom layer (Infinite pool) can be understood as global knowledge, including all web content, offline data, and so on. The middle layer (Context) includes structures like Memory, Retrieval, and Prompt used to "prepare inputs"; the top layer (Working Memory) is the space where the model currently processes tokens.

What we're focused on here are actually two properties:

One is Locality: Most tasks don't require global knowledge, but rather focus on some sub-block within the context;

The other is Dynamicity: The information blocks required for each task change dynamically.

Because the optimization space for Attention comes from these two dimensions.

Doing this well remains very challenging, especially in terms of training data quality, multimodal alignment details, and long video segmentation encoding strategies. But from an architectural perspective, there are no fundamental technical obstacles.

It is precisely in these long-context, multimodal practical systems that we increasingly realize: performance bottlenecks often lie not in the model itself, but in the massive and ever-growing K/V Cache system behind it.

— Dr. Yang

Oasis Capital: So your research has actually already extended beyond the Attention layer to the Memory layer?

Dr. Yang: That's right. We experimented early with approaches similar to Graph RAG and commercial applications. Though we didn't publish formal papers on them, we were already using them in projects. For us, this layer is a very natural and necessary area of work in agent systems.

Oasis Capital: But compared to the flourishing of new structures in the Attention layer, doesn't the Memory layer still lack real technical breakthroughs?

Dr. Yang: Actually, no. This layer is currently a very hot research area. Many open-source or closed-source agent memory implementations have emerged — these are all innovative attempts at underlying memory architecture.

It's just that from a technical perspective, many methods and concepts originate from search engine and knowledge graph era technologies — only the execution methods are completely different now.

For example, we used to slice dialogue content by rules or topics; now large models automatically complete these tasks. Entity extraction has also evolved from early regex and BERT approaches to generative extraction using different scales of language models at the SLM level, which can both improve performance and control costs.

This also raises a core trade-off: computation vs. accuracy. Standard attention is token-level full connection, with strong expressiveness but extremely high computational cost; at the same time, such strong connectivity also reduces robustness to noise, as models tend to "use all information" even when that information is wrong. New corpora now heavily incorporate agent-style interaction data, such as tool use, ranking, and search, which in turn enhances models' ability to handle complex information structures.

Oasis Capital: So does this understanding of the Attention + Memory structure also apply in multimodal systems?

Dr. Yang: Completely applicable. On one hand, work like MMInference has proven that sparse attention mechanisms can absolutely be used in multimodal models, and the Attention + Memory architecture can serve as a "universal backbone" for multimodal systems. We recently had a paper accepted at NSDI called AVAS, which does Video-RAG — users can ask questions directly to a video that's several hours long. For example, in a soccer match video, a user could ask, "Who took the third shot?" In this system, we use an Event Knowledge Graph to process raw video information, and have a large model retrieve and traverse the graph to answer user questions. More importantly: tokenization and alignment between mainstream modalities has now been largely achieved, and many models can already process visual and audio modalities. Even more cutting-edge modalities, like WiFi sensing signals, can be aligned with video modalities through representation learning — something our lab is also working on.

Of course, actually doing this well remains very challenging, especially around training data quality, multimodal alignment details, and long-video segmentation encoding strategies. But fundamentally, we don't see any insurmountable technical barriers.

And it's precisely in these real-world systems with long context and multiple modalities that we've increasingly realized: the performance bottleneck often isn't the model itself, but rather the massive and ever-growing KV Cache (intermediate computation result cache) system behind it.

Oasis Capital: Could you expand on this ever-growing KV Cache system behind the scenes?

Dr. Yang: In our evaluations, we actually found that when context length rises to 100K or even 1M, TTFT increases roughly quadratically, with latency reaching tens of minutes; GPU memory consumption increases almost linearly. The main issue isn't that the model is large — it's that there's too much KV Cache data.

Oasis Capital: So you started focusing on "write" and "read" scheduling strategies to optimize this?

Dr. Yang: Right. In our SCBench work, we categorized common Sparse Attention methods from the perspective of the KV Cache lifecycle (such as reads and writes). There are two particularly typical approaches:

The first category compresses KV at write time, selectively discarding some KV during the write phase;

The second writes everything but selects at read time. All KVs are retained first, and selective loading is performed during the Decode phase.

Our experiments found that the ceiling for "selective reading" is higher than "selective writing." Information discarded at write time cannot be recovered, whereas selective loading during reading can be more intelligent and task-relevant. So our current optimization strategies focus on two directions:

• First, making writes and reads structured and semantically aware, reducing meaningless full-KV operations;
• Second, building a hybrid KV Cache system across GPU + CPU + RAM, moving low-frequency information down to lower-cost memory systems to achieve storage tiering.

This is the path from Attention module optimization toward true system-level scheduling.

(On NSA's work) From a technical perspective, I think this is currently the clearest and most robust Block-wise sparse selection solution. It also inspires us that future Attention system designs should: don't force the selection, but let the model itself learn "how to select."

— Dr. Yang

Oasis Capital: We've mentioned many optimization strategies. At their core, the goal of Dynamic sparse attention is: for each Token, choose the positions it should most attend to. How is this lookup process implemented? What are the key challenges?

Dr. Yang: The core challenge is efficiently implementing Top-K matching.

Each time a new Token is generated, the system needs to pick out the most relevant Keys from existing ones. The traditional approach is Dense softmax, but the computational cost is too high. The mainstream solution is to divide Keys into Blocks (for example, every 16 or 32 Tokens per block), select one or more representative points for each block, and have the Query compute Attention only with these representative points. How well these representative points are chosen determines the performance ceiling of Sparse attention.

Oasis Capital: So how do you choose "representative points" appropriately? Many methods perform poorly precisely because of this, right?

Dr. Yang: Exactly. Early commonly used strategies include mean vectors (average of all Keys), representative tokens, and extreme value concatenation (constructing representative tokens using max/min values per dimension).

But the problem with these methods is: clustering in high-dimensional space isn't linear, and neither means nor extremes can represent the true distribution of Tokens. The result is either decreased sparsity (that is, to access one Token, you end up reading multiple Blocks), or information degradation (that is, representative points become ineffective, and the model reads a "blurry average" that actually misleads judgment).

Oasis Capital: Are there better solutions now?

Dr. Yang: Yes. DeepSeek's NSA (Native Sparse Attention) is a very good solution.

NSA's core idea is: don't select representative points, learn how to generate representative vectors. It uses convolution + pooling to extract information from each Block, and trains a generation function that dynamically outputs abstract representations of the Block.

The benefits of this approach are:

• First, it can combine with pretraining distribution to automatically learn which directions are "important";
• Second, the parameterized generation process can be fine-tuned to adapt to different context distributions;
• Third, compared to directly selecting Tokens, this learnable approach is more expressive for representations in high-dimensional space.

This idea is actually isomorphic to the Q/K/V mappings we normally see — all project original Token representations into task-specific spaces, except this time the goal is sparse selection rather than prediction.

NSA essentially isn't selecting points in Token space, but mapping Blocks to another task-specific vector space, where the most representative expressions are selected.

Oasis Capital: Sounds like it's changed from "picking center points" to "learning a generation mechanism."

Dr. Yang: Right, and NSA's important contribution also lies in explicitly pointing out: don't directly learn the positions of center points, but learn a generation mechanism. This is exactly opposite to what some previous work did. Some earlier work tried to directly learn certain center points to fit the distribution of the entire space, with worse results.

Why? Because Token distributions in high-dimensional space aren't linearly separable, nor easily indexable. NSA's approach lets the model itself learn a low-dimensional space suitable for "representative expression," thereby indirectly improving the efficiency and effectiveness of Sparse attention.

So from a technical perspective, I think this is currently the clearest and most robust Block-wise sparse selection solution. It also inspires us that future Attention system designs should: don't force the selection, but let the model itself learn "how to select."

Oasis Capital: Besides representative point selection, is Block partitioning method also very critical?

Dr. Yang: Extremely critical. Traditional partitioning is equidistant, such as every 16 Tokens as one block, but this is actually "positionally adjacent," not "semantically related." A more effective approach comes from Retrieval Attention — introducing Vector Index and KNN clustering: after mapping Tokens to vectors, partition Blocks by semantic similarity.*

This approach has two major challenges: first, it must support dynamic vector relevance computation, and can't use precomputed rankings;

second, real-time requirements are extremely high — Block index construction and invocation must be fast enough.

Of course, the benefits are also obvious: more cohesive semantic blocks, less redundant reading/writing, stronger cache hit rates, especially suitable for Retrieval-heavy scenarios.

Oasis Capital: So besides grouping and representative points, what other Attention structures can be optimized?

Dr. Yang: There's a key but underestimated point: the existence of Patterns.

We used to think of Sparse attention as random, but Attention maps actually have structure in training. Beyond common sliding windows and block sparsity, a new sparse pattern has been found: the Vertical-Slash Pattern, which contains two parts:

One is the vertical pattern: some Tokens (such as numbers, times, place names) are important at multiple positions;

The other is the slash pattern: representing Relative position, such as Sliding window, syntactic chains.

These patterns aren't accidental, but are areas that appear with high frequency in training data and are repeatedly attended to. We can use probabilistic modeling to dynamically predict high-probability access regions, thereby reducing Block count and improving prefetch efficiency. This also inspires us to move from "static partitioning" toward "distribution-based dynamic grouping" — no longer mechanically splitting by index, but letting data "cluster into blocks" itself.

Oasis Capital: Do these mechanisms only act on the current Q? Is there any structure to follow between different Qs?

Dr. Yang: Yes, and it's very critical. There is actually structure between Q and Q, especially in language where adjacent Tokens often talk about the same thing — they should share similar Attention regions. And in Group Query Attention (GQA), Q also has a natural dimension: Qs in the same Query group share the same K, which is also a commonly used processing method currently.

There's another very interesting observation: different parts of the model will develop endogenous differences and differentiation during training, for example:

The first is head differentiation: different "heads" specialize into distinct functions — some handle local context, some handle long-range retrieval, and others lean toward summarization. The heads don't overlap, but remain stable over time.

The second is layer differentiation: lower layers are denser, processing fine-grained language; deeper layers become sparser, expressing higher-level semantics.

And this structural differentiation is essentially the model "self-organizing" an information scheduling logic internally — it's not statically designed, but spontaneously formed during large-scale training.

Many people talk about Agents in terms of model scheduling and retrieval efficiency;

But we have always believed that the value of an Agent lies not just in computational speed, but in whether it can become a person's "second operating system" — especially for those who need more assistance to begin with.

— Dr. Yang

Oasis Capital: Beyond foundational attention optimization and system architecture design, as we briefly mentioned at the start of our interview, you're also pushing to have these systems actually serve specific populations, right?

Dr. Yang: Yes. We pay close attention to how Agent systems land in human-centered scenarios, especially for users who are highly dependent on technology but poorly served by traditional systems.

Oasis Capital: Do you have any specific cases you could share with us?

Dr. Yang: For example, we're currently running a research project on the programming experience for developers with low vision. Programming itself is a relatively accessible form of work for people with visual impairments — they only need a device, without requiring frequent spatial interaction. We're investigating how to optimize Agent feedback strategies and interaction structures so that these users can fully benefit from technological advances.

We're also advancing another project: a personalized cognitive training framework called "Yiwo" Reme. This innovative tool will help advance cognitive training research and provide new methods for various cognitive impairments, including early prevention and non-pharmaceutical intervention for mild cognitive impairment.

Oasis Capital: So the goal isn't whether the technology "can run," but whether "real people can actually use it" — building an AI that works?

Dr. Yang: Exactly. We prefer to define our overall technical direction with the concept of an "Agent-native system": it encompasses structural optimization (such as scheduling, computation, and storage layering) while also focusing on who is using it, how they're using it, and whether they can actually access it.

We want this system to achieve deep optimization in inference efficiency, computational cost, and resource layering, while also becoming a platform that developers genuinely want to use and that vulnerable populations can draw support from. Just like the optimization middleware we recently designed: the goal is to decouple the Agent developer from the system optimizer, so developers who don't understand the underlying layers can "click two buttons" and get a more suitable optimization solution.

Many people talk about Agents in terms of model scheduling and retrieval efficiency; but we have always believed that the value of an Agent lies not just in computational speed, but in whether it can become a person's "second operating system" — especially for those who need more assistance to begin with.

Starting from structural optimization, TriangleMix allows us to re-examine the underlying logic of the attention mechanism: it is not a module, but a systematic methodology about information selection, focal allocation, and system scheduling.

And it is precisely this methodology that lets Attention travel from inside the model to system architecture, and further extend to Agent behavior encapsulation, context expression structures, and even the cognitive boundaries of human attention itself.

The boundaries between technology and system, and between system and human, are being drawn closer, overlapping, and reconstructed at the point of the attention mechanism.

We believe that Attention is no longer merely one link in model technology, but an ongoing discussion about how humans and AI can co-build an information world. From TriangleMix to Flash Attention, from retrieval paths to cognitive structures, from token-level to system-level, from efficiency questions to philosophical boundaries — the meaning of Attention is far from exhausted.

In praise of vitality.

Let's Think Together

For humans, the attention mechanism can help us better "choose."

Do you think Agents will also develop better attention mechanisms, even surpassing humans, thereby demonstrating greater flexibility and adaptability in complex decision-making?

Feel free to share your thoughts in the comments.

We will also organize offline events from time to time to discuss these topics face-to-face with scholars and technology enthusiasts — stay tuned!

To view the original research paper, click "Read Original" to jump to it.

We thank Dr. Yang for accepting Oasis Capital's in-depth interview, and extend our respect and sincere gratitude to him and his colleagues for their work.

Oasis Capital continues to sincerely invite top researchers, developers, and thinkers worldwide to jointly advance this exploration of attention.

If you are working on attention-related research — whether in algorithms, systems, cognition, neuroscience, or product design and content construction — and would like to explore and dialogue with us, please contact us through the QR code at the end of this article!