MaHui Founder | Shunhao Fan of Hongyu Technology: What Kind of Operating System Does the XR Era Need?

源码资本源码资本··0·3

#MaHui Entrepreneurs# Resilient and Bold, Exploring Without End

By: Shunhao Fan, Founder of Hongyu Technology.

The History of Personal Computing Is Also a History of Human-Computer Interaction Revolution

Every transformation is, at its core, a shift in how humans and machines interact.

When personal computers first emerged, command lines were the earliest form of human-computer interaction. The Apple II launched the personal computer industry in the late 1970s, followed by the "IBM-compatible PC + DOS operating system" combination that gradually became the mainstream configuration. Understanding these terms is, for today's Gen Z users, essentially an archaeological expedition into tech history. Every operation required pulling command characters from memory, inputting them accurately, to get accurate output.

The 1984 debut of the Macintosh changed everything. The graphical user interface (GUI) fundamentally transformed how computers presented information and enabled interaction. A light click of the mouse, dragging windows at will — this dramatically lowered the barrier for ordinary people to operate computers. The Macintosh ran its own macOS, while Microsoft's Windows operating system targeted the compatible PC market. Together, they drove mass adoption of personal computers and transformed human productivity and lifestyles. A wave of world-changing tech companies emerged in their wake. Leading companies including Adobe, Google, Facebook, and Amazon continue to shape the direction of technology today.

From PC to mobile, human-computer interaction opened up an entirely new world. The defining moment came when the iPhone redefined what a phone could be, making touch the dominant form of human-computer interaction. At the time, the iOS running on iPhones still served only Apple hardware. Other hardware manufacturers on the market had to choose alternative operating systems to meet end-user needs. Ultimately, Android became the biggest winner in the open ecosystem compatible with diverse hardware.

Touchscreens swept the world. Their natural, intuitive interaction style gave rise to the thriving mobile internet industry atop them. Compared to the PC internet era, far more users could now use electronic devices to make their lives more convenient. Internet companies began competing for scenarios — smartphones, tablets, car center consoles, anywhere a screen could reach, the mobile internet followed.

One characteristic or principle of human-computer interaction evolution is that it continuously approaches how humans acquire and process information in the real world — more natural experiences, richer and more dimensional forms of information presentation. Images carry more information than text; video is richer still than images.

Image source: Hongyu Technology

Multiple factors determine and support changes in human-computer interaction: the continuous iteration of hardware computing power, displays, and input devices. On the software side, the operating system is the foundation.

Downward, it serves as the hardware interface, ensuring any model or specification of hardware can run smoothly. Upward, it defines and carries the application ecosystem — what development languages apps choose, what service capabilities they provide, even what they look like, all determined by the operating system.

Operating systems have attracted numerous companies and technical luminaries at home and abroad to throw themselves into development, generation after generation. Whether in the PC or mobile era, even facing numerous challenges, no one has ever abandoned the effort to build operating systems.

What Kind of Operating System Does XR Need?

Everyone wants to know: what comes after the mobile internet? From hardware devices to user experience to content construction, what transformations will occur across the entire system?

As early as 2014, Facebook acquired the young company Oculus for $2 billion. At the time, Mark Zuckerberg said VR would be the next computing platform after mobile. In 2021, Oculus Quest 2 shipments approached ten million units. To demonstrate its all-in commitment to the Metaverse, Facebook rebranded as Meta, and the Oculus Quest series became Meta Quest.

An increasingly clear consensus is emerging: the most probable future direction is VR/AR. From foundational technologies like spatial computing and AI that tech companies are investing in, to the strategic moves of giants like Facebook (now Meta), Apple, Tencent, and ByteDance — all increasingly support this conclusion.

Image source: Hongyu Technology

Compared to the previous generation of mobile internet, the most fundamental change VR/AR brings is that users can interact with立体, realistic information in three-dimensional space. VR (Virtual Reality) refers to computer-created, rendered simulated 3D spaces where information is presented. AR (Augmented Reality) refers to projecting information directly into real three-dimensional space as its backdrop.

We understand VR/AR may evolve through three development stages: Stage one, immersive VR experiences for specific domains and scenarios, such as verticals like gaming and training. Stage two, generalization of scenarios, combining virtual and real — what AR is currently attempting to achieve. Stage three is the ability to fully create a truly integrated virtual-real social system, the story told in the film Ready Player One. But reaching this third stage will require much longer accumulation.

For stage one, relatively mature hardware products like Quest already exist. For stage two, AR devices are currently constrained by engineering challenges, seeing more use in B2B contexts without truly mature consumer products yet. The closest to mature solution currently is Video See-Through VR (VST VR), which builds on relatively mature VR hardware by adding cameras that capture the real world, reconstructing reality before the user's eyes to achieve an AR-like effect. This approach is also sometimes called MR (Mixed Reality). VR, AR, and MR are collectively referred to as XR.

Traditional VR resembles a game console, while VST VR is more like an extension of the PC, and AR more like an extension of the phone.

Currently, the most commonly used personal hardware devices in ordinary life fall into two categories: portable and mobile. Phones are truly mobile devices — use them wherever you go. Portable is different: laptops are portable, but you can't just open them anywhere, anytime; you still need to consider the environment.

Correspondingly, VR headsets can be portable, but if you wear them walking down the street or riding a bike, you'll likely feel disoriented, and others will find you strange. So they still need relatively fixed scenarios for use, making them more of a PC extension. AR, meanwhile, lets you handle things anytime, anywhere — more like a phone extension.

Image source: Hongyu Technology

From the Operating System Perspective, XR Will Bring Three Levels of Change to the Application Ecosystem.

One: Changes in Interaction Form

Intuitively, XR frees content from flat surfaces, presenting it directly in 3D form within real space. This brings richer spatial interaction methods like gestures, eye tracking, and spatial awareness. Additionally, compared to smartphones, XR can present content across a full 360-degree range, giving users a larger field of view (FOV). Users will naturally expect to operate multiple tasks simultaneously. For VST VR and AR, the characteristic of 3D multi-tasking means XR system interaction forms are fundamentally different from the mobile era.

Image source: Hongyu Technology

Two: Changes in Application Patterns

Compared to smartphones, XR devices have limited thermal dissipation space (glasses-form-factor AR has even less thermal headroom than phones). Computing power is constrained by power consumption, yet XR scenarios demand more computing power to present 3D content. For developers, especially for AR, local computing power is limited. To deliver excellent user experiences, they need to leverage multi-end computing power — phones, cloud, PCs, even cars may participate in processing. This places demands on the operating system's ability to allocate and coordinate computing power across multiple devices. From the user perspective, in multi-tasking scenarios, super-apps native to Android deliver degraded experiences. Because content can now integrate with real space, users increasingly want scene-based services delivered directly. One imaginable experience: obtaining services immediately when needed without downloading or installing apps, somewhat similar to "3D mini-programs."

Three: Changes in Content Distribution Mechanisms

Compared to PCs and smartphones, XR devices can collect far more user and environmental information. Combined with changes in application patterns, next-generation smart terminals can enable richer content distribution mechanisms — like intelligent assistants that push information in real-time based on environmental awareness. Achieving such experiences requires collaboration across the industry chain. Our current work focuses primarily at the operating system level, designing system底层 architecture and interaction frameworks around multi-sensory perception characteristics, enabling efficient and convenient integration of various perception algorithms.

VST VR and AR, as general-scenario smart terminals based on virtual-real integration, bring fundamental changes across three levels — interaction form, application patterns, and content distribution. These characteristics all place new demands on the operating system.

Image source: Hongyu Technology

In the context of VST VR and AR, existing Android systems have clear shortcomings.

The most obvious problem: XR applications should be able to present in 3D form within space, and support spatial interaction methods like gestures. Yet Android itself was designed around 2D touchscreens. From the system perspective, an app is simply a full-screen frame; interaction is limited to taps and swipes within that frame.

Furthermore, XR needs multi-tasking support. In Android's existing APK system, application lifecycle design was originally conceived for single-task scenarios. Although Android's underlying Linux supports multi-processing, Android's design primarily allows only one foreground app to interact with the user. This creates a series of problems in multi-tasking scenarios. For example, Android's "background app killing" — where user apps are automatically terminated — is unacceptable in a multi-tasking experience.

Similarly based on 2D single-task design philosophy, Android is not restrained in resource consumption. Portions of Android system code are implemented in Java, which brings considerable additional overhead. On the application side, the same app on Android can consume 3x the memory of its PC version. For XR devices with relatively scarce resources that need 3D multi-tasking experiences, this is a massive problem.

Therefore, XR requires an entirely new operating system.

Hongyu in the New Wave

Compared to the inflection point of the smartphone era, in today's context of China's rapid development and intense US-China competition, China has greater need and greater opportunity to create a truly XR-native operating system.

We began developing XR operating systems in 2016. Before 2019, XR hardware manufacturers in the industry were primarily startups. We collaborated with many of them. Starting in 2019, riding the 5G wave, domestic first-tier hardware giants began paying more attention to XR. We were fortunate to provide underlying operating systems for several first-tier manufacturers. To date, we have adapted our system for several dozen XR hardware devices, and we hope to provide more help to industry partners.

Since Hongyu's founding in 2016, we have been continuously exploring and practicing how applications should be defined under next-generation human-computer interaction forms, and what kind of system is needed to support them. Today, we can increasingly clearly feel the arrival of consumer electronics and internet transformation. In this wave, as a Chinese force, we hope to contribute to industry progress and ride the wave forward.

For more MaHui companies, click "Read Original" to view