Three Months, 300 Plugins: What Kind of Ecosystem Is ChatGPT Building? | Yunqi Tech Talks

In our "FutureScope" column (click to jump straight there), we shared observations on the "large model ecosystem": problems that can't be solved directly with model capabilities are being addressed by the "ecosystem," with various high-quality tools further lowering the technical barrier to AI and continuously energizing the AI community's vitality and influence.

Plugins are an indispensable part of the ChatGPT ecosystem. Observing where plugins are headed may reveal OpenAI's strategic thinking on ecosystem layout, as well as the key directions worth watching as models and applications evolve in tandem.

In this edition of "Yunqi Tech Talk," we share with you the development status of ChatGPT plugins over the past three months, hoping to bring inspiration to entrepreneurs actively embracing AI's new changes. Enjoy~

The following content is republished from the WeChat account "Overseas Unicorn"

Authors: Cage, Tianyi Cheng

Editor: penny

"OpenAI's App Store moment" — that was the market's reaction three months ago when ChatGPT plugins first launched. Over the past three months, OpenAI has taken a rather conservative approach to scaling this ecosystem. Fewer than 300 plugins have passed review, and plugin users remain in the hundreds of thousands.

On one hand, this is by design. As OpenAI president Greg Brockman put it, the cost of opening up ChatGPT plugins is extremely low — basically just API documentation, except written for language models to read. On the other hand, this also exposes that OpenAI isn't omnipotent: it lacks extensive experience in cultivating a safe, thriving, and rapidly scaling open platform and application ecosystem. Sam Altman has even admitted that ChatGPT plugins haven't found product-market fit yet.

Despite not being the world-shaking force some imagined, plugins still represent a milestone for ChatGPT as a product, and another step toward "openness" for OpenAI. In this article, we explore some genuinely important long-term questions now that the hype around plugins has subsided:

• Which companies and developers are actively integrating with plugins?
• How can plugins ensure safety, performance, and overall system stability simultaneously?
• What is the analytical framework for observing an application ecosystem built on top of an open platform?
• What new opportunities do plugins create?

...

Beyond these questions, the most promising near-term development is what Microsoft can do with this ecosystem once plugin standards become interoperable:

OpenAI lacks reviewers, product managers, experts who can operate this ecosystem, and better, more extensive UIs beyond chat interfaces with widgets. Microsoft has all of these — ample personnel to deploy, a wide range of products where plugins can be embedded and prove useful (Bing Chat and Windows 11 Copilot on the consumer side, Microsoft 365 Copilot and Dynamics 365 Copilot on the enterprise side, and more).

Below is the article's table of contents; we suggest targeted reading based on key points of interest.

👇

01 Background

02 Plugin's Existing Partnerships and Future Impact

03 Plugin Development Methods and Capability Boundaries

04 Plugin's App Store Ambitions

05 Will Plugins Kill LangChain?

06 Plugin Roadmap and Ripple Effects

Background

OpenAI released ChatGPT plugins on March 23, 2023. Though the official emphasis was that this move aimed to better study ChatGPT's real-world usage to address safety concerns, outsiders pinned high hopes on its potential for OpenAI's ecosystem building and commercial prospects. Chinese- and English-language communities briefly considered this OpenAI's "App Store moment", one that would directly "kill LangChain and Fixie."

In April, developers and ChatGPT users continued their enthusiasm for this ecosystem. Beyond official partners, numerous third-party plugins began development based on the framework and standards OpenAI defined, while users flooded the waitlist and became organic promoters on Twitter for star plugins like Code Interpreter.

The most concentrated changes came in May, when OpenAI finally began scaling more broadly, announcing in late May that it would roll out plugins to all Plus subscribers with over 70 third-party plugins available. However, this full rollout remained highly restrained: Plus users had to flip the Beta features switch, the iOS app didn't display plugins by default, and users had to find plugins they'd used in previous sessions.

Using plugins on mobile Source: Pietre Schiano

Running parallel to this, Microsoft announced at its Build conference that it would adopt OpenAI's plugin standards, allowing developers to build plugins across both the OpenAI (ChatGPT) and Microsoft (Bing Chat, Copilots across product lines) ecosystems. It laid out a comprehensive plugin framework through ChatGPT plugins, Teams message extensions, and Power Platform connectors. Given OpenAI's research-oriented nature, Microsoft is more likely to carry the plugin ecosystem forward.

After this large-scale rollout, the reality ChatGPT faces on both sides is that plugins remain a very early-stage ecosystem:

• Given that total plugin users number only in the hundreds of thousands, official plugin partners likely can't expect particularly large traffic boosts in the short term; ChatGPT's current UI itself has many limitations; Airbnb felt that this chat-box-plus-bottom-widgets UI wasn't truly suitable for travel products and terminated its plugin project before launch;
• Individual developers with existing APIs available can submit API docs and other files according to OpenAI's standards in mere tens of minutes, but meeting UI and data privacy specifications and passing review often involves waiting periods filled with uncertainty;
• Users found that most plugins proved rougher than demos suggested, making it difficult to have a unified product experience within ChatGPT; most discussion centered on Code Interpreter and Web Browsing, and the Web Browsing exploration process was frequently maddening.

ChatGPT's frequent "trial and error" Source: Reddit

We feel it's time to treat OpenAI as an earthly species — it has world-class AI researchers, but they can't magically, naturally solve the inevitable challenges of product development and two-sided ecosystem building — to more realistically imagine plugins' current state, ecosystem potential, and evolution possibilities is the focus of this article.

Plugin's Existing Partnerships and Future Impact

The ChatGPT plugin ecosystem once ignited imaginations about ChatGPT, the product with the steepest user growth curve in history. Over two months, plugins have seen feature updates, new plugin introductions, and scaling of usage. Here we summarize some current plugin capabilities and assess whether recent updates met expectations.

As of late May 2023, the ChatGPT plugin store offered 85 plugins, roughly divisible into 6 categories by capability:

1. External Interaction: These plugins provide the ability to interact with external web pages and products.

Among these, Browsing is an official OpenAI plugin, building on the same conceptual thread as the lab's late-2021 paper WebGPT: using a language model to explore and comprehend content from the external web. In multiple user interviews, when users have a clear objective, its browsing and summarization capabilities outperform New Bing, though its stability remains relatively weaker.

Of the external tools, Zapier stands out as the most useful. As a leader among aggregators, Zapier's core capability is interacting with APIs from over 5,000 external applications, giving it mature API interaction logic. However, due to input context limitations (which can't accommodate prompt documentation for dozens or hundreds of APIs) and OpenAI's differing product engineering priorities, aggregation capabilities remain a weak point in the Plugin ecosystem in the near term. As a result, Zapier benefits in the medium-to-short term from ChatGPT's text comprehension capabilities.

2. Programming & Development: Providing development environments or productivity tools for coding.

The most attention-grabbing is Code Interpreter, developed officially by OpenAI. It provides a sandboxed development environment where users can experiment with code execution and data analysis. OpenAI has invested heavily in Python security to ensure that code executed in this environment remains safely isolated without directly altering or affecting external network environments. Because of this characteristic, usage of Code Interpreter has concentrated on another Python strength that doesn't require deployment — data analysis. Code Interpreter can summarize and process data, then produce substantive data visualizations on command, showing glimmers of surpassing traditional BI tools, though controllability still needs improvement.

3. Personalized Interaction: Bringing memory and personalization capabilities to ChatGPT.

As mentioned in previous Pinecone research, Greg considers the Retrieval plugin the most distinctive of all plugins, because it fills a gap ChatGPT previously lacked: retrieval augmentation from personalized memory and proprietary data. The plugin structure is simple — it retrieves semantically relevant content from various vector database APIs based on the prompt, then hands it to ChatGPT for comprehension and learning. Additionally, DAN is an interesting plugin in this category, offering the ability to alter ChatGPT's personality.

4. Life Assistant: Information integration for daily needs — food, clothing, housing, transportation.

This category has seen the heaviest influx of large companies. Expedia and Kayak handle travel and hotel bookings, OpenTable manages restaurant reservations, and Instacart is a leading grocery delivery platform. These products hope to capitalize on ChatGPT's natural language understanding to capture this new platform as an incremental traffic acquisition channel. On one hand, ChatGPT has been relatively restrained and conservative in fully rolling out this functionality, so it hasn't yet delivered obvious traffic-driving effects. On the other hand, the user quality filtered through chat-based interaction is higher than traditional customer acquisition channels.

5. Vertical Domains: Windows into proprietary data.

Much high-quality data in specialized verticals falls outside ChatGPT's core competencies, so vertical domain plugins can bring more professional, proprietary content. For example, Vogue has fashion industry data, Crypto Prices has public web3 data, and the UN has annual policy change documentation. These represent critical information for understanding and generating content within specific verticals.

6. Workplace Efficiency: Productivity tools.

These plugins are mostly collections of workplace efficiency utilities — PDF reading and summarization, to-do tracking, email composition, and text-to-speech are the main current capabilities.

Having covered the specific capabilities of current Plugins, we can see they touch many different verticals, with some potentially disrupting existing product forms at large companies, while others represent opportunities for independent developers to accumulate early high-quality users. Below we analyze the impact across these domains.

1. Search

Examples: Google, Bing, Perplexity

• Usage frequency: ★★★★★
• Short-term impact: ★★★★ (short)
• Long-term impact: ★★★ (short)

ChatGPT's integrated browsing functionality will, in the short term, deliver some shock to search engine capabilities. Since LLMs largely amount to having learned all public information on the internet, combining that with browsing capabilities and inherited content synthesis and reasoning abilities means ChatGPT users will significantly reduce their search usage in the near term.

But Google has responded quickly, already rolling out an LLM-integrated search engine internally. Given Google's infrastructure strength and deep understanding of search, it's entirely possible they could build a better AI search engine than either New Bing or ChatGPT. Yet regardless of which company prevails, perceptions of search will in the long term be partially disrupted by LLMs, as the traditional index-and-rank query mentality gradually erodes under the advance of generative prompt-based interaction.

2. Aggregators

Example: Zapier

• Usage frequency: ★★★
• Short-term impact: ★★★★ (long)
• Long-term impact: ★★★★ (short)

Cross-application scheduling and interaction demands will grow significantly in the LLM era. So despite currently relatively low usage frequency for aggregators, we've given this a medium-frequency rating.

Currently ChatGPT has strong tool-use capabilities but lacks API aggregation know-how. Thus Plugin's emergence benefits aggregator products like Zapier in the medium-to-short term. Zapier has deep accumulation in this space — now when people want to perform complex operations through ChatGPT, such as summarizing text and posting to social media, or recording in Google Workspace, they choose the ChatGPT + Zapier approach. In many use cases, ChatGPT only needs to plug into an aggregator to deliver excellent user experience; it doesn't need to integrate numerous APIs directly, with the SEO-like portion effectively handled entirely by the aggregator.

But in the long term, such products face several challenges: First, API organizational forms may change, and cross-product interaction frequency in the LLM era may [shift]. OpenAI's recent function calling capability release has significantly improved API usability, changes that may weaken Zapier's moat. Second, aggregators may become subsumed into operating system opportunities, with Microsoft, Google, and Apple all potentially building corresponding capabilities on their own systems — a fiercely competitive landscape.

3. Online Booking Platforms

Examples: Expedia, Kayak

• Usage frequency: ★★
• Short-term impact: ★★★ (long)
• Long-term impact: ★★★★★ (short)

These platforms primarily provide information organization and fulfillment capabilities. Organization means better understanding users to help them efficiently book suitable flights and hotels; fulfillment means ensuring transaction security and smoothness, mitigating default risks.

With the ChatGPT Plugin launch, these platforms may in the short term gain a high-quality customer acquisition channel, but will likely face significant long-term disruption. Because chat's improved semantic understanding enables better information organization, users can efficiently express needs and find suitable flight information. The platform's remaining core defense becomes its transaction fulfillment value — a much thinner value proposition than before.

4. O2O / E-commerce Platforms

Example: Instacart

• Usage frequency: ★★★
• Short-term impact: ★★★ (long)
• Long-term impact: ★★★ (long)

Compared to online booking platforms, O2O and e-commerce platforms add offline supply chain organization capabilities. This portion is immune to LLM-driven disruption — the AI algorithms involved are largely traditional ML optimization approaches, not task types that LLMs excel at handling or orchestrating. Thus these platforms face relatively smaller impact, mostly gaining an additional high-quality customer acquisition channel.

5. Independent Developers & Entrepreneurs

Example: Giftwrap

• Short-term impact: ★★★★ (long)
• Long-term impact: ★★★ (long)

For entrepreneurs, finding product-market fit and early high-quality users is challenging, and joining the ChatGPT Plugin ecosystem offers a relatively fast shortcut. Chat-form users are higher quality than those from traditional paid acquisition channels, since they've been retained through a higher-interaction-threshold filtering process, yielding better conversion rates. Moreover, integrated products gain access to users' complete conversation prompts — critically valuable for understanding consumer user profiles.

How Plugins Are Built and Where Their Limits Lie

I. The barrier to plugin development isn't high

OpenAI provides a plugin integration framework that both companies and individual developers can follow to connect their services. This low-barrier approach is by design — as Greg put it, you're essentially writing API documentation for a language model:

The integration requires just two core components:

1. API endpoints containing multiple functions that define data inputs and outputs for different scenarios. Take Speak, which we'll examine later: when a translation task is needed, it calls the Translate endpoint; when explaining a specific expression, it uses the Explain Phrase endpoint. Which specific logic to invoke depends on the second component;

2. A manifest file that teaches the product to call APIs through natural language prompts. The heart of this file is a natural language description helping the system understand what the plugin itself can do. When users activate certain plugins, ChatGPT evaluates whether the capabilities needed in the prompt match the descriptions in the plugin's manifest file, then determines when to call an API and which endpoint would most accurately achieve the goal.

For example, Speak's manifest file for this multilingual translation and learning plugin was reverse-engineered, revealing roughly this content:

• When users ask about content in another language, call the Speak plugin to serve their language-learning intent;
• When users provide a clear phrase or sentence for translation, invoke the Translate endpoint;
• When users give a more ambiguous task like "how do I compliment someone's outfit in Spanish," call the explainTask endpoint;
• When users need detailed explanation of an expression like "what does putain mean in French," invoke the explainPhrase endpoint.

The current writing style resembles development documentation and comments. But as more APIs come online, documentation styles will diverge — products will need to differentiate themselves from competitors in the same category, and product documentation will increasingly follow an SEO-like approach to highlight unique advantages, optimizing both docs and APIs.

II. Plugins provide a security evaluation capability

As plugin development and usage scale, security concerns will become increasingly important. On one hand, whether plugin providers overuse user data; on the other, whether plugins game the system to boost their own priority for traffic.

In 2017, Adobe landed in controversy after adding silent installation and site access permissions to its Chrome extension. The plugin included functionality to convert every webpage users opened into PDFs, requiring permissions to read and modify webpage content — sparking significant user backlash.

Security and permissions are critical components of product usability, and both Adobe and Chrome bore some responsibility in that controversy. Adobe's product likely wasn't excessively tracking user browsing history, but it should have been less aggressive in pushing a feature users didn't clearly want; Chrome's permission management at the time didn't break down read and modify permissions into separate tiers, leaving users worried the plugin could alter content during generation.

ChatGPT Plugin's security evaluation approach is genuinely novel: leveraging LLM comprehension and role-playing capabilities to make it the plugin security reviewer. According to information reverse-engineered by Twitter user rez0, who researches prompt injection and hacking, the AI reviewer's instructions divide into three sections: directives, facts, and policies.

The directives section primarily defines the AI's role: act as a product security engineer at OpenAI, analyzing a third-party plugin containing two files for compliance (covering six basic security questions such as whether it collects personal information, whether it has capabilities for illicit activities, etc.), then scoring the plugin's security level and age appropriateness based on these criteria.

The facts and policies sections provide the AI reviewer with decision-making foundations. The policies section explicitly prohibits political and sexually explicit content; the facts section defines risk tier classifications:

1. Low-risk plugins use only public data, with no personal information involved;

2. Medium-risk plugins involve interactions between individuals or enterprises and third parties;

3. High-risk plugins use high-risk data (financial data, medical data, other privacy-sensitive user information), or carry potential fraud risks.

III. Plugins will evolve toward more complex systems

Current plugins still have several immature aspects:

1. The number of plugins the model can invoke simultaneously is very limited:

Currently plugins only support activating three at once, with ChatGPT reading their manifest files into the context window. Given the 32k input context limit, descriptions for 5-10 plugins may be the upper bound of what the model can process in the near term.

2. Most current plugin API designs remain fairly traditional:

After receiving a prompt, ChatGPT processes and interprets it based on descriptions to compose structured inputs for conventional APIs. The advantage is developers can efficiently repurpose previously built APIs; the downside is insufficient flexibility for developers.

Future API formats will likely change — rather than transmitting processed structured data as input, the raw prompt will be passed directly to developers, who will build their own prompt comprehension and utilization into the API.

3. Current model description documents require continuous adjustment based on competitive landscape and model comprehension:

For instance, when a new competitor enters a vertical domain with more granular, more focused specialization, the large model will route all opportunities in that focus area to the competitor.

Addressing these issues will require a more sophisticated plugin retrieval system. This system would likely contain several layers:

• Plugin Store: Provides unified documentation standards to manage tens of thousands of plugins, allowing API developers to register, update, and delete after review. Upon entry to the Plugin Store, specific tags should be added for each plugin (like categories and information in the App Store), used for subsequent retrieval and usage;

• Plugin Retriever: Responsible for recalling and recommending the 5-10 most relevant APIs based on user needs. During retrieval, the Retriever matches prompt information against tags and descriptions in the plugin store to find the most relevant plugins;

• Action Executor: Responsible for calling APIs to execute generated action code, invoking APIs, and returning final execution results (with a potential ranking step here — the model's API selection process resembles the fine-ranking stage in recommendation systems).

The Plugin App Store Ambition

I. "App Store" isn't a new concept

Given OpenAI's outsized influence, its plugin launch has been framed as an iPhone App Store moment. We believe that objectively speaking, having an app and plugin ecosystem doesn't guarantee an open platform's success. Looking at SaaS giant history, companies reaching certain scale inevitably build ISV ecosystems, using "open platform + ISV" to avoid custom development demands and capture value through revenue sharing. Whether such a platform can be built successfully is a crucial test for reaching hundred-billion-dollar platform status.

Snowflake Platform Take Rate Data

Over the past decade in the US, few heavyweight ecosystems have been built on the consumer side — Meta never managed to create a mini-program-style app distribution ecosystem like WeChat's. But once you achieve App Store-level scale on the consumer side, it means:

• Ecosystem value 1-2 orders of magnitude higher than SaaS platforms;
• Ecosystem app count 2-3 orders of magnitude higher than SaaS platforms;
• Take Rate roughly double that of SaaS platforms.

A fun fact: Apple's App Store wasn't the first "App Store," but the concept did originate with Steve Jobs.

Around 2000, Salesforce founder Marc Benioff was feeling uncertain about his company's direction and sought Jobs's advice. Jobs gave him three recommendations:

1. Grow 10x within 24 months, or it's over;
2. Land a major enterprise customer, like Avon;
3. You must build an App Economy.

Benioff acted immediately. He realized his product resembled Exchange more than Store, and eventually launched AppExchange in 2005. Benioff had previously purchased the App Store trademark and appstore.com URL on Jobs's advice, then gifted both to Apple in 2008.

II. Deriving the Decisive Factors for Plugin Ecosystem Success Through One Framework and Five Case Studies

The Framework

Setting aside the "App Store" lens and focusing specifically on plugin ecosystem construction atop open platforms, we believe Figma offers the best framework for analyzing an ecosystem's success potential:

- Security: Including user data privacy, permission management, and the boundary of permissions and capabilities between ISVs and the platform;

- Stability: Platform speed shouldn't be degraded by plugins; platform updates shouldn't break existing plugins; the platform should also provide unified plugin installation management across multiple devices;

- Low-barrier development: The platform's defined framework and language should be sufficiently accessible — while meeting other prerequisites — to let developers quickly get up to speed and contribute to a robust plugin ecosystem;

- Performance: Plugins themselves should be fast and stable.

Beyond this framework's inherent universality, we were struck by the remarkable similarity between ChatGPT and Figma — their plugin ecosystems are fully cloud-native, web-first, built on robust open platforms, and embedded within the core product. At the same time, we believe this framework lacks coverage of developer incentives and distribution mechanisms after plugin proliferation.

Five Case Studies

We applied this framework to evaluate five open platform ecosystems widely regarded as massively successful, using this multi-case study approach to derive guiding insights on ChatGPT plugin prospects and pitfalls:

Apple App Store: The Most Successful OS App Store

Security: ★★★★★

Stability: ★★★★★

Low-barrier development: ★★

Performance: ★★★★

Incentives: ★★★★

Distribution: ★★

Beyond its well-known successes, Apple's lessons in incentives, distribution, and low-barrier development offer highly relevant guidance for OpenAI today:

• Apple successfully built a user account system atop its OS and constructed its own end-to-end global payment network — when Apple launched the App Store in 2008, it only supported lifetime买断制付费 [one-time purchase], introduced In-App Purchase in 2009, and In-App Marketplace in 2010, giving developers complete incentive capabilities. OpenAI's account system and payment infrastructure remain very early-stage, though its strategic partnership with Stripe may be a promising sign;

• After more than a decade of evolution, the App Store has shown clear distribution problems, with pronounced Matthew Effect dynamics where mass-market apps struggle to get distributed and properly matched. After experimenting with personalized recommendations via Genius, LBS distribution through Near Me, and discovery incentives via Explore, Apple ultimately failed to find smarter matching logic and reverted to the Curation and Editorial approach that persists today. OpenAI may have new ways to break through on plugin distribution — more intelligent, non-user-initiated selection methods could support matching at much greater scale;

• App development barriers are quite high, though starting with the 2010 iPad launch, Apple began giving developers lead time to build apps leveraging new device and OS capabilities. So when users made purchases, they'd find a rich selection of new apps available. ChatGPT has already adopted this approach with GPT-4 and plugin launches, but clearly lacks experience in scaling both sides of the marketplace.

Salesforce AppExchange: Empowering Sales Atop the Platform

Security: ★★★★

Stability: ★★★★

Low-barrier development: ★★★

Performance: ★★★★

Incentives: ★★★★

Distribution: ★★★

AppExchange is a success that many overlook, containing substantial foundational work well worth watching to see whether OpenAI can continue optimizing user experience along these dimensions: Salesforce's frontend framework for developers evolved from its own design system to Lightning Web Components; the backend grew from basic SDKs, APIs, and Metadata frameworks to an event-driven Pub/Sub API architecture; and external integrations progressed from IDE integrations like VS Code to the comprehensive integration solution of Salesforce Flow.

Most apps that gained traction on AppExchange initially targeted SME and mid-market segments. For those pursuing enterprise accounts, apps could anchor to 3-5 named accounts, then leverage Salesforce's sales channel for introductions before spinning up independent sales processes. If OpenAI needs to build grassroots success stories in the future, this would be a solid paradigm.

Chrome Extensions: An Once-Overlooked Ecosystem

Security: ★★★★

Stability: ★★

Low-barrier development: ★★★★

Performance: ★★★

Incentives: ★★★

Distribution: ★★

Most Chinese entrepreneurs and investors were so fixated on "mobile internet" that they overlooked opportunities growing from Chrome Extensions. Beyond Grammarly and Loom (which we've written about), the first validation of Chrome's massive ecosystem success was Honey, the e-commerce coupon aggregator that sold to PayPal for $4 billion in cash.

Chrome's ecosystem offers some food for thought regarding OpenAI: unlike the App Store's position on mobile, Extensions were long viewed as merely one of several web channels, alongside official websites, apps, and social media accounts. Only after 2019 did more companies begin treating it as a primary acquisition channel. If ChatGPT plugins become merely a "secondary acquisition channel" for KAYAK, Instacart, and Expedia, it could fall into a similarly awkward position. However, based on developer feedback, despite launching with numerous partners, the actually running plugin ecosystem leans more toward new, independently developed, grassroots plugins.

Figma Community: Community as Differentiator

Security: ★★★★

Stability: ★★★★★

Low-barrier development: ★★★★★

Performance: ★★★★★

Incentives: ★★

Distribution: ★★

Figma skillfully balanced security, stability, and performance through technical means, while demonstrating the importance of "low barriers" for both sides of the marketplace: Adobe XD and Sketch both had longer-established plugin ecosystems, but users typically had to discover and download plugins outside the community, while developers needed to write plugins in languages like C. Figma, by contrast, offered designers more familiar languages like TypeScript for in-framework development, and created a seamless experience integrated with the core product.

VSCode Extensions: Ecosystem Surpassing Internal Optimization

Security: ★★★

Stability: ★★★★★

Low-barrier development: ★★★★

Performance: ★★★★★

Incentives: ★

Distribution: ★★

The VSCode Extension Store is another strong case study, using an excellent plugin ecosystem (various Python plugins, auto-completion) to surpass internally optimized products like IDEA IntelliJ and PyCharm in developer experience. In VSCode's design, they made the IDE extremely extensible, allowing developers to easily build useful plugins for themselves, then open them to the community for collective use and refinement.

Revisiting ChatGPT Plugin

Plugins Killing LangChain?

Following the "App Store" narrative, another line of thinking holds that ChatGPT plugins eliminate the value proposition of LangChain, Fixie AI, and even Adept. This significantly overestimates what plugins can do.

Looking solely at the plugin strategy itself, its direct impact on LangChain today:

1. Some commercial pressure, since following the open-source project itself, LangChain's nearly only commercial path is building prebuilt hosted services;

2. LangChain and OpenAI are not competitors at all:

   • The open-source project implemented plugins under LangChain abstractions just two days after their launch;

   • It decoupled Chains and Agents from LangChain VectorStore and QA logic to better support future retrievers outside LangChain, similar to OpenAI Retrieval.

Developer feedback has also been quite mixed:

Independent developers who haven't previously invested effort learning LangChain abstractions but want to achieve model composability are largely building plugins through Replicate — conveniently calling other open-source model capabilities and hosting the plugin via Replit. The plugin below nicely combines ChatGPT, Stable Diffusion, and ControlNet:

Setting plugins aside, OpenAI's function calling feature launched in June actually has greater implications for LangChain and Fixie.

Plugin Roadmap and Ripple Effects

From an incremental perspective, ranked short-to-long term, here's how ChatGPT could drive change:

1. Microsoft's AI ecosystem strengthens further

OpenAI lacks reviewers, product managers, and experts who can operate this ecosystem. It lacks broader UI beyond chat interfaces with widgets. Microsoft has all of these — substantial personnel to deploy, a wide range of products for plugins to embed into and operate (Bing Chat and Windows 11 Copilot on the consumer side; Microsoft 365 Copilot and Dynamics 365 Copilot on the enterprise side).

Microsoft sharing ChatGPT's existing hundreds of plugins will massively accelerate its own ecosystem's cold start, enabling customers to build more plugins around internal private data. Microsoft has laid out a complete product and service narrative around this: Azure AI can provide the ability to run and test plugins on enterprise private data and cloud, while VSCode and GitHub can help enterprises build new plugins more effectively.

2. Model call optimization and a new API ecosystem

While a true advertising ecosystem like SEO or ASO (App Store Optimization) remains distant, early forms of model optimization have already emerged with ChatGPT plugins — intense competition is happening in the "Description for Model" field. Plugins describing themselves as "pet e-commerce" can get precise matching; this traffic won't be allocated to Shopify plugins with coarser descriptions, while more granular ones like "pet e-commerce built specifically for US users" can capture US-related traffic. Optimizing this Description has become a fascinating competition among plugin developers.

Beyond working the JSON file, another path is specifically optimizing one's API for ChatGPT. OpenAI's new Chat Completions API function calling feature in June can help developers achieve a "no-code" experience; traditional APIs have substantial room for model-centric refactoring.

3. Composability across multiple models creates more use cases

Similar to the plugin example we showed at the end of 05, ChatGPT plugins provide an easy-to-use UI for users to combine LLMs with other LLMs, external knowledge and APIs, and different models. Creative composability like this can produce many workflows and products that would be difficult to imagine without LLMs and Diffusion Models.

4. Plugin demands drive ChatGPT product evolution

This is self-evident — if OpenAI still harbors ambitions for ChatGPT to become an enduring great consumer product, and assembles the right team to achieve this. It very well may happen, as OpenAI just recruited product veteran Peter Deng from Facebook, Uber, and Airtable as VP of Consumer Product.

5. Chromebook-like hardware opportunities emerge

External expectations for OpenAI entering hardware are quite high. Google's Chrome strategy represents a decent example of hardware-software ecosystem coordination — Chrome Extensions dominate the Education category, birthing companies like Grammarly, largely because Chromebook hardware distributed the extension ecosystem to massive numbers of student and teacher users. Even without explosive hardware interaction innovation, establishing a unique user distribution channel through hardware could benefit the entire ChatGPT plugin ecosystem.

6. Internal and external model data accelerates AI Agent emergence and evolution

Thinking about plugin significance from an open platform perspective yields interesting conclusions — it's an important data interface for OpenAI to open to the outside world, enabling third-party data and user queries to interact, while Microsoft and Azure seem committed to driving more enterprise private data to interact with models. This will accelerate the birth of AI assistants that many already have high expectations for.