Starting with Cursor Going Viral: Why AI Coding Is a Massive Opportunity?

十字路口Crossing十字路口Crossing··0·0

💡 A quick note before we begin: Cursor has been everywhere lately. One Sunday morning, inspired by @Fanhan, I used Cursor to build an "Emoji AI Translator" in ten minutes flat — and that ten minutes included signing up for SiliconFlow, calling Llama through its API. Mind blown!

Throughout the whole process, I could strongly feel that Cursor was designed from Day 0 to be an AI-native IDE — the thinking and interaction are incredibly smooth.

Going forward, to become a great programmer, the most important skill probably won't be "the ability to keep learning" but rather "the ability to overcome your fear of code."

Greylock wrote an excellent overview on AI coding back in May. Thanks to Minghao Zhuang for sharing it, and "Crossing" for translating it for our readers.

AI Coding Is a Massive Opportunity

The premise: creating high-fidelity, reliable AI for code generation and engineering workflows.

Programming and engineering tasks are exceptionally well-suited for AI augmentation or replacement for several reasons:

  • (i) Coding inherently requires engineers to break problems down into smaller, more manageable tasks;
  • (ii) There is abundant existing training data;
  • (iii) Tasks demand a combination of judgment and rule-based work;
  • (iv) Solutions leverage composable modules (open-source libraries, etc.);
  • (v) In many cases, outputs can be empirically tested for correctness. This means reliable, accurate AI coding tools can deliver quantifiable value.

Given these factors, attempts at AI coding tools have exploded just in the past year. Yet many open questions remain about what technical hurdles must be cleared to build coding tools that match or exceed human engineers in production environments.

In this post, I'll outline three approaches we're seeing in the startup ecosystem, and three open challenges these tools face:

  1. How do we create more powerful context awareness?
  2. How do we make AI agents perform better on end-to-end coding tasks?
  3. Will owning code-specific models provide long-term differentiation?

Current Market Landscape

Over the past year, we've seen startups pursue three approaches:

  1. AI copilots and chat interfaces that work alongside engineers in their tools.
  2. AI agents capable of replacing engineering workflows by executing tasks end-to-end.
  3. Code-specific foundation models. These are custom models trained on code-specific data and vertically integrated with user-facing applications.

Even before answering these first-order questions, we believe each of the above approaches can deliver meaningful impact in the near term. Let's examine the landscape in detail:

1. Augmenting Existing Workflows

Today, the vast majority of AI coding startups take the form of IDE-integrated copilots or chat interfaces to augment engineering workflows.

While companies like Tabnine have been building coding assistants for years, the defining moment for AI coding tools came with GitHub Copilot's launch in 2021. Since then, we've seen an explosion of startups targeting every aspect of an engineer's work.

Startups finding product-market fit are targeting workflows around code generation or code testing. This is because:

  • They are central to engineers' work
  • They can be sufficiently useful with relatively limited context
  • In most cases, they can be bundled within a single platform
  • In a world where reliability is scarce, placing outputs directly in front of users (i.e., in the IDE) allows them to take responsibility for any needed corrections

The elephant in the room is the challenge of competing with GitHub Copilot, which already commands significant distribution and mindshare (congrats to Devin[1] on its recent Microsoft partnership). Startups are addressing this by finding differentiated entry points. For example, Codeium takes an enterprise-first approach, while Codium started with code testing and review and expanded from there.

We also believe there's significant opportunity for tools targeting code refactoring, code review, and software architecture. These may be more complex because they require not just broader understanding within code, but comprehension of knowledge graphs across different files, awareness of external libraries, understanding of business context, end-use patterns of software, and complex tool selection.

Regardless of entry point, one recurring challenge we see at this layer is how to acquire relevant context to address broader tasks across a company's codebase. Exactly how to do this is an open question we'll explore in the final section.

2. AI Coding Agents

If augmenting engineering workflows is valuable, the bigger opportunity lies in figuring out which workflows can be fully replaced.

AI coding products capable of executing engineering tasks end-to-end — working in the background while human engineers do other things — would unlock entirely new levels of productivity and innovation. Far beyond AI copilots, this could move us from selling tools to selling labor. In a world where coding agents become excellent, you could have one human supervising multiple "AI engineers."

The fundamental capability of AI agents goes beyond merely predicting the next token in a line of code. It requires combining that capability with the ability to execute complex tasks involving potentially dozens of steps, and thinking about products from the user's perspective like an engineer would. For example, if asked to fix a bug, it needs to know where the bug is, the nature of the problem, how it affects the product, any downstream changes the fix might cause, and much more before taking its first action. Context must come from sources like ingesting Jira tickets, larger chunks of codebase, and other information sources. The ability to write detailed code specifications and accurate code planning will be central to adopting AI engineers.

Companies and projects we're seeing in this space include (but are not limited to) Devin, Factory, CodeGen, SWE-Agent, OpenDevin, AutoCodeRover, Trunk, and others.

So the question is: what needs to happen for agents to complete a greater proportion of tasks end-to-end? This question is answered in my open questions section.

3. Code-Specific Foundation Model Companies

Some founders believe that to build lasting differentiation at the code application layer, you need to own a code-specific model underpinning it.

This isn't an unreasonable thesis, but there appear to be open questions that have kept other startups away from this capital-intensive approach — primarily, uncertainty about whether code-specific models will be leapfrogged by improvements at the foundation model layer. I'll discuss this topic further in the open questions section.

First, let's recall that most foundation LLMs are not specifically trained on code, and many existing code-specific models like CodeLlama and AlphaCode were created by taking an LLM base model, feeding it millions of publicly available code points, and fine-tuning for programming needs.

Note: Timeline shows selected code-specific models and LLMs widely used for coding use cases

Today, startups like Magic, Poolside, and Augment are trying to go further by generating their own code data and using human feedback on coding examples to train their own code-specific models (Poolside calls this "reinforcement learning from code execution feedback[2]"). The theory is that doing so will lead to better outputs, reduce dependence on GPT-4 or other LLMs, and ultimately create the most durable moat.

The core question here is whether a new team can outpace the improvement speed of frontier models. Foundation models are evolving so rapidly that if you try to go deep on code-specific models, you risk a better base model emerging and surpassing yours before your new model finishes training. Given the capital intensity of model training, there's substantial time and money at stake if you get this wrong.

I know some teams are taking the (very appealing) approach of doing code-specific fine-tuning on top of foundation models for specific tasks, which allows them to benefit from base model advances while improving performance on code tasks — I'll discuss this in detail in Open Question 3.

Open Questions

Regardless of approach, several technical challenges must be solved to unlock reliable code generation tools with low latency and good user experience:

  • How do we create more powerful context awareness?
  • How do we make AI agents perform better on end-to-end (E2E) coding tasks?
  • Does owning models and model infrastructure lead to long-term differentiated products?

Open Question 1: How Do We Create More Powerful Context Awareness?

The heart of the context problem is that certain coding tasks require information and context beyond the open file an engineer is working on, which cannot simply be accessed by increasing context window size.

Retrieving this information from different parts of the codebase (or externally) is not only challenging but can also increase latency, which is deadly in a world of instant autocomplete.

This creates a massive opportunity for startups that can accurately and safely find and ingest the context needed for coding tasks.

Currently, there are two approaches to doing this:

  • Continual fine-tuning: I've heard customers say "I wish a company could safely fine-tune their model on my codebase." While theoretically adjusting a model on your own codebase might make sense, in practice there's a catch: once you tune the model, it becomes static unless you're doing continual pre-training (which is expensive and may have the effect of perpetuating existing biases). Without this, it may perform well for a limited time, but it's not actually learning as the codebase evolves. That said, fine-tuning is becoming increasingly accessible, so periodically fine-tuning a model on your codebase may be feasible — for example, Codeium states[3] that they do offer "customer-specific fine-tuning," but they explicitly note it should be used cautiously, as the best approach is context-aware RAG.
  • Context-aware RAG: RAG is probably the best currently available method for improving context, by retrieving relevant snippets from the codebase. The challenge here is that retrieval ranking across very large codebases is non-trivial. Concepts like Agentic RAG and RAG fine-tuning are gaining popularity and may be powerful ways to better leverage context. For example, Codeium shared in a blog post[4] how they use textbook RAG supplemented with more sophisticated retrieval logic, crawling imports and directory structures, and incorporating user intent (like files you've opened in the past) as context. Being able to use this granular level of detail in retrieval could become a significant moat for startups.

Open Question 2: How Do We Make AI Agents Perform Better on End-to-End Coding Tasks?

While we're still some distance from perfect AI engineers, a handful of companies and projects like Cognition, Factory, Codegen, SWE-Agent, OpenDevin, and AutoCodeRover are making meaningful progress.

SWEBench evaluations show most foundation models can only fix 4%[5] of issues, SWE-Agent reaches 12%[6], Cognition reportedly hits 14%[7], and OpenDevin achieves up to 21%[8]. One interesting idea (reiterated by Andrej Karpathy [here[9]]) is around the concept of process engineering, which goes beyond single prompts or chain-of-thought prompting to focus on iterative code generation and testing. Admittedly, prompt engineering is a great way to improve performance without training models, but it's unclear how much of a moat this can provide for a company in the long term.

  • Note that this measurement has some limitations: for context, SWE-bench consists of GitHub issue and pull request pairs, so when models are tested on it, they only get a small slice of the code repository (which is a form of prompting that also introduces bias), rather than being given the entire repository and left to solve it themselves. Nevertheless, I think SWE-Bench is a good benchmark at this stage to begin understanding these agents.

Code planning will play a central role in AI agents, and I'm excited to see more companies focus on generating code specifications that can help agents build goals, plan features, and define their implementation and architecture. Multi-step agent reasoning remains a broadly unsolved problem, and by rumor this is a key focus area for OpenAI's next model. Indeed, some (like Jim Fan in [this post[10]]) would argue that the moat for AI coding agents doesn't actually come from "wrappers" but from the LLM itself and its "ability to solve real-world software engineering problems, with human-level tool access... searching StackOverflow, reading documentation, self-reflection, self-correction, and executing long-horizon consistent plans."

This brings us to our last — and possibly biggest — open question.

Open Question 3: Does Owning Models and Model Infrastructure Lead to Long-Term Differentiated Products?

The billion-dollar question is whether a startup should rely on existing models (whether calling GPT/Claude models directly or fine-tuning base models) or go through the capital-intensive process of building its own code-specific model — that is, pre-training a model specifically for code using high-quality coding data. We genuinely don't know whether a code-specific model will produce better results than the next generation of large language models.

This question boils down to several fundamental unknowns:

  • Can a smaller code model outperform a much larger foundation model?
  • How much does a model need to be pre-trained on code data to see significant improvement?
  • Is there enough high-quality code data available for training?
  • Do foundation models' large-scale reasoning capabilities trump everything?

Poolside, Magic, and Augment's hypothesis is that owning the underlying model and training it on code can significantly determine code generation quality. Given the competition, this potential advantage makes sense: to my knowledge, GitHub Copilot doesn't have a fully trained-from-scratch model, but rather runs a smaller, heavily code-fine-tuned GPT model. I suspect these companies aren't trying to build a foundation-scale model, but rather a smaller, more specialized one. Based on my conversations with people working in this emerging space, my conclusion is that we still don't know how much improvement this approach will yield until results are published.

One counterargument to the code model approach comes from the fact that existing successful coding assistants like Cursor and Devin are known to be built on top of GPT models, not code-specific models.

And reportedly[11], DBRX Instruct outperforms the specifically trained CodeLLaMA-70B. If training on coding data helps reasoning, then frontier models will certainly incorporate code execution feedback in future models, making them more suitable for code generation. Meanwhile, large models trained primarily on language may have sufficient contextual information for their reasoning capabilities to outweigh the need for code-specific data — after all, this is how humans work.

The key question here is whether the pace of foundation model improvement exceeds the performance gains of code-specific models over time. I suspect most assistant companies will likely start by taking frontier models and fine-tuning on their own data — for example, using Llama3-8b and doing reinforcement learning with code execution feedback on top of it — which allows companies to benefit from base model evolution while very efficiently biasing the model toward code performance.

Conclusion

Building AI tools for code generation and engineering workflows is one of the most exciting and valuable endeavors we see today.

The ability to elevate and eventually fully automate engineering work opens a developer tools market far larger than anything we've seen in history. While technical obstacles remain to be overcome, the upside in this market is unlimited.

We believe this space is large enough for many companies to develop specialized approaches across agents, assistants, and models.

Source: greylock[12]

References

[1]Devin: https://x.com/cognition_labs/status/1792988218750603764

[2]Reinforcement learning from code execution feedback: https://www.poolside.ai/

[3]States: https://codeium.com/blog/personalization-context-awareness-vs-customer-specific-finetuning

[4]Blog post: https://codeium.com/blog/personalization-context-awareness-vs-customer-specific-finetuning

[5]4%: https://swe-agent.com/

[6]12%: https://swe-agent.com/

[7]14%: https://www.cognition.ai/blog/swe-bench-technical-report

[8]21%: https://xwang.dev/blog/2024/opendevin-codeact-1.0-swebench/

[9]Here: https://x.com/karpathy/status/1748043513156272416

[10]This post: https://x.com/DrJimFan/status/1778105360685080644

[11]Reportedly: https://www.databricks.com/blog/introducing-dbrx-new-state-art-open-llm?utm_source=tldrai

[12]greylock: https://greylock.com/greymatter/code-smarter-not-harder/