New York University's Alex Sheng: Devin the AI Programmer's Path to Self-Iteration | Agent Insights

Alex Sheng has been studying machine learning since high school and now has seven years of independent research experience, with backing from Google and IBM. Inspired by LSTM pioneer Jürgen Schmidhuber, he developed a self-improving computer agent. While the Self-Improving Computer Agent still faces certain limitations, technological progress often moves from imagination toward reality. Enjoy.

"From Language Models to Practical Self-Improving Computer Agents"

An Algorithm for Self-Improving Computer Agents

Paper link: https://arxiv.org/pdf/2404.11964

Problem addressed: Traditional agent enhancement methods include RAG retrieval, CoT reasoning, WebGPT internet language augmentation, program evaluation, and ReAct LLM augmentation. However, these approaches lack effective real-time adjustment of generated content during the process, preventing agents from achieving self-iteration. The team developed the Self-Improving Computer Agent, enabling computer agents to self-augment and refine — continuously strengthening their capabilities through generation, identifying and resolving their own problems, and achieving automated command execution for computer tasks.

Model architecture: The framework's front end connects to a large model API, with a Prompt Loop added at the middle layer to continuously converge user input for iterative optimization. The process re-parses instructions and generated content, feeding them back into the Prompt Loop to form a closed loop. The outermost layer deploys the entire framework onto a computer system for engineering implementation.

Performance: The team's self-improving computer agent offers unlimited extensibility and out-of-the-box usability. It achieves retrieval augmentation through suggested file creation and editing tool functionalities, incorporates human collaboration instructions, and completes flexible deployment for computer task execution. Compared to other computer agents, it demonstrates stronger editability, creativity, and flexibility.

Application space: Computer agents

The human collaboration process within the Prompt Loop

Oasis Capital: Could you briefly introduce your research background and your understanding of agents?

Alex: I have seven years of independent research experience in AI/ML. I initially focused on few-shot learning in machine learning — enhancing learning capabilities with limited data. Before 2020, neural network training required massive amounts of training data, and I was already searching for ways to iterate and optimize few-shot algorithms. Google's TPU Research Cloud Program funded my research for several years; Google provided researchers with sufficient TPU access to complete research papers. Later, after joining NYU, I began collaborating with IBM on other papers.

Alex's zero-shot paper in collaboration with IBM Research AI

Before that, I also published a large model-related paper titled Self-Programming Artificial Intelligence Using Code-Generating Language Models. In that paper, our team explained how to apply zero-shot techniques in large models, using code-generating models to adjust the code they produced. In this latest paper, we explain how to build a comprehensive LLM-based agent system that directly generates and executes code within an OS. Compared to pure AI agent researchers, my background is more interdisciplinary — for instance, I focus more on how to use code to enable agents to learn self-repair and self-improvement. I'm also closely attuned to developments in prompt engineering. These interests led me to develop a computer agent with self-improving capabilities at the code level.

Oasis Capital: You started researching machine learning in high school?

Alex: I've always been deeply interested in technology. Actually, the trend of AI transforming the world was already quite obvious seven years ago, so I chose this exciting industry. The entire AI field is remarkably open — most research and papers are open-source, and non-undergraduate and non-PhD students can easily self-study. I went from learning about the latest research, to participating in some research, to eventually completing full AI system development on my own — the whole progression felt very natural. Additionally, getting computing resources from Google helped me tremendously. Google is the company that developed the Transformer, and their understanding of machine learning fundamentals runs deep.

Oasis Capital: How did Google fund your machine learning research?

Alex: I participated in Google's TRC Program (TPU Research Cloud Program). TPUs are Google's proprietary accelerators designed specifically for machine learning. Unlike GPUs, Google never sells TPU devices publicly — they're fully integrated into Google Cloud as a cloud computing service with mounted TPUs. TPUs pack serious computing power; the latest TPU v3-8 is equivalent to eight GPUs. TRC members get free access to a cluster of over 1,000 cloud TPU devices, using frameworks including TensorFlow, PyTorch, Julia, and JAX. I became one of the earliest Google TPU users. My earliest papers were trained on Google TPUs, using supercomputers for distributed evolutionary optimization algorithms applied to distributed trading models and similar extensions — scaling algorithms to large clusters and distributing them.

Alex's paper published through the Google AI TRC Program

Oasis Capital: After your first paper, what motivated you to pursue the second paper and develop the Self-Improving Agent?

Alex: My earliest inspiration came from the renowned German computer scientist Jürgen Schmidhuber. His Gödel machine model suggests that to achieve ultimate intelligence, humanity needs AI systems capable of self-evolution. In the long term, I hope to see agents that assist researchers in their work and make humans smarter. So in the near term, I chose to develop agents capable of self-evolution in the code domain. The reason for choosing code is that it's a highly modifiable form of creation during the creative process itself, and code is deployed within computer OS systems. Once an agent learns to self-iterate, it can very naturally adjust its own generation process, and even potentially iterate out a new AI system — just as Jürgen Schmidhuber described.

Oasis Capital: Could you briefly introduce the basic structure of the Self-Improving Agent?

Alex: The overall agent structure is quite simple. The back end connects to an API similar to GPT-4, the middle layer incorporates our designed Prompt Loop, and the outermost layer deploys the entire framework onto a computer system with some engineering work. The Prompt Loop continuously converges user input for iteration and optimization, generating code and terminal commands. All actions ultimately execute within the computer environment, with the entire process being re-parsed and fed back into the Prompt Loop to form a closed loop.

After the initialization instruction is issued, the agent generates and runs code blocks in the computer, which the Prompt Loop parses and saves to files or other accessible locations in the computer environment. The large model then generates terminal commands to copy that code, runs it in Python, and ultimately outputs execution commands. Prompts and the Prompt Loop are crucial throughout this process.

Oasis Capital: Does this framework work on computer systems other than Windows OS?

Alex: Yes, it's fully applicable on macOS and Linux as well. The framework's final output is terminal commands, so you can see it's very easy to deploy and run on other computer systems. I personally use Windows, so the paper discusses Windows more frequently.

Oasis Capital: What are the current limitations of the Self-Improving Agent, and how will we improve them in the future?

Alex: There are three major limitations currently. First, the Self-Improving Agent still relies heavily on the underlying model's capabilities. Second, its self-improvement ability remains highly constrained by prompt quality — the Prompt Loop involves extensive prompt interactions, and sometimes complex automated tasks require human assistance. Third, safety: code generated by large models may still contain security vulnerabilities; it can access anything you don't want it to access, and even cause damage. In the long term, improving the safety of large model-generated content can be achieved through alignment. Additionally, many researchers worldwide are continuously contributing to AI safety.

Oasis Capital: In what scenarios can computer Self-Improving Agents be applied?

Alex: I believe the Self-Improving Agent provides an excellent low-code runtime framework, enabling creators to use natural language to modify agents and promote their self-iteration. Once mature, it could even generate computer instructions that previously didn't exist in computing environments.

For example, with the recent buzz around Devin, our agent framework can achieve effects similar to SWE Agent. I'm also experimenting with having Self-Improving Agents generate their own tools to help users search the internet. I've previously tested having an agent conduct literature reviews based on research topics. The advantage lies in its modifiability — researchers no longer need to use other specific agents, but can develop any type of agent through its self-improvement capabilities. In the long term, I expect it to serve developers with some technical background as a copilot agent.

Oasis Capital: Do you believe in AGI? If so, how long do you think it will take to fully achieve the AGI you envision?

Alex: I completely believe in AGI's arrival, and I think within about five years we'll see something like JARVIS from the Marvel movies become reality. My current research motivation is to develop an agent that can accomplish any task you tell it to, self-improving to solve tasks it initially couldn't handle. Additionally, underlying large models are growing more powerful every year — we've seen GPT-4 demonstrate extraordinary code generation capabilities. We'll gradually see improvements in LLM and agent capabilities driven by attention mechanisms and scaling laws. So in the near term, there's a real opportunity to see ordinary agents accomplish increasingly complex real-world tasks.

Oasis Capital: What does AGI mean to you?

Alex: I think AGI is difficult to define. I've discussed this with many other researchers, and everyone ultimately arrives at different opinions. But actually, if we had a time machine, we've already achieved AGI to some degree — because seven years ago, none of us could have imagined applications like ChatGPT. Seven years ago we were doing foundational research, back before GPT, when we were still using RNNs and LSTMs. We completely couldn't foresee the later releases of GPT-2, 3, 4 and the development of attention mechanisms. These are what truly motivated me to continue researching along the AGI path.

So today is the same — it's hard to define what AGI will ultimately become. What I'm working on now is addressing the underlying capabilities AGI requires: self-improvement, safety, value alignment, and so on — contributing valuable, meaningful, and forward-looking foundational frameworks to the entire AGI process.