AIGC "Peak Series" | Battle-Tested Insights and Pain Points Every AIGC Founder Needs to Know

The ChatGPT phenomenon has slowly opened up a space for AIGC entrepreneurship rich with imagination. As founders bring AI tools into their own business scenarios, what technical challenges have they encountered, what operational changes have they seen, and what lessons have they learned? These are the questions on everyone's mind as the AIGC wave moves from conceptual discussion to real-world implementation.

In the fourth installment of our AIGC series — "How Will AIGC Profoundly Impact Product, R&D, and Operations?" — we shared Li Lei, General Manager of Data Science and AI Solutions at Microsoft (China) Co., Ltd.'s, unique perspectives on "the underlying logic of ChatGPT" and "exploring the technical and commercial value of OpenAI" in our article AIGC "Peak Series" | With Global Players Racing to Build Large Models, Entrepreneurs Can Consider These Three Opportunities. Today, we bring you the second major piece from a practical and results-oriented angle — "AIGC Practice and Reflections from the Perspectives of an Enterprise R&D Management Platform and a Game AI Platform" — as shared by Feng Bin, Co-founder & CTO of ONES, and David Zhang, Founder & CEO of game AI platform GamesMind.

What role will AI ultimately play in this world dominated by humans? Can AI become a highly effective assistant? Their shares may offer us some answers.

Selected highlights from the speakers:

I'm glad to be here today to share some of our ideas and practices at work. ONES primarily helps software teams with collaboration and project management across product, design, and adjacent teams. ONES is essentially a software company with multiple R&D management products. In 2020, we acquired Tower, a team collaboration tool, to cover multi-scenario project management for teams of all sizes.

The rapid iteration and development of AIGC over the past six months has been evident to all. This disruptive technological transformation is bound to have a major impact on industries across the board, and ONES is actively exploring both in our daily work and product planning. Today, I'll share some practical cases from our team.

Documentation & Writing

Documentation and writing is one of the more fully developed business scenarios where ONES has implemented AIGC practices. Let's start with translation. While ONES has reached top position in domestic software project management, we're also expanding into overseas markets. This involves a substantial amount of copy translation work. Previously, we would typically have colleagues with relevant language skills collaborate with outsourced translation companies. But compared to what ChatGPT produces, I always felt their translations weren't quite natural enough. Currently, we've basically reached an internal decision: from a cost and time perspective, translation work will be replaced by AIGC in the short term.

Moving on to document collaboration: ONES offers not just project management functionality but also knowledge base management. GPT integrated with documents is now the most typical use case — generating summaries from documents, creating outlines, expanding content. These features are basically ready for workplace application. We've also found that highly structured content like weekly reports and proposals is something GPT handles very well. We're also experimenting with connecting our entire customer service and technical support system to ChatGPT.

ONES is a SaaS product with both standard offerings and a developer platform. Both users and service providers can do secondary development based on ONES. We call this the ONES Open Platform. Whether in China or globally, large teams more or less need to customize their software according to their specific situations. Open capabilities can serve as the boundary between custom functionality and standard products, enabling truly efficient service for large teams.

ONES has rich open capabilities, which also requires extensive developer documentation as user manuals. We've tried using GPT to provide a Q&A format that lets users more quickly extract desired information from large volumes of documentation. Like many companies, we face a situation in this scenario where documentation quality and completeness matter. If we want this conversational process to go further — answering questions about ONES open capabilities with corresponding code — that also requires feeding GPT large amounts of our previous relevant code. This approach has certain requirements for both the quantity and quality of code (naming, comments, etc.), and we're still following up on this work.

Code Writing

The second part: let's talk about coding. When GitHub Copilot's enterprise edition launched, our entire company basically started using it right away. Our feeling is that Copilot is an enhanced version of code completion. It might improve people's mood and offers some efficiency gains for formalistic code, but it doesn't bring more fundamental differences. We also have colleagues researching AutoGPT, focusing more on how it handles task decomposition.

For simple, repetitive, easily outsourced content, GPT works relatively well, including previous Copilot plugins — these are slightly better solutions. But for complex scenarios like "consistency issues in distributed environments," given insufficient current training data and scenarios, human work is still faster.

Regarding unit test case writing — this is another scenario we're currently implementing company-wide. Modern programming platforms have their own automated testing frameworks. GPT can quickly generate unit test cases, i.e., unit inputs and expected outputs. When I give GPT a function's code and comments, it can help us generate the complete relevant test cases covering all output boundaries. Before GPT, many large companies had tools that generated test cases based on function input parameter types and descriptions — these tools would automatically exhaust all boundary cases from input parameters. Now GPT can directly accomplish this.

Currently, roughly 70-80% can be tested this way, saving considerable time. If coverage is sufficient, it actually helps our quality too.

UI & Visual Design

I often discuss with design colleagues whether there's opportunity to make their work more efficient too. ONES mainly uses Stable Diffusion. Prompts largely determine the efficiency of generating usable images, and designers may not be familiar with professional English — here translation becomes an issue, but we can have GPT solve this. Let me share how we combine ChatGPT and Stable Diffusion.

Our colleague referenced a prompt template from online, organized the requirements, then used GPT to generate corresponding English prompts, using these English prompts as input for Stable Diffusion to generate images. For example, our company logo means "nautilus with gear structure." We gave ChatGPT a description of the logo, but the results were still quite far off. We asked if it could use a minimalist style? Finally it gave a new prompt, and when we put this new prompt in, it looked more ideal.

The greater insight here is: often we think things are hard to accomplish because we don't know how to interact with them. With ChatGPT's help, we can ultimately use natural language to interact with these tools.

Another insight: previously learning was one-directional; with GPT, it becomes conversational. For example, when we don't know what design styles currently exist, we can find answers through follow-up questions — equivalent to having an extra erudite teacher. This method is especially useful for helping us understand foundational knowledge in a field.

Some time ago, we used Stable Diffusion to generate a series of icons. Previously this might have taken one to two months; now it's basically done in half a day. How did the process work? Using the fine-tuning plugin LoRA, we fine-tuned based on a large model — adjusting style, adjusting characters, including combined details, etc. Fine-tuning is essentially doing a small round of "alchemy," doable on your own computer or using cloud services.

We have some internal design materials that were previously all hand-drawn. We extracted many elements from them as the most basic first batch of training materials. First we found a large model on Civitai that matched our needs, and used these 10 images to generate a LoRA model. The problem with this approach was that the style was too fixed (overfitting). So we first used the first-generation images to generate second-generation images through img2img, then selected satisfactory ones from the second generation, combined them, and retrained the LoRA model. Finally, when we asked it to generate things like pencils or puppies, it could draw in this style. Then we used some tools for simple cutout, and it became an icon library. The entire process took only 3 hours; if colleagues drew it themselves, it would take at least two to three weeks. These are some of the more grounded things we've tried over the past three months.

Finally, let me summarize a few of my views on the AIGC field:

1. Context greatly affects ChatGPT's output quality. Multi-turn dialogue is the norm; I recommend using the 5W2H structure to construct prompts.

2. In the future, a work paradigm of AIGC "countering" AIGC will inevitably emerge.

3. For image generation scenarios, two advantageous use cases: fast-food image needs; design concept brainstorming. The more refined the scenario, the higher the post-production adjustment costs.

4. Conversational learning will become widespread.

5. Setting goals, information gathering, breaking down execution, and doing retrospectives are the basic steps of human decision-making. AI excels at the latter three, but the first one will most likely not be delegated to AI, because it has no authority and cannot be held accountable. So we won't give it something like human rights. Without authority there is no responsibility; ultimately AI will become an assistant for human decision-making. That's all for today's share. If we meet again in a month, there may be many new and different things.

Hello everyone, I'm David Zhang, Founder & CEO of GamesMind. I previously worked as a researcher at Microsoft Research Asia, and later founded GamesMind. Today, I'll share from the perspective of game AIGC — the industry's latest developments and our own practical cases.

Content Generation Pain Points in the Gaming Industry

What pain points does the gaming industry face at the large model and AIGC level? The global gaming market is around $200 billion, but has barely grown in recent years. The entire industry is competing on story and art. Although R&D costs are rising year by year, revenue and market size haven't changed much. From what we understand, art accounts for roughly half of R&D expenses in the gaming industry. Whether from research reports or general consensus, gaming is the sector most impacted by AIGC — text corresponding to quests, storylines, intelligent NPCs; images corresponding to concept art, character illustrations, assets, etc. — all are prime application scenarios for AIGC.

Every industry is thinking about what bottlenecks they'll encounter when introducing AIGC. Taking game developers as an example: first, high barriers to external integration. Currently basically all game developers have used Midjourney or Stable Diffusion, but their feedback is that while the results look exquisite, there's still a large gap from actual production use. Second, low internal R&D efficiency. Game developers generally report low efficiency in building their own AI R&D teams; they still prefer partnering with a professional third-party company.

GamesMind was founded in 2020. As early as 2020, we released what was then the world's first generative commentary for MOBA games, and we created the professional-grade game content production platform GamesMind AI Creator quite early. While AIGC has only recently become a buzzword, the underlying technology has existed for quite some time.

AIGC Application Practice in Gaming Scenarios

In current gaming scenarios, how do we carry out cutting-edge AIGC work? Game developers have told us about a pain point: Midjourney generates beautiful images, but most people won't use them directly, because most developers pursue their own style, effects, or IP. They feel tools like Midjourney simply aren't usable — no matter how much they refine their prompts, they can't get the desired effects.

Previously, when we made a style model for clients, we typically needed the developer to provide large amounts of data. However, most developers couldn't provide sufficient data, and couldn't accept overly long training wait times. Now in the image generation AIGC field, we adopt a large model + small model approach — building a base model for the gaming vertical at the bottom layer, with small models training additional attachment layers, and then controlling and training style on these attachment layers.

In this regard, every industry differs. Why does gaming need a vertical base model? If you've tried some general generation platforms, you'll find their coverage of concepts and forms in the gaming domain is insufficient. Considering specific industry characteristics, we chose to develop our own base model for gaming. After we train a small model for a client, combining it with the base model ensures we can generate the desired high-quality effects they want, at acceptable costs.

Character script generation corresponds to narrative design work in games. A fundamental issue with general large models is relatively limited Chinese-language training data; the gaming domain is even narrower. We have years of accumulation in NLP, including our early-released generative commentary model, so for gaming we adopted a self-developed gaming vertical model. Everyone can evaluate based on actual scenarios in their own industry — use large models if suitable, otherwise consider self-development or partnership. For story-heavy games like MMOs, storytelling games, and visual novels, models can generate plots and branches, and with personalization technology can even achieve dynamic plots unique to each player. And in large-scale games like metaverse experiences, when facing thousands of NPCs needed, we can generate NPC content fitting the game background, enriching game settings and interactive experiences.

In the art pipeline, game developers face various pain points when using general platforms or open-source software. For example, after generating a character in Midjourney that looks decent, we want to change the eyes to green. The instinctive reaction is to add "green eyes" to the original prompt, but regenerating gives a completely new image. This new image might have green eyes, but everything else has changed too. In actual scenarios, users don't expect perfect images on the first try — they expect to meet requirements through several rounds of modification and fine-tuning.

You might wonder: why not just download the image and modify it in Photoshop? The fact is, model-generated images don't come with layer information, and no artist is willing to modify images without layers. In this situation, we provide clients with mask operations similar to Adobe's — they can select or paint over parts of generated images, then do localized generation for that portion. During generation, it calculates based on surrounding context; after several rounds of modification it basically achieves what the client wants.

Additionally, the prompt-based interaction method has many limitations. In gaming, for example, artists have professional knowledge, techniques, and ideas that can't be incorporated into the generation process through prompts alone. This is like chatbots — without connecting to knowledge graphs and knowledge bases, it's just a casual chatbot. Only after connecting to corresponding knowledge graphs or knowledge bases does it become productive. Based on the same thinking, it's the same for images: we make artists' professional knowledge into a knowledge module, effectively integrating their expertise into the generation process.

For structural control, the same approach applies. For example, when generating an image, we laypeople might think it looks great, but artists can spot issues with lighting, depth of field, pose, composition, etc., and naturally won't be satisfied. Here too we can introduce additional structural control — before final image generation, doing structural presetting through attachment layers, then having the art director review to meet their expected image quality.

There are many more pain points like these in gaming. Everyone can reference this thinking in combination with their own industry.

Every industry can essentially use large model + small model, or large model + fine-tuning, or prompt-based approaches. In gaming specifically, we've also made corresponding modifications on the input side. For games, from art director sketches to base art completing effect images, to producer feedback — even just the initial art selection process takes 1-3 months. Our solution: after the art director finishes sketch or line art, AI handles the rest, dramatically reducing initial art selection time. Some clients always feel they can't generate what they want through prompts, so they draw part of the effect themselves, then use GamesMind's AI to further generate the rest. This same approach, I believe, has broad application space across all industries.

Our client types include 2D, 2.5D, and 3D, corresponding to different game types. We can basically help them reduce art pipeline time by 70% to 90%. Taking a typical 2.5D SLG game as an example: previously artists needed 15 workdays to complete assets; after partnering with us, it's basically done in half a day. Based on this advantage, clients have widely applied AIGC technology to their skins, event prizes, and other scenarios. Another example: a war-themed game where tanks from different countries or factions aren't very different. If drawn separately for each, it would be extremely labor-intensive. Our solution: draw one base version, then use AI to quickly generate variants combining characteristics of each country or faction.

Everything discussed above is about the art pipeline during game production. Beyond this, every industry also has marketing, promotion, and advertising needs. With our current technology, we can similarly fulfill advertising images, posters, and other needs. For video, our self-developed model is two orders of magnitude faster than comparable models, enabling rapid and efficient fulfillment of clients' image and video generation needs.

Next, let me share some specific cases. For example, finding human models for games in overseas markets is quite cumbersome, and if new scenes are needed later, that person might no longer be available. There are also copyright requirements, etc. Through our model configuration, we can generate without violating portrait rights, copyrights, etc. For some in-game assets, rushed work or insufficient skill occasionally occurs. When we encounter issues with color, gloss, or details, we can also leverage AI for assisted generation. And browser game developers previously had large amounts of assets, but these were far inferior to mobile and PC games. If unwilling to spend money on redrawing, style transfer and detail redrawing needs can be efficiently completed with AI.

To mention a deeper need: for example, creating a completely new character typically takes far longer than creating a building. Here AI can help — according to the client's ideas, needs, and inspiration sources, generating dozens or even hundreds of options, selecting preferred results for further detail optimization, thereby dramatically shortening character creation time.

That's all for my share today. Hope it's helpful to everyone!