OpenAI recently released an API for its ChatGPT and Whisper neural networks. This will allow developers to directly integrate these programs into their applications, work tools, and even games. AI enthusiasts have already held the first hackathon, where developers tested ideas on using the ChatGPT API.
The entire gaming industry is following the development of neural networks. Some see them as new opportunities, others as a threat to developers. What kind of generative AI is already being used in game development and why? What do developers think about this technology? What are the bosses of game companies expecting? We have tried to answer your questions in this review.
This is not a text about the history of neural networks. There will be no detailed explanation of how they work, only examples, opinions and forecasts of industry professionals.
And no, in this material there is not a single paragraph written by a neural network. These jokes are no longer funny in 2021.
But you can’t resist publishing pictures created by a neural network.
Starring
-
ChatGPT, Bloom – these neural networks can create text or code. All you need to do is enter a description of the task. Useful in creating a script, in-game texts, dialogues and, of course, programming.
-
DALL·E 2, Midjourney, Stable Diffusion – help you create a 2D image or edit an already finished one. In game development, they are used for concept art. But you can also work with textures – change or improve them.
-
Riffusion, Jukebox, VALL-E, AIVA will help with sound. The neural network can write music, including vocals, voice character dialogues or audio diaries. And Whisper technology is able to recognize speech in different languages.
-
Dream Fusion, Point-E – trained to work with 3D models. You can create assets from a simple sketch in a 2D editor or generate them through text commands.
They are already among us
Players have come across neural network products before. The world in Minecraft, the planets in No Man’s Sky, the inhabitants of London in Watch Dogs: Legion are all the result of procedural generation. And this is not to mention the internal tools of developers – take at least algorithms for generating buildings for large cities. Below are examples of the work of new AI, clearly showing the scope.
voice acting
The developers of the shooter High on Life openly stated that they used AI to create a prototype voice acting for their characters. Later, the characters were finalized, and professional actors gave them their voices. But at least one minor character speaks in synthesized words even after release.
Domestic indie developers Will & Reason Team are preparing a 4X-strategy with many quests. They also used the help of neural networks to voice dialogues and tasks in the game. In the video below you can hear how it works. The authors decided to combine neural networks and human voices, so a number of the main characters will be voiced by actors, and the rest by AI.
Scenario
Since ChatGPT has become much smarter only relatively recently, we have hardly seen a game where the script or dialogues are written by a neural network. But do developers use this tool in their current projects? Yes, for example, several screenwriters on LinkedIn admitted this.
They don’t just copy the results of neural network “creativity” into their game. One developer said that ChatGPT helps him overcome his writer’s block, to see which parts of his story are too banal and need to be worked on. Another writer uses a neural network to create dialogues. By adjusting the context of the situation and the characteristics of the characters, he gets a lot of options for remarks with different tone and figures of speech that look authentic.
You can clearly see the capabilities of the neural network storyteller in AI Dungeon 2, but this is still a game, not a development tool. In this article, you can see step by step how the author creates lore, plot, characters, items, description and other in-game texts. The neural network understands the context and can even save the user from logical inaccuracies. But as the author of the article notes, ChatGPT abuses the borrowing of names and names from other universes and gives not too original plot twists. Without a talented editor, a neural network can create at most something of the Forspoken level.
Code and level design
ChatGPT became famous for its ability to write code. On the Web, you can find many videos with the title “The neural network created a game for me.” However, they all boil down to showing primitive side-scrollers. It would have been possible to completely replace a human developer with such a neural network in the 80s of the last century, but now the code for games is much more complicated.
You can also use generative AI to create levels, just like the creators of MarioGPT did. Their system is trained to interact with the classic platformer Super Mario Bros. – just enter a text description, and the neural network will create a level that can be completed.
A typical video about coding a game on ChatGPT.
3D modeling and design
A new plug-in for the Blender editor should significantly simplify the work for 3D modelers and artists. It allows you to generate textures for models without interrupting your work through the Stable Diffusion neural network. You just need to enter a text description, and then the AI will create a picture of a suitable size.
It’s also worth mentioning High on Life again. In this shooter, neural networks took part not only in the voice acting, but also in the design of the game. The authors did not specify what exactly was generated and what was created by people. The developers used a neural network to achieve the effect of an alternative world and make the “finishing touches”. Reddit users believe that there are definitely traces of generative AI in the posters that can be found in the game.
Testing
Algorithms have penetrated deepest into the work of QA engineers. Testing has been using bug bots for several years now. In many ways, the task of a tester in a large team comes down not to writing reports, but to automating this process with the help of AI.
The main benefit of learning algorithms is not that they can run through a level 20 times faster than a human. The neural network can be trained in different scenarios of behavior: to play as a beginner or an esportsman, as a passive or active user. Such testing gives better results.
This video is almost three years old. AI testers can now perform the same operations even faster.
A Word to the Developers
Of course, all examples of using neural networks cannot fit in one article. But it is already clear that AI will be introduced into development more and more deeply. Not all developers are happy about this – some say that AI will allow indie teams to create projects of a higher level, while others believe that neural networks will deprive many employees of their jobs.
Shahid Ahmad, the author of the Code Is Just project, compared the use of AI with the use of industrial machines in production:
Modern equipment has removed a lot of routine manual work from our lives. Robots also help with this. However, video game studios that produce content on a large scale have not yet benefited from total mechanization.
Yes, at first someone may be uncomfortable with the news about process automation, which many consider art. But most of the assets created in video games are craft. And if humanity was able to mechanize other crafts, then why can’t this be done with respect to digital work?
Shahid Ahmad
Philip Oliver, co-founder of Panivox and experienced developer, shares a similar opinion. In his opinion, the appearance of AI in the work of developers will be able to increase their efficiency in the same way that tractors, which replaced plows, did in agriculture. And in both examples, people remain in key positions. No matter how smart the AI is, it will not be able to create a single line of dialogue or a pixel of an image on its own.
Treat ChatGPT and Midjounrey as problem solvers, process accelerators, and creative assistants. On their own, they will never replace developers. Our tools do not work for us. The same applies to generative AI.
Philip Oliver
Developers from Proxy Studios used neural networks to create concept art for their upcoming 4X strategy Zephon. Rok Breulj sees technology as both a great future and potential problems. AI is helping to streamline the content creation process that has taken developers more and more time every year. Together with neural networks, you can create larger worlds, larger games.
We should not be afraid of generative AI. Give it a try, experiment with it, explore its limitations and see if it can improve your workflow. The market is filled with games created from ready-made assets, and we can expect that soon it will be flooded with projects that use only neural networks. More thoughtful content will have a harder time getting through to players. Therefore, companies will have to increase their marketing budgets.
Rock Breoul
Because of the neural networks, the industry is waiting for fundamental shocks and shifts, Imre Jele, co-founder of Bossa Studios, is sure. As in any industry, these changes will entail inevitable losses. People and companies will be forced to adapt to new realities, but this will not happen instantly, the developer believes. There remain legal issues of using the results of AI work.
Neural networks in most cases will accelerate the work of existing specialists. However, there are serious and legitimate concerns about job cuts. One of my subordinates believes that generative AI will eliminate the need for low-skilled employees in the art departments. This means that it will be more difficult for young workers to find their first job and they will not be able to gain the experience necessary to improve their skills in the industry. This will reduce the number of true professionals in the market in the long run.
Imre Jehle
The developer added that progress should not be held back. The best thing people can do is to carefully consider their steps and prepare for the inevitable consequences that will occur in game development. At the same time, Jele is looking forward to seeing the projects that will become possible thanks to neural networks. It is possible that generative AI can be built into the games themselves in order to create content focusing on each user.
top view
The bosses of large companies are still reluctant to speak out about the use of neural networks in the gaming industry. The head of Take-Two, Strauss Zelnick, sees no threat in these technologies. In his opinion, generative AI will allow developers to become more creative. These tools will help cut costs, but will not fundamentally change the video game industry and the process of making video games.
When the calculator first appeared, parents became worried: do children really need to learn math more? Need to. The calculator is just a tool. And the same should be true of neural networks like ChatGPT. I am in awe of what these technologies show. In the end, their implementation will simply raise the bar even higher. Our teams will want to make their projects even bigger, even better. And no one can ever just ask the AI, “Please develop a Grand Theft Auto competitor that is better than Grand Theft Auto,” and then sell the result. It is likely that someone will try, but they will not succeed.
Strauss Zelnik
Meanwhile, Microsoft has already announced that they will implement neural networks in all their future products. This was told by the CEO of the company Satya Nadella (Satya Nadella). Although there were no specific examples from game development in his speech, it is obvious that this part of the corporation’s business will not be left aside. Microsoft has invested a lot of money in the development of OpenAI, the company behind ChatGPT.
We will include AI in our own applications, so you can be sure that we will use neural networks. Every Microsoft product will have generative AI capabilities.
Satya Nadella
Recently, the video game developer conference DICE Summit 2023 was held in Las Vegas. It confirmed that the topic of generative AI is being discussed by all developers. Few people are already interested in the blockchain, which someone predicted a great future in video games. Even members of the God of War: Ragnarök team were optimistic about AI. The only thing that keeps authors from making full use of most of the tools is copyright ambiguities, which are gradually being eliminated.
